1. 程式人生 > >kafka offset 設置

kafka offset 設置

ssa offset fse you cts nbsp code pri over

http://stackoverflow.com/questions/36579815/kafka-python-how-do-i-commit-a-partition
from kafka import KafkaConsumer
from kafka import TopicPartition from kafka.structs import OffsetAndMetadata ... topic = ‘your_topic‘ partition = 0 tp = TopicPartition(topic,partition) kafkaConsumer = KafkaConsumer(config here...) kafkaConsumer.assign([tp]) offset = 15394125 kafkaConsumer.commit({ tp: OffsetAndMetadata(offset, None) })



meta = consumer.partitions_for_topic(topic)
options = {}
options[partition] = OffsetAndMetadata(message.offset, meta)
consumer.commit(options)

如果consumer.commit()不可以,可以使用seek(),使用seek(),如果有多個partition,需要為每個partition都手動進行consumer assign.
topic_partition = TopicPartition("TOPIC_TEST", 1)
# 格式為topic, partition, 1表示partition 1.
consumer.assign([topic_partition])

consumer.seek(topic_partition, 1660000)
同樣的代碼,有些用commit可以,有些不可以,用seek()就可以。
 

kafka offset 設置