技術標籤：大資料kafkaflumekafka大資料flumehadoop

flume讀取kafka訊息並存儲到HDFS上出現異常Attempt to heart beat failed since member id is not valid, reset it and try to re-jo

1 報錯情景:

flume根據kafka不同的topic進行收集資訊,但是啟動flume發現flume日誌報錯: Attempt to heart beat failed since member id is not valid, reset it and try to re-jo

在這裡插入圖片描述：

2 原因:

由於f1,f2監聽kafak時，需要接收kafka訂閱的topic裡面的資訊,所以需要配置group.id,則f1,f2預設groupId都是"flume"
,f1,f2屬於同一個消費者組(consumerGroup),在我們這裡是訂閱不同的topic，所以需要指定groupId為不同的topic。

3 解決辦法:

為兩個flume配置不同的group.id即可:

a1.sources.r1.type=org.apache.flume.source.kafka.KafkaSource
a1.sources.r1.kafka.bootstrap.servers=
 
Hadoop102:9092，Hadoop103:9092，Hadoop104:9092
a1.sources.r1.kafka.topics=topic_start
a1.sources.r1.kafka.consumer.group.id=group_start

a2.sources.r2.type=org.apache.flume.source.kafka.KafkaSource
a2.sources.r2.kafka.bootstrap.servers=Hadoop102:9092，Hadoop103:9092，Hadoop104:9092
a2.sources.r2.kafka.topics=topic_event
a2.
 
sources.r2.kafka.consumer.group.id=group_event