1. 程式人生 > 實用技巧 >kafka CommitFailedException rebalanced poll

kafka CommitFailedException rebalanced poll

1、異常詳情

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

2、原因

kafka等待消費者返回消費訊息成功與否的響應時間超時。

3、解決方法

配置引數:session.timeout.ms:是消費者心跳響應的週期,若在這段時間內,沒心跳響應,kafka的broker就會移除這個consumer然後rebalance。max.poll.records的大小應該置為一個合理的範圍,最好可以在session.timeout.ms的範圍內處理完所有的請求。

The timeout used to detect consumer failures when using Kafka's group management facility. The consumer sends periodic heartbeats to indicate its liveness to the broker. If no heartbeats are received by the broker before the expiration of this session timeout, then the broker will remove this consumer from the group and initiate a rebalance. Note that the value must be in the allowable range as configured in the broker configuration bygroup.min.session.timeout.ms

andgroup.max.session.timeout.ms.