【Azure 事件中心】開啟 Apache Flink 製造者 Producer 示例程式碼中的日誌輸出 (連線 Azure Event Hub Kafka 終結點)
問題描述
Azure Event Hub 在標準版以上就預設啟用的Kafka終結點,所以可以通過Apache Kafka協議連線到Event Hub進行訊息的生產和消費。通過示例程式碼下載到本地執行後,發現沒有Kafka Producer 的詳細日誌輸出。當檢視SDK原始碼中,發現使用的是org.slf4j.Logger 輸出日誌,如:
但是,當執行 Producer 程式碼後,得到的輸出取沒有包含連線的詳細資訊,對出現連線問題的Debug沒有任何幫助。
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Test Data #0 from thread #18 org.apache.kafka.common.errors.IllegalSaslStateException: Invalid SASL mechanism response, server may be expecting a different protocol
那麼如何來輸出更加詳細的日誌呢?
問題解決
根據日誌顯示,SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".明確指出是因為沒有載入到 org.slf4j.impl.StaticLoggerBinder 類,因為在程式執行的過程中,必須提供實際的日誌記錄實現,否則SLF4J講忽略所有日誌資訊,SLF4J API 通過 SLF4J 繫結與實際的日誌記錄實現進行通訊Log4j。所以需要在pom.xml中引入 org.slf4j 的相關依賴。
在pom.xml中加入
<dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.7.25</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>1.7.25</version> </dependency> <dependency> <groupId>log4j</groupId> <artifactId>log4j</artifactId> <version>1.2.17</version> </dependency> <dependency> <groupId>ch.qos.logback</groupId> <artifactId>logback-classic</artifactId> <version>1.0.13</version>
然後,新增上log4j的配置檔案,在resources資料夾下新增名為 log4j.properties檔案,內容為:
# Root logger option
log4j.rootLogger=INFO, stdout
# Direct log messages to stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1} - %m%n
修改後的檔案內容如截圖所示:
修改完成,執行得到完整日誌
SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/Users/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/Users/.m2/repository/ch/qos/logback/logback-classic/1.0.13/logback-classic-1.0.13.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 2022-01-11 20:03:12 INFO ProducerConfig - ProducerConfig values: acks = 1 batch.size = 16384 bootstrap.servers = [testeventxxxxxx.servicebus.chinacloudapi.cn:9093] buffer.memory = 33554432 client.id = KafkaExampleProducer compression.type = none connections.max.idle.ms = 540000 enable.idempotence = false interceptor.classes = null key.serializer = class org.apache.kafka.common.serialization.LongSerializer linger.ms = 0 max.block.ms = 60000 max.in.flight.requests.per.connection = 5 max.request.size = 1048576 metadata.max.age.ms = 300000 metric.reporters = [] metrics.num.samples = 2 metrics.recording.level = INFO metrics.sample.window.ms = 30000 security.protocol = SASL_SSL send.buffer.bytes = 131072 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1] ssl.endpoint.identification.algorithm = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLS ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS transaction.timeout.ms = 60000 transactional.id = null value.serializer = class org.apache.kafka.common.serialization.StringSerializer 2022-01-11 20:03:16 INFO AbstractLogin - Successfully logged in. 2022-01-11 20:03:17 INFO AppInfoParser - Kafka version : 1.0.0 2022-01-11 20:03:17 INFO AppInfoParser - Kafka commitId : aaa7af6d4a11b29d 2022-01-11 20:03:21 INFO TestProducer - test java logs : info 2022-01-11 20:03:21 ERROR TestProducer - test java logs : error 2022-01-11 20:03:21 WARN TestProducer - test java logs : warn Test Data #0 from thread #18 2022-01-11 20:03:22 ERROR NetworkClient - [Producer clientId=KafkaExampleProducer] Connection to node -1 failed authentication due to: Invalid SASL mechanism response, server may be expecting a different protocol org.apache.kafka.common.errors.IllegalSaslStateException: Invalid SASL mechanism response, server may be expecting a different protocol org.apache.kafka.clients.producer.KafkaProducer@47dec663
參考資料
Slf4j Configuration File Example:https://examples.javacodegeeks.com/enterprise-java/slf4j/slf4j-configuration-file-example/
將 Apache Flink 與適用於 Apache Kafka 的 Azure 事件中心配合使用:https://docs.azure.cn/zh-cn/event-hubs/event-hubs-kafka-flink-tutorial
在微軟雲中國區 (Mooncake) 上實驗以Apache Kafka協議方式傳送/接受Event Hubs訊息 (Java版) :https://www.cnblogs.com/lulight/p/14375190.html
當在複雜的環境中面臨問題,格物之道需:濁而靜之徐清,安以動之徐生。 雲中,恰是如此!