spark streaming報錯 WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel! WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel! WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel! WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel! WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel! WARN sink.TransactionProcessor: Error while processing transaction. java.lang.IllegalStateException: begin() called when transaction is OPEN! at com.google.common.base.Preconditions.checkState(Preconditions.java:145) at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131) at org.apache.spark.streaming.flume.sink.TransactionProcessorKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲populateEvents$…

anonfun$populateEvents$1.apply(TransactionProcessor\mathrm{.}scala:113)atscala\mathrm{.}Option\mathrm{.}foreach(Option\mathrm{.}scala:236)atorg\mathrm{.}apache\mathrm{.}spark\mathrm{.}streaming\mathrm{.}flume\mathrm{.}sink\mathrm{.}TransactionProcessor\mathrm{.}populateEvents(TransactionProcessor\mathrm{.}scala:113)atorg\mathrm{.}apache\mathrm{.}spark\mathrm{.}streaming\mathrm{.}flume\mathrm{.}sink\mathrm{.}TransactionProcessor\mathrm{.}call(TransactionProcessor\mathrm{.}scala$

:243)atorg.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)atjava.util.concurrent.FutureTask.run(FutureTask.java:266)atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)atjava.util.concurrent.ThreadPoolExecutor1.apply(TransactionProcessor.scala:113) at scala.Option.foreach(Option.scala:236) at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor

Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) WARN sink.TransactionProcessor: Spark was unable to successfully process the events. Transaction is being rolled back. WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel! WARN sink.TransactionProcessor: Error while processing transaction. java.lang.IllegalStateException: begin() called when transaction is OPEN! at com.google.common.base.Preconditions.checkState(Preconditions.java:145) at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131) at org.apache.spark.streaming.flume.sink.TransactionProcessorKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲populateEvents$…anonfun$populateEvents$1.apply(TransactionProcessor.scala:113) at scala.Option.foreach(Option.scala:236) at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

解決方法： 在啟動Flume之前會拷貝三個jar(commons-lang3-3.5.jar、scala-library-2.11.8.jar、spark-streaming-flume-sink_2.11-2.2.0.jar)到lib目錄下，檢視lib目錄下的這三個jar是否有衝突，刪掉衝突的jar即可。