1. 程式人生 > >Spark Streaming的foreachRDD把處理後的資料寫入外部儲存系統中

Spark Streaming的foreachRDD把處理後的資料寫入外部儲存系統中

1.程式碼

dstream.foreachRDD { rdd =>

  rdd.foreachPartition { partitionOfRecords =>

    // ConnectionPool is a static, lazily initialized pool of connections

    val connection = ConnectionPool.getConnection()

    partitionOfRecords.foreach(record => connection.send(record))

    ConnectionPool.returnConnection(connection)  // return to the pool for future reuse

  }}