KafkaFlume-聯合配置數---據源是Syslog
在flume中關於資料來源Syslog的配置為以下幾種:Syslog Tcp Source、Multiport Syslog Tcp Source、Syslog Udp Source,ta們的配置基本一致,下面以Syslog Tcp為例來進行說明。其他幾種的配置可以參考官網:http://flume.apache.org/FlumeUserGuide.html#syslog-sources
關於資料來源的配置有兩種方式:
(1)使用工具直接向flume傳送:Jperf
(2)利用Socket進行傳送模擬資料,程式碼如下:
Socket client = null;
client = new Socket(“192.168.100.110”,5000);
out = client.getOutputStream();
String evrent = “Hello World\n”;
while(true){
out.write(event.getBytes());
out.flush();
}
out.close();
#####Sink是kafka
a1.sources = a1
a1.sinks = k1
a1.channels = c1
a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSinka1.sinks.k1.topic = flume a1.sinks.k1.brokerList = localhost:9092 a1.sinks.k1.requiredAcks = 1 a1.sinks.k1.batchSize = 100 a1.sinks.k1.channel=c1
a1.sources = r1 a1.channels = c1 a1.sources.r1.type = syslogtcp a1.sources.r1.port = 5000 a1.sources.r1.host=localhost a1.sources.r1.channels=c1
a1.channels.c1.type = memeory
a1.channels.c1.capacity = 10000
a1.channels.c1.transactionCapacity = 1000