flume的一個例子:從指定網路埠採集資料輸出到控制檯
阿新 • • 發佈:2022-03-18
需求一:從指定網路埠採集資料輸出到控制檯
flume官網:https://flume.apache.org/
例子:Flume 1.9.0 User Guide — Apache Flume
使用flume的關鍵就是寫配置檔案:
A)配置Source
B)配置Channe
C)配置Sink
D)把以上三個元件串起來
a1:agent的名稱
r1:source的名稱
k1:sinks的名稱
c1:channels的名稱
注意:
一個source可以輸出到多個channels
配置過程:
(1)
1、cd /kkb/install/apache-flume-1.7.0-bin/conf
2、在/conf
下建立exmaple.conf
example.conf:
# example.conf: A single-node Flume configuration # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source
#繫結的source a1.sources.r1.type = netcat a1.sources.r1.bind =localhost a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type= logger # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
(2)啟動agent:
./flume-ng agent -n a1 -c $FLUME_HOME/conf -f /kkb/install/apache-flume-1.7.0-bin/conf/exmaple.conf -Dflume.root.logger=INFO,console
(3)使用telnet
進行測試
另起個終端,輸入:telnet localhost 44444
輸出成功!!!
其他:
Event: { headers:{} body: 68 65 6C 6C 30 0D hell0. }
Event是flume資料傳輸的基本單元
Event = 可選的headers +bety array
參考:https://blog.csdn.net/fanfan4569/article/details/82957998