Kafka0.10.0安裝配置
1 解壓檔案
tar -zvxf kafka_2.11-0.10.0.0.tgz
2 修改配置server.properties
vim server.properties
broker.id=1
zookeeper.connect=hdp1:2181,hdp2:2181,hdp3:2181
3 修改配置 zookeeper.properties
vim zookeeper.properties
# the directory where the snapshot is stored.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
4 複製到hdp2,hdp3節點修改broker.id
修改配置server.properties 分別修改為2 ,3
vim server.properties
broker.id=2
zookeeper.connect=hdp1:2181,hdp2:2181,hdp3:2181
5 編輯叢集啟動指令碼
#!/bin/bash
for host in hdp1 hdp2 hdp3
do
ssh $host "source /etc/profile;/mnt/software/kafka_2.11-0.10.0.0/bin/kafka-server-start.sh -daemon /mnt/software/kafka_2.11-0.10.0.0/config/server.properties"
done
sleep 3
for host in hdp1 hdp2 hdp3
do
echo "---------"
ssh $host "source /etc/profile;jps"
done
6 啟動測試
[[email protected] /mnt/software/shell]#start-all-kafka.sh
7 建立topic
kafka-topics.sh --create --zookeeper hdp1:2181,hdp2:2181,hdp3:2181 --replication-factor 1 --partitions 1 --topic test
8 檢視topic
[[email protected] /mnt/software/shell]#kafka-topics.sh --list --zookeeper hdp1:2181,hdp2:2181,hdp3:2181
__consumer_offsets
cmcc
hellokafka
hellokafka1
test
topicA
topicB
9 之前需要自行安裝好zookeeper叢集,hdp1,hdp2,hdp3