1. 程式人生 > >Kafka0.10.0安裝配置

Kafka0.10.0安裝配置

1 解壓檔案

  tar -zvxf kafka_2.11-0.10.0.0.tgz

2 修改配置server.properties

  vim server.properties

  broker.id=1

  zookeeper.connect=hdp1:2181,hdp2:2181,hdp3:2181

3 修改配置 zookeeper.properties

   vim zookeeper.properties  

  

# the directory where the snapshot is stored.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181

 

4 複製到hdp2,hdp3節點修改broker.id

  修改配置server.properties  分別修改為2 ,3 

  vim server.properties

  broker.id=2

  zookeeper.connect=hdp1:2181,hdp2:2181,hdp3:2181

 

5 編輯叢集啟動指令碼

  

#!/bin/bash
for host in hdp1 hdp2 hdp3
do
ssh $host "source /etc/profile;/mnt/software/kafka_2.11-0.10.0.0/bin/kafka-server-start.sh -daemon /mnt/software/kafka_2.11-0.10.0.0/config/server.properties"
done
sleep 3
for host in hdp1 hdp2 hdp3
do
echo "---------"
ssh $host "source /etc/profile;jps"
done

 

6 啟動測試

[[email protected] /mnt/software/shell]#start-all-kafka.sh

7 建立topic

  kafka-topics.sh --create --zookeeper hdp1:2181,hdp2:2181,hdp3:2181 --replication-factor 1 --partitions 1 --topic test

 

8 檢視topic

[[email protected] /mnt/software/shell]#kafka-topics.sh --list --zookeeper hdp1:2181,hdp2:2181,hdp3:2181
__consumer_offsets
cmcc
hellokafka
hellokafka1
test
topicA
topicB

 

9 之前需要自行安裝好zookeeper叢集,hdp1,hdp2,hdp3