1. 程式人生 > >docker 部署 zookeeper+kafka 叢集

docker 部署 zookeeper+kafka 叢集

主機三臺
172.16.100.61
172.16.100.62
172.16.100.63
Docker 版本 當前最新版


# 部署zk有2種方法

## 注意 \後不要跟空格

一 . 埠對映

172.16.100.61 執行

docker run -tid --name=zk-1 \
--restart=always \
--privileged=true \
-p 2888:2888 \
-p 3888:3888 \
-p 2181:2181 \
-e ZOO_MY_ID=1 \
-e ZOO_SERVERS=server.1="0.0.0.0:2888:3888 server.2=172.16.100.62:2888:3888 server.3=172.16.100.63:2888:3888
" \ -v /data/kafka_cluster/zookeeper/data:/data \ zookeeper:latest

 


172.16.100.62 執行

docker run -tid --name=zk-2 \
--restart=always \
--privileged=true \
-p 2888:2888 \
-p 3888:3888 \
-p 2181:2181 \
-e ZOO_MY_ID=2 \
-e ZOO_SERVERS=server.1="172.16.100.61:2888:3888 server.2=0.0.0.0:2888:3888 server.3=172.16.100.63:2888:3888
" \ -v /data/kafka_cluster/zookeeper/data:/data \ zookeeper:latest

 

172.16.100.63 執行

docker run -tid --name=zk-3 \
--restart=always \
--privileged=true \
-p 2888:2888 \
-p 3888:3888 \
-p 2181:2181 \
-e ZOO_MY_ID=3 \
-e ZOO_SERVERS=server.1="172.16.100.61:2888:3888 server.2=172.16.100.62:2888:3888 server.3=0.0.0.0:2888:3888
" \ -v /data/kafka_cluster/zookeeper/data:/data \
zookeeper:latest


#檢視叢集狀態

docker exec -it zk-1 bash ./bin/zkServer.sh status

 



二.直接走宿主機IP

每臺機器都執行一次,只需要修改 ZOO_MY_ID 即可 只要ZOO_SERVERS 和你的ID一致即可

docker run -tid --name=zk-3 \
--restart=always \
--privileged=true \
--net=host \
-e ZOO_MY_ID=1 \
-e ZOO_SERVERS=server.1="172.16.100.61:2888:3888 server.2=172.16.100.62:2888:3888 server.3=172.16.100.63:2888:3888" \
-v /data/kafka_cluster/zookeeper/data:/data \
zookeeper:latest

 
### 上述zk檢查過後沒有問題就可以進行下一步了 ###

解析主機名

 

注意: kafka-logs-data-1 是主機的名字 如果你的主機是xxx 那這裡應該改為kafka-logs-xxx

            同時我映射了本地的hosts到容器裡,所有容器獲取到的主機名就是宿主機的名字   

在61執行

docker run -itd --name=kafka \
--restart=always \
--net=host \
--privileged=true \
-v /etc/hosts:/etc/hosts \
-v /data/kafka_cluster/kafka/data:/kafka/kafka-logs-data-1 \
-v /data/kafka_cluster/kafka/logs:/opt/kafka/logs \
-e KAFKA_ADVERTISED_HOST_NAME=172.16.100.61 \
-e HOST_IP=172.16.100.61 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e KAFKA_ZOOKEEPER_CONNECT=172.16.100.61:2181,172.16.100.62:2181,172.16.100.63:2181 \
-e KAFKA_BROKER_ID=0 \
-e KAFKA_HEAP_OPTS="-Xmx8196M -Xms8196M" \
wurstmeister/kafka:0.10.1.1 


在62執行

docker run -itd --name=kafka \
--restart=always \
--net=host \
--privileged=true \
-v /etc/hosts:/etc/hosts \
-v /data/kafka_cluster/kafka/data:/kafka/kafka-logs-data-2 \
-v /data/kafka_cluster/kafka/logs:/opt/kafka/logs \
-e KAFKA_ADVERTISED_HOST_NAME=172.16.100.62 \
-e HOST_IP=172.16.100.62 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e KAFKA_ZOOKEEPER_CONNECT=172.16.100.61:2181,172.16.100.62:2181,172.16.100.63:2181 \
-e KAFKA_BROKER_ID=1 \
-e KAFKA_HEAP_OPTS="-Xmx8196M -Xms8196M" \
wurstmeister/kafka:0.10.1.1


在63執行

docker run -itd --name=kafka \
--restart=always \
--net=host \
--privileged=true \
-v /etc/hosts:/etc/hosts \
-v /data/kafka_cluster/kafka/data:/kafka/kafka-logs-data-3 \
-v /data/kafka_cluster/kafka/logs:/opt/kafka/logs \
-e KAFKA_ADVERTISED_HOST_NAME=172.16.100.63 \
-e HOST_IP=172.16.100.63 \
-e KAFKA_ADVERTISED_PORT=9092 \
-e KAFKA_ZOOKEEPER_CONNECT=172.16.100.61:2181,172.16.100.62:2181,172.16.100.63:2181 \
-e KAFKA_BROKER_ID=2 \
-e KAFKA_HEAP_OPTS="-Xmx8196M -Xms8196M" \
wurstmeister/kafka:0.10.1.1

 

##建立topic test

#訂閱
docker exec -it kafka  bash /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server 172.16.100.61:9092,172.16.100.62:9092,172.16.100.63:9092 --topic test --from-beginning

#傳送訊息
docker exec -it kafka  bash /opt/kafka/bin/kafka-console-producer.sh --broker-list 172.16.100.61:9092,172.16.100.62:9092,172.16.100.63:9092 --topic test