docker 部署 zookeeper+kafka 叢集
阿新 • • 發佈:2018-12-11
主機三臺
172.16.100.61
172.16.100.62
172.16.100.63
Docker 版本 當前最新版
# 部署zk有2種方法
## 注意 \後不要跟空格
一 . 埠對映
172.16.100.61 執行
docker run -tid --name=zk-1 \ --restart=always \ --privileged=true \ -p 2888:2888 \ -p 3888:3888 \ -p 2181:2181 \ -e ZOO_MY_ID=1 \ -e ZOO_SERVERS=server.1="0.0.0.0:2888:3888 server.2=172.16.100.62:2888:3888 server.3=172.16.100.63:2888:3888" \ -v /data/kafka_cluster/zookeeper/data:/data \ zookeeper:latest
172.16.100.62 執行
docker run -tid --name=zk-2 \ --restart=always \ --privileged=true \ -p 2888:2888 \ -p 3888:3888 \ -p 2181:2181 \ -e ZOO_MY_ID=2 \ -e ZOO_SERVERS=server.1="172.16.100.61:2888:3888 server.2=0.0.0.0:2888:3888 server.3=172.16.100.63:2888:3888" \ -v /data/kafka_cluster/zookeeper/data:/data \ zookeeper:latest
172.16.100.63 執行
docker run -tid --name=zk-3 \ --restart=always \ --privileged=true \ -p 2888:2888 \ -p 3888:3888 \ -p 2181:2181 \ -e ZOO_MY_ID=3 \ -e ZOO_SERVERS=server.1="172.16.100.61:2888:3888 server.2=172.16.100.62:2888:3888 server.3=0.0.0.0:2888:3888" \ -v /data/kafka_cluster/zookeeper/data:/data \
zookeeper:latest
#檢視叢集狀態
docker exec -it zk-1 bash ./bin/zkServer.sh status
二.直接走宿主機IP
每臺機器都執行一次,只需要修改 ZOO_MY_ID 即可 只要ZOO_SERVERS 和你的ID一致即可
docker run -tid --name=zk-3 \ --restart=always \ --privileged=true \ --net=host \ -e ZOO_MY_ID=1 \ -e ZOO_SERVERS=server.1="172.16.100.61:2888:3888 server.2=172.16.100.62:2888:3888 server.3=172.16.100.63:2888:3888" \ -v /data/kafka_cluster/zookeeper/data:/data \ zookeeper:latest
### 上述zk檢查過後沒有問題就可以進行下一步了 ###
解析主機名
注意: kafka-logs-data-1 是主機的名字 如果你的主機是xxx 那這裡應該改為kafka-logs-xxx
同時我映射了本地的hosts到容器裡,所有容器獲取到的主機名就是宿主機的名字
在61執行
docker run -itd --name=kafka \ --restart=always \ --net=host \ --privileged=true \ -v /etc/hosts:/etc/hosts \ -v /data/kafka_cluster/kafka/data:/kafka/kafka-logs-data-1 \ -v /data/kafka_cluster/kafka/logs:/opt/kafka/logs \ -e KAFKA_ADVERTISED_HOST_NAME=172.16.100.61 \ -e HOST_IP=172.16.100.61 \ -e KAFKA_ADVERTISED_PORT=9092 \ -e KAFKA_ZOOKEEPER_CONNECT=172.16.100.61:2181,172.16.100.62:2181,172.16.100.63:2181 \ -e KAFKA_BROKER_ID=0 \ -e KAFKA_HEAP_OPTS="-Xmx8196M -Xms8196M" \ wurstmeister/kafka:0.10.1.1
在62執行
docker run -itd --name=kafka \ --restart=always \ --net=host \ --privileged=true \ -v /etc/hosts:/etc/hosts \ -v /data/kafka_cluster/kafka/data:/kafka/kafka-logs-data-2 \ -v /data/kafka_cluster/kafka/logs:/opt/kafka/logs \ -e KAFKA_ADVERTISED_HOST_NAME=172.16.100.62 \ -e HOST_IP=172.16.100.62 \ -e KAFKA_ADVERTISED_PORT=9092 \ -e KAFKA_ZOOKEEPER_CONNECT=172.16.100.61:2181,172.16.100.62:2181,172.16.100.63:2181 \ -e KAFKA_BROKER_ID=1 \ -e KAFKA_HEAP_OPTS="-Xmx8196M -Xms8196M" \ wurstmeister/kafka:0.10.1.1
在63執行
docker run -itd --name=kafka \ --restart=always \ --net=host \ --privileged=true \ -v /etc/hosts:/etc/hosts \ -v /data/kafka_cluster/kafka/data:/kafka/kafka-logs-data-3 \ -v /data/kafka_cluster/kafka/logs:/opt/kafka/logs \ -e KAFKA_ADVERTISED_HOST_NAME=172.16.100.63 \ -e HOST_IP=172.16.100.63 \ -e KAFKA_ADVERTISED_PORT=9092 \ -e KAFKA_ZOOKEEPER_CONNECT=172.16.100.61:2181,172.16.100.62:2181,172.16.100.63:2181 \ -e KAFKA_BROKER_ID=2 \ -e KAFKA_HEAP_OPTS="-Xmx8196M -Xms8196M" \ wurstmeister/kafka:0.10.1.1
##建立topic test
#訂閱
docker exec -it kafka bash /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server 172.16.100.61:9092,172.16.100.62:9092,172.16.100.63:9092 --topic test --from-beginning
#傳送訊息
docker exec -it kafka bash /opt/kafka/bin/kafka-console-producer.sh --broker-list 172.16.100.61:9092,172.16.100.62:9092,172.16.100.63:9092 --topic test