docker部署kafka叢集
阿新 • • 發佈:2020-12-17
利用docker可以很方便的在一臺機子上搭建kafka叢集並進行測試。為了簡化配置流程,採用docker-compose進行進行搭建。
kafka搭建過程如下:
- 編寫docker-compose.yml檔案,內容如下:
version: '3.3' services: zookeeper: image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/zookeeper:latest ports: - 2181:2181 container_name: zookeeper networks: default: ipv4_address: 172.19.0.11 kafka0: image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/kafka:latest depends_on: - zookeeper container_name: kafka0 ports: - 9092:9092 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka0:9092 KAFKA_LISTENERS: PLAINTEXT://kafka0:9092 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_BROKER_ID: 0 volumes: - /root/data/kafka0/data:/data - /root/data/kafka0/log:/datalog networks: default: ipv4_address: 172.19.0.12 kafka1: image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/kafka:latest depends_on: - zookeeper container_name: kafka1 ports: - 9093:9093 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9093 KAFKA_LISTENERS: PLAINTEXT://kafka1:9093 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_BROKER_ID: 1 volumes: - /root/data/kafka1/data:/data - /root/data/kafka1/log:/datalog networks: default: ipv4_address: 172.19.0.13 kafka2: image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/kafka:latest depends_on: - zookeeper container_name: kafka2 ports: - 9094:9094 environment: KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9094 KAFKA_LISTENERS: PLAINTEXT://kafka2:9094 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_BROKER_ID: 2 volumes: - /root/data/kafka2/data:/data - /root/data/kafka2/log:/datalog networks: default: ipv4_address: 172.19.0.14 networks: default: external: name: zookeeper_kafka
- 建立子網
docker network create --subnet 172.19.0.0/16 --gateway 172.19.0.1 zookeeper_kafka
- 執行docker-compose命令進行搭建
docker-compose -f docker-compose.yaml up -d
輸入docker ps -a
命令如能檢視到我們啟動的三個服務且處於執行狀態說明部署成功
- 測試kafka
輸入docker exec -it kafka0 bash
進入kafka0容器,並執行如下命令建立topic
cd /opt/kafka_2.13-2.6.0/bin/
./kafka-topics.sh --create --topic chat --partitions 5 --zookeeper 111.110.111.111:2181 --replication-factor 3
輸入如下命令開啟生產者
./kafka-console-producer.sh --broker-list kafka0:9092 --topic chat
開啟另一個shell介面進入kafka2容器並執行下列命令開啟消費者
./kafka-console-consumer.sh --bootstrap-server kafka2:9094 --topic chat --from-beginning
回到生產者shell輸入訊息,看消費者shell是否會出現同樣的訊息,如果能夠出現說明kafka叢集搭建正常。