1. 程式人生 > 實用技巧 >docker部署kafka叢集

docker部署kafka叢集

利用docker可以很方便的在一臺機子上搭建kafka叢集並進行測試。為了簡化配置流程,採用docker-compose進行進行搭建。

kafka搭建過程如下:

  1. 編寫docker-compose.yml檔案,內容如下:
version: '3.3'

services:
  zookeeper:
    image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/zookeeper:latest
    ports:
      - 2181:2181
    container_name: zookeeper
    networks:
      default:
        ipv4_address: 172.19.0.11
  kafka0:
    image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/kafka:latest
    depends_on:
      - zookeeper
    container_name: kafka0
    ports:
      - 9092:9092
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka0:9092
      KAFKA_LISTENERS: PLAINTEXT://kafka0:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 0
    volumes:
      - /root/data/kafka0/data:/data
      - /root/data/kafka0/log:/datalog
    networks:
      default:
        ipv4_address: 172.19.0.12
  kafka1:
    image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/kafka:latest
    depends_on:
      - zookeeper
    container_name: kafka1
    ports:
      - 9093:9093
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka1:9093
      KAFKA_LISTENERS: PLAINTEXT://kafka1:9093
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 1
    volumes:
      - /root/data/kafka1/data:/data
      - /root/data/kafka1/log:/datalog
    networks:
      default:
        ipv4_address: 172.19.0.13
  kafka2:
    image: registry.cn-shenzhen.aliyuncs.com/zisefeizhu/kafka:latest
    depends_on:
      - zookeeper
    container_name: kafka2
    ports:
      - 9094:9094
    environment:
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka2:9094
      KAFKA_LISTENERS: PLAINTEXT://kafka2:9094
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 2
    volumes:
      - /root/data/kafka2/data:/data
      - /root/data/kafka2/log:/datalog
    networks:
      default:
        ipv4_address: 172.19.0.14
networks:
  default:
    external:
      name: zookeeper_kafka
  1. 建立子網
    docker network create --subnet 172.19.0.0/16 --gateway 172.19.0.1 zookeeper_kafka
  2. 執行docker-compose命令進行搭建
    docker-compose -f docker-compose.yaml up -d
    輸入docker ps -a 命令如能檢視到我們啟動的三個服務且處於執行狀態說明部署成功
  1. 測試kafka
    輸入docker exec -it kafka0 bash 進入kafka0容器,並執行如下命令建立topic
    cd /opt/kafka_2.13-2.6.0/bin/
    ./kafka-topics.sh --create --topic chat --partitions 5 --zookeeper 111.110.111.111:2181 --replication-factor 3

    輸入如下命令開啟生產者
    ./kafka-console-producer.sh --broker-list kafka0:9092 --topic chat
    開啟另一個shell介面進入kafka2容器並執行下列命令開啟消費者
    ./kafka-console-consumer.sh --bootstrap-server kafka2:9094 --topic chat --from-beginning

回到生產者shell輸入訊息,看消費者shell是否會出現同樣的訊息,如果能夠出現說明kafka叢集搭建正常。