技術標籤：DockerKafkadockerkafkazookeeper

前提

由於 Kafka 藉助 ZooKeeper 進行資料同步，所以安裝 Kafka 叢集時，需先安裝 ZooKeeper 叢集。

為了降低耦合，分別安裝 ZooKeeper 叢集和 Kafka 叢集，然後通過配置在同一個 Docker 網路中保證 Kafka 能和 ZooKeeper 正常通訊。

首先通過docker network create zookeeper_network命令建立一個名為zookeeper_network的 Docker 網路。

ZooKeeper Compose 配置檔案

建立一個 docker-compose.yml 檔案，內容如下：

version: '3.1'
networks:
  default:
    external:
      name: zookeeper_network
services:
  zoo1:
    image: zookeeper
    restart: always
    container_name: zoo1
    hostname: zoo1
    ports:
      - 2181:2181
    volumes:
      - "./zoo1/data:/data"
      - "./zoo1/datalog:/datalog"
 

    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181

  zoo2:
    image: zookeeper
    restart: always
    container_name: zoo2
    hostname: zoo2
    ports:
      - 2182:2181
    volumes:
      - "./zoo2/data:/data"
 

      - "./zoo2/datalog:/datalog"
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181

  zoo3:
    image: zookeeper
    restart: always
    container_name: zoo3
    hostname: zoo3
    ports:
      - 2183:2181
    volumes:
      - "./zoo3/data:/data"
      - "./zoo3/datalog:/datalog"
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181

通過docker-compose up -d命令啟動 ZooKeeper 叢集，可以進入容器檢視叢集狀態。可以參考這篇文章：docker-compose安裝ZooKeeper叢集

Kafka Compose 配置檔案

建立一個 docker-compose.yml 檔案，內容如下：

version: '3.1'

networks:
  default:
    external:
      name: zookeeper_network
services:
  kafka1:
    image: wurstmeister/kafka
    restart: unless-stopped
    container_name: kafka1
    hostname: kafka1
    ports:
      - "9092:9092"
    external_links:
      - zoo1
      - zoo2
      - zoo3
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.75.1:9092    ## 宿主機IP
      KAFKA_ADVERTISED_HOST_NAME: kafka1
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181,zoo2:2181,zoo3:2181"
    volumes:
      - "./kafka/kafka1/data/:/kafka"


  kafka2:
    image: wurstmeister/kafka
    restart: unless-stopped
    container_name: kafka2
    hostname: kafka2
    ports:
      - "9093:9092"
    external_links:
      - zoo1
      - zoo2
      - zoo3
    environment:
      KAFKA_BROKER_ID: 2
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.75.1:9093    ## 宿主機IP
      KAFKA_ADVERTISED_HOST_NAME: kafka2
      KAFKA_ADVERTISED_PORT: 9093
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181,zoo2:2181,zoo3:2181"
    volumes:
      - "./kafka/kafka2/data/:/kafka"

  kafka3:
    image: wurstmeister/kafka
    restart: unless-stopped
    container_name: kafka3
    hostname: kafka3
    ports:
      - "9094:9092"
    external_links:
      - zoo1
      - zoo2
      - zoo3
    environment:
      KAFKA_BROKER_ID: 3
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://192.168.75.1:9094   ## 宿主機IP
      KAFKA_ADVERTISED_HOST_NAME: kafka3
      KAFKA_ADVERTISED_PORT: 9094
      KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181,zoo2:2181,zoo3:2181"
    volumes:
      - "./kafka/kafka3/data/:/kafka"

  kafka-manager: # Kafka 圖形管理介面
    image: sheepkiller/kafka-manager:latest
    restart: unless-stopped
    container_name: kafka-manager
    hostname: kafka-manager
    ports:
      - "9000:9000"
    links:            # 連線本compose檔案建立的container
      - kafka1
      - kafka2
      - kafka3
    external_links:   # 連線外部compose檔案建立的container
      - zoo1
      - zoo2
      - zoo3
    environment:
      ZK_HOSTS: zoo1:2181,zoo2:2181,zoo3:2181
      KAFKA_BROKERS: kafka1:9092,kafka2:9093,kafka3:9094

在啟動了 ZooKeeper 集群后再啟動 Kafka 叢集。之後可以通過訪問 http://localhost:9000 訪問 kafka-manager。

我的更多文章盡在：我的個人部落格