Kafka動態調整topic分割槽partition
我們在使用kafka時,初期建立時所指定topic屬性需要修改,如何動態修改kafka屬性?kafka提供了命令列工具—kafka-topics.sh.
- kafka-topics.sh工具介紹
kafka-topics.sh工具也是我們用來建立topic、檢視topic詳情的工具。
直接執行kafka-topics.sh可以產出它是用來建立、刪除、檢視以及更新topic
[email protected]:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh
Create, delete, describe, or change a topic.
Option Description
–alter Alter the number of partitions,
replica assignment, and/or
configuration for the topic.
–config <String: name=value> A topic configuration override for
…
- 更新或者修改topic
注意:我的kafka版本是1.1.0, 並且我只有一個broker。
1, 首先我們建立一個topic,然後檢視詳情
[email protected]:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --create --zookeeper 192.168.119.131:2181 --replication-factor 1 --partitions 4 --topic yqtopic1
Created topic “yqtopic1”.
[email protected]:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --describe --zookeeper 192.168.119.131:2181 --topic yqtopic1
Topic:yqtopic1 PartitionCount:4 ReplicationFactor:1 Configs:
Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
2,修改剛建立的topic,並檢視修改的情況
將分割槽數有4修改為12
[email protected]:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --alter --zookeeper 192.168.119.131:2181 --topic yqtopic1 --partitions 12
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!
[email protected]:/opt/kafka_2.11-1.1.0/bin#
[email protected]:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh --describe --zookeeper 192.168.119.131:2181 --topic yqtopic1 Topic:yqtopic1 PartitionCount:12 ReplicationFactor:1 Configs:
Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 4 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 5 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 7 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 8 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 9 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 10 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 11 Leader: 0 Replicas: 0 Isr: 0
[email protected]:/opt/kafka_2.11-1.1.0/bin# ls -al /tmp/kafka-logs/
total 72
drwxr-xr-x 14 root root 4096 Oct 13 14:34 .
drwxrwxrwt 17 root root 4096 Oct 13 14:34 …
-rw-r–r-- 1 root root 0 Oct 13 14:10 cleaner-offset-checkpoint
-rw-r–r-- 1 root root 0 Oct 13 14:10 .lock
-rw-r–r-- 1 root root 4 Oct 13 14:33 log-start-offset-checkpoint
-rw-r–r-- 1 root root 54 Oct 13 14:10 meta.properties
-rw-r–r-- 1 root root 163 Oct 13 14:33 recovery-point-offset-checkpoint
-rw-r–r-- 1 root root 163 Oct 13 14:34 replication-offset-checkpoint
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-0
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-1
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-10
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-11
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-2
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-3
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-4
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-5
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-6
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-7
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-8
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-9
[email protected]:/opt/kafka_2.11-1.1.0/bin#
修改後的截圖如下