kafka的shell命令

缺乏、安全感 2022-05-10 05:50 159阅读 0赞

kafka的 shell 命令

启动集群每个节点的进程

nohup [kafka-server-start.sh][]  
/home/hadoop/kafka/kafka\_2.11-1.1.0/config/server.properties  
1>~/logs/kafka\_std.log  
2>~/logs/kafka\_err.logs &

创建kafka的topic  
[kafka-topics.sh][]  
–create  
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181  
–replication-factor 3  
–partitions 10  
–topic kafka\_my01

kafka查看topic的list  
[kafka-topics.sh][]  
–list  
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181

查看某个指定的kafka topic的详细信息：

[kafka-topics.sh][]  
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181  
–describe  
–topic kafka\_test01

Topic:kafka\_test01 PartitionCount:10 ReplicationFactor:3 Configs:  
Topic: kafka\_test01 Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3  
Topic: kafka\_test01 Partition: 1 Leader: 2 Replicas: 2,3,1 Isr: 2,1,3  
Topic: kafka\_test01 Partition: 2 Leader: 3 Replicas: 3,1,2 Isr: 1,2,3  
Topic: kafka\_test01 Partition: 3 Leader: 1 Replicas: 1,3,2 Isr: 1,2,3  
Topic: kafka\_test01 Partition: 4 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3  
Topic: kafka\_test01 Partition: 5 Leader: 3 Replicas: 3,2,1 Isr: 1,2,3  
Topic: kafka\_test01 Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3  
Topic: kafka\_test01 Partition: 7 Leader: 2 Replicas: 2,3,1 Isr: 2,1,3  
Topic: kafka\_test01 Partition: 8 Leader: 3 Replicas: 3,1,2 Isr: 1,2,3  
Topic: kafka\_test01 Partition: 9 Leader: 1 Replicas: 1,3,2 Isr: 1,2,3

第一行是对所有分区的一个描述，然后每个分区都会对应一行  
Topic：topic 名称  
Partition：topic 的分区编号  
leader：负责处理消息的读和写，leader 是从所有节点中随机选择的。  
replicas：列出了所有的副本节点，不管节点是否在服务中。  
isr：正在服务中的节点。

开启生产者模拟生成数据  
[kafka-console-producer.sh][]  
–broker-list hadoop01:9092,hadoop02:9092,hadoop03:9092  
–topic kafka\_test01

开启消费者  
[kafka-console-consumer.sh][]  
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181  
–from-beginning  
–topic kafka\_test01

查看某topic某个分区的偏移量最大值和最小值

[kafka-run-class.sh][]  
kafka.tools.GetOffsetShell  
–topic kafka\_test01  
–time -1  
–broker-list hadoop01:9092,hadoop02:9092,hadoop03:9092  
–partitions 0

增加topic分区数  
[kafka-topics.sh][]  
–alter  
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181  
–topic kafka\_test01  
–partitions 20 \\

删除topic  
[kafka-topics.sh][]  
–delete  
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181  
–topic kafka\_my01

[kafka-server-start.sh]: http://kafka-server-start.sh
[kafka-topics.sh]: http://kafka-topics.sh
[kafka-console-producer.sh]: http://kafka-console-producer.sh
[kafka-console-consumer.sh]: http://kafka-console-consumer.sh
[kafka-run-class.sh]: http://kafka-run-class.sh