启动kafka报错

川长思鸟来 2021-11-17 05:08 517阅读 0赞

执行启动kafka服务的命令时报错：nohup bin/kafka-server-start.sh config/server.properties 2>&1 &

\[2019-07-28 12:58:44,760\] ERROR \[ReplicaManager broker=1\] Error while making broker the follower for partition Topic: \_\_consumer\_offsets; Partition: 41; Leader: None; AssignedReplicas: ; InSyncReplicas:  in dir None (kafka.server.ReplicaManager)  
org.apache.kafka.common.errors.KafkaStorageException: Error while reading checkpoint file /export/servers/kafka\_2.11-1.0.0/logs/replication-offset-checkpoint  
Caused by: java.io.IOException: Malformed line in checkpoint file (/export/servers/kafka\_2.11-1.0.0/logs/replication-offset-checkpoint): '  
 at kafka.server.checkpoints.CheckpointFile.malformedLineException$1(CheckpointFile.scala:84)  
 at kafka.server.checkpoints.CheckpointFile.liftedTree2$1(CheckpointFile.scala:117)  
 at kafka.server.checkpoints.CheckpointFile.read(CheckpointFile.scala:86)  
 at kafka.server.checkpoints.OffsetCheckpointFile.read(OffsetCheckpointFile.scala:61)  
 at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:147)  
 at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:142)  
 at kafka.utils.Pool.getAndMaybePut(Pool.scala:65)  
 at kafka.cluster.Partition.getOrCreateReplica(Partition.scala:141)  
 at kafka.server.ReplicaManager$$anonfun$makeFollowers$3.apply(ReplicaManager.scala:1227)  
 at kafka.server.ReplicaManager$$anonfun$makeFollowers$3.apply(ReplicaManager.scala:1204)  
 at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)  
 at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)  
 at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)  
 at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)  
 at scala.collection.mutable.HashMap.foreach(HashMap.scala:130)  
 at kafka.server.ReplicaManager.makeFollowers(ReplicaManager.scala:1204)  
 at kafka.server.ReplicaManager.becomeLeaderOrFollower(ReplicaManager.scala:1065)  
 at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:173)  
 at kafka.server.KafkaApis.handle(KafkaApis.scala:103)  
 at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:65)  
 at java.lang.Thread.run(Thread.java:748)  
\[2019-07-28 12:58:44,764\] ERROR Error while reading checkpoint file /export/servers/kafka\_2.11-1.0.0/logs/replication-offset-checkpoint (kafka.server.LogDirFailureChannel)  
java.io.IOException: Malformed line in checkpoint file (/export/servers/kafka\_2.11-1.0.0/logs/replication-offset-checkpoint): '  
 at kafka.server.checkpoints.CheckpointFile.malformedLineException$1(CheckpointFile.scala:84)  
 at kafka.server.checkpoints.CheckpointFile.liftedTree2$1(CheckpointFile.scala:117)  
 at kafka.server.checkpoints.CheckpointFile.read(CheckpointFile.scala:86)  
 at kafka.server.checkpoints.OffsetCheckpointFile.read(OffsetCheckpointFile.scala:61)  
 at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:147)  
 at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:142)  
 at kafka.utils.Pool.getAndMaybePut(Pool.scala:65)  
 at kafka.cluster.Partition.getOrCreateReplica(Partition.scala:141)  
 at kafka.server.ReplicaManager$$anonfun$makeFollowers$3.apply(ReplicaManager.scala:1227)  
 at kafka.server.ReplicaManager$$anonfun$makeFollowers$3.apply(ReplicaManager.scala:1204)  
 at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)  
 at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)  
 at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)  
 at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)  
 at scala.collection.mutable.HashMap.foreach(HashMap.scala:130)  
 at kafka.server.ReplicaManager.makeFollowers(ReplicaManager.scala:1204)  
 at kafka.server.ReplicaManager.becomeLeaderOrFollower(ReplicaManager.scala:1065)  
 at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:173)  
 at kafka.server.KafkaApis.handle(KafkaApis.scala:103)  
 at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:65)  
 at java.lang.Thread.run(Thread.java:748)  
\[2019-07-28 12:58:44,765\] ERROR \[ReplicaManager broker=1\] Error while making broker the follower for partition Topic: \_\_consumer\_offsets; Partition: 32; Leader: None; AssignedReplicas: ; InSyncReplicas:  in dir None (kafka.server.ReplicaManager)  
org.apache.kafka.common.errors.KafkaStorageException: Error while reading checkpoint file /export/servers/kafka\_2.11-1.0.0/logs/replication-offset-checkpoint  
Caused by: java.io.IOException: Malformed line in checkpoint file (/export/servers/kafka\_2.11-1.0.0/logs/replication-offset-checkpoint): '  
 at kafka.server.checkpoints.CheckpointFile.malformedLineException$1(CheckpointFile.scala:84)  
 at kafka.server.checkpoints.CheckpointFile.liftedTree2$1(CheckpointFile.scala:117)  
 at kafka.server.checkpoints.CheckpointFile.read(CheckpointFile.scala:86)  
 at kafka.server.checkpoints.OffsetCheckpointFile.read(OffsetCheckpointFile.scala:61)  
 at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:147)  
 at kafka.cluster.Partition$$anonfun$getOrCreateReplica$1.apply(Partition.scala:142)  
 at kafka.utils.Pool.getAndMaybePut(Pool.scala:65)  
 at kafka.cluster.Partition.getOrCreateReplica(Partition.scala:141)  
 at kafka.server.ReplicaManager$$anonfun$makeFollowers$3.apply(ReplicaManager.scala:1227)  
 at kafka.server.ReplicaManager$$anonfun$makeFollowers$3.apply(ReplicaManager.scala:1204)  
 at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)  
 at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)  
 at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)  
 at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)  
 at scala.collection.mutable.HashMap.foreach(HashMap.scala:130)  
 at kafka.server.ReplicaManager.makeFollowers(ReplicaManager.scala:1204)  
 at kafka.server.ReplicaManager.becomeLeaderOrFollower(ReplicaManager.scala:1065)  
 at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:173)  
 at kafka.server.KafkaApis.handle(KafkaApis.scala:103)  
 at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:65)  
 at java.lang.Thread.run(Thread.java:748)  
\[2019-07-28 12:58:44,766\] INFO \[ReplicaFetcherManager on broker 1\] Removed fetcher for partitions  (kafka.server.ReplicaFetcherManager)  
\[2019-07-28 12:58:44,774\] INFO \[ReplicaFetcherManager on broker 1\] Added fetcher for partitions List() (kafka.server.ReplicaFetcherManager)  
\[2019-07-28 12:58:44,785\] INFO \[ReplicaManager broker=1\] Partitions  are offline due to failure on log directory /export/servers/kafka\_2.11-1.0.0/logs (kafka.server.ReplicaManager)  
\[2019-07-28 12:58:44,804\] INFO \[ReplicaFetcherManager on broker 1\] Removed fetcher for partitions  (kafka.server.ReplicaFetcherManager)  
\[2019-07-28 12:58:44,812\] INFO \[ReplicaManager broker=1\] Broker 1 stopped fetcher for partitions  because they are in the failed log dir /export/servers/kafka\_2.11-1.0.0/logs (kafka.server.ReplicaManager)  
\[2019-07-28 12:58:44,820\] INFO Stopping serving logs in dir /export/servers/kafka\_2.11-1.0.0/logs (kafka.log.LogManager)  
\[2019-07-28 12:58:44,824\] FATAL Shutdown broker because all log dirs in /export/servers/kafka\_2.11-1.0.0/logs have failed (kafka.log.LogManager)

原因：kill -9 强制杀掉kafka进程，导致停止kafka时某些步骤没有处理，比如replication-offset-checkpoint文件没有处理好。

解决方法：备份/export/servers/kafka\_2.11-1.0.0/logs/replication-offset-checkpoint文件，删掉该文件，然后重启kafka，就好了。

转载于:https://www.cnblogs.com/mediocreWorld/p/11261357.html