admin管理员组文章数量:1566604
前言
最近在使用 java
消费 kafka
服务时,遇到报错,报错内容如下:
环境信息
- kafka 2.5.0
- kerberos
报错内容
Connection to node -1) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.
[2020-05-18 18:14:38,615] INFO [Group Metadata Manager on Broker 193]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2020-05-18 18:24:38,615] INFO [Group Metadata Manager on Broker 193]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2020-05-18 18:34:38,616] INFO [Group Metadata Manager on Broker 193]: Removed 0 expired offsets in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2020-05-18 18:44:02,879] WARN Unexpected error from ; closing connection (org.apache.kafkamonwork.Selector)
org.apache.kafkamonwork.InvalidReceiveException: Invalid receive (size = -720899)
at org.apache.kafkamonwork.NetworkReceive.readFromReadableChannel(NetworkReceive.java:89)
at org.apache.kafkamonwork.NetworkReceive.readFrom(NetworkReceive.java:71)
at org.apache.kafkamonwork.KafkaChannel.receive(KafkaChannel.java:160)
at org.apache.kafkamonwork.KafkaChannel.read(KafkaChannel.java:141)
at org.apache.kafkamonwork.Selector.poll(Selector.java:286)
at kafkawork.Processor.run(SocketServer.scala:413)
at java.lang.Thread.run(Thread.java:748)
[2020-05-18 18:44:38,615] INFO [Group Metadata Manager on Broker 193]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2020-05-18 18:54:38,615] INFO [Group Metadata Manager on Broker 193]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)
排查方向
- 时钟同步(应用与kafka集群之间、kafka集群内部之间)重要
- 使用主机名是否与KDC、KAFKA服务器网络/端口正确通信。
- 防火墙是否关闭,检查
selinux
。 krb5.conf kafka_client_jaas.conf xxx.keytab
三个配置文件内容是否正确,文件所属用户,权限是否正确。- 是否指定 JVM 参数,或者在代码里面是否指定环境变量。
- 如下配置,是否正确配置
properties.setProperty("sasl.kerberos.service.name", "kafka");
properties.setProperty("sasl.mechanism", "GSSAPI");
properties.setProperty("security.protocol", "SASL_PLAINTEXT");
Kafka
相关jar
包版本是否一致,是否有依赖冲突。- 与
KDC
服务器网络是否通信良好(有无丢包),因为kerberos
默认是udp
通信,建议换成tcp
。
对于使用spark-kafka 程序消费 kafka 的,需要注意 "sasl.kerberos.service.name”,“sasl.mechanism”, “security.protocol” 三项配置需要增加前缀 “kafka.”
补充
另外查到网上有别的解决方式:
- 调整
socket.request.max.bytes
参数 - 缺少
jackjson
的pom
依赖
目前个人感觉,上面两个应该不是正确的解决方向,这里也顺便贴出来,供大家参考。
本文标签: nodeConnectionKafkaterminateddue
版权声明:本文标题:【kafka】Connection to node -1) terminated during authentication. This may happen due to any of 内容由热心网友自发贡献,该文观点仅代表作者本人, 转载请联系作者并注明出处:https://www.elefans.com/dongtai/1726225127a1061418.html, 本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容,一经查实,本站将立刻删除。
发表评论