Error contacting service. It is probably not running问题"/>
ZK的Error contacting service. It is probably not running问题
记录测试的时候遇到了一个bug
发起请求后返回502 bad gateway(nginx/1.1.19)查看到nginx正常启动后查看后端的日志发现了报错
127.0.0.1 localhost
java.ConnectException: Connection refusedat sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
报错显示:dubbo链接zookeeper链接不上,服务不可用
1.排查问题
查看hosts配置正常
root@ubuntu:~# cat /etc/hosts
127.0.0.1 localhost
查看zk的状态
● zk.service - Zookeeper ServiceLoaded: loaded (/etc/systemd/system/zk.service; enabled; vendor preset: enabled)Active: activating (start-post) (Result: exit-code) since Mon 2021-10-11 20:17:54 CST; 2s agoProcess: 9433 ExecStop=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh stop (code=exited, status=0/SUCCESS)Process: 9448 ExecStart=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh start (code=exited, status=0/SUCCESS)Main PID: 9458 (code=exited, status=1/FAILURE); : 9485 (sleep)Tasks: 1Memory: 256.0KCPU: 3.695sCGroup: /system.slice/zk.service└─control└─9485 /bin/sleep 4Oct 11 20:17:54 ubuntu systemd[1]: Starting Zookeeper Service...
Oct 11 20:17:54 ubuntu zkServer.sh[9448]: ZooKeeper JMX enabled by default
Oct 11 20:17:54 ubuntu zkServer.sh[9448]: Using config: /opt/zk1/zookeeper-3.4.10/bin/../conf/zoo.cfg
Oct 11 20:17:55 ubuntu zkServer.sh[9448]: Starting zookeeper ... STARTED
Oct 11 20:17:56 ubuntu systemd[1]: zk.service: Main process exited, code=exited, status=1/FAILURE
发现是因为zk没有启动成功导致的
2. 解决问题
找到zookeeper的bin目录 使用./zkServer.sh start 可以正常启动
但是使用./zkServer.sh status的时候发现报错Error contacting service. It is probably not running.
ZooKeeper JMX enabled by default
Using config: /opt/zk1/zookeeper-3.4.10/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
使用systemctl status zk命令查看状态(灰色小圆点)
● zk.service - Zookeeper ServiceLoaded: loaded (/etc/systemd/system/zk.service; enabled; vendor preset: enabled)Active: activating (start-post) (Result: exit-code) since Mon 2021-10-11 20:17:54 CST; 2s agoProcess: 9433 ExecStop=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh stop (code=exited, status=0/SUCCESS)Process: 9448 ExecStart=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh start (code=exited, status=0/SUCCESS)Main PID: 9458 (code=exited, status=1/FAILURE); : 9485 (sleep)Tasks: 1Memory: 256.0KCPU: 3.695sCGroup: /system.slice/zk.service└─control└─9485 /bin/sleep 4Oct 11 20:17:54 ubuntu systemd[1]: Starting Zookeeper Service...
Oct 11 20:17:54 ubuntu zkServer.sh[9448]: ZooKeeper JMX enabled by default
Oct 11 20:17:54 ubuntu zkServer.sh[9448]: Using config: /opt/zk1/zookeeper-3.4.10/bin/../conf/zoo.cfg
Oct 11 20:17:55 ubuntu zkServer.sh[9448]: Starting zookeeper ... STARTED
Oct 11 20:17:56 ubuntu systemd[1]: zk.service: Main process exited, code=exited, status=1/FAILURE
尝试启动systemctl start zk
Job for zk.service failed because the control process exited with error code. See "systemctl status zk.service" and "journalctl -xe" for details.
再次查看systemctl status zk
● zk.service - Zookeeper ServiceLoaded: loaded (/etc/systemd/system/zk.service; enabled; vendor preset: enabled)Active: activating (start-post) since Mon 2021-10-11 20:18:16 CST; 1s agoProcess: 9894 ExecStop=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh stop (code=exited, status=0/SUCCESS)Process: 9910 ExecStart=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh start (code=exited, status=0/SUCCESS)Main PID: 9920 (java); : 9953 (sleep)Tasks: 26Memory: 170.6MCPU: 3.268sCGroup: /system.slice/zk.service├─9920 java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,ROLLINGFILE -cp /opt/zk1/zookeeper-3.4.10/bin/../build/classes:/opt/zk1/zookeeper-3.4.10/bin/../build/lib/*.jar:/└─control└─9953 /bin/sleep 4Oct 11 20:18:16 ubuntu systemd[1]: Starting Zookeeper Service...
Oct 11 20:18:16 ubuntu zkServer.sh[9910]: ZooKeeper JMX enabled by default
Oct 11 20:18:16 ubuntu zkServer.sh[9910]: Using config: /opt/zk1/zookeeper-3.4.10/bin/../conf/zoo.cfg
Oct 11 20:18:17 ubuntu zkServer.sh[9910]: Starting zookeeper ... STARTED
发现还是不行。
查看配置文件less zoo.cfg 找到日志路径
dataLogDir=/opt/zk1/zookeeper-3.4.10/datalog
进入/opt/zk1/zookeeper-3.4.10/datalog/version-2 发现有一个日志疑似损坏(log.220770)
-rw-r--r-- 1 root root 67108880 Sep 22 00:21 log.1ce8b6
-rw-r--r-- 1 root root 67108880 Sep 22 02:30 log.1e0dbe
-rw-r--r-- 1 root root 67108880 Sep 22 11:15 log.1e4a05
-rw-r--r-- 1 root root 67108880 Sep 22 20:26 log.1f407d
-rw-r--r-- 1 root root 67108880 Sep 23 02:30 log.20427c
-rw-r--r-- 1 root root 67108880 Sep 23 12:32 log.20ecc9
-rw-r--r-- 1 root root 0 Sep 23 12:33 log.220770
删除此日志 rm log.220770再次启动并查看状态
systemctl start zk systemctl status zk (绿色小圆点)
● zk.service - Zookeeper ServiceLoaded: loaded (/etc/systemd/system/zk.service; enabled; vendor preset: enabled)Active: active (running) since Mon 2021-10-11 20:19:30 CST; 5s agoProcess: 11032 ExecStop=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh stop (code=exited, status=0/SUCCESS)Process: 11088 ExecStartPost=/bin/sleep 4 (code=exited, status=0/SUCCESS)Process: 11047 ExecStart=/opt/zk1/zookeeper-3.4.10/bin/zkServer.sh start (code=exited, status=0/SUCCESS)Main PID: 11057 (java)Tasks: 28Memory: 222.5MCPU: 4.229sCGroup: /system.slice/zk.service└─11057 java -Dzookeeper.log.dir=. -Dzookeeper.root.logger=INFO,ROLLINGFILE -cp /opt/zk1/zookeeper-3.4.10/bin/../build/classes:/opt/zk1/zookeeper-3.4.10/bin/../build/lib/*.jar:Oct 11 20:19:25 ubuntu systemd[1]: Starting Zookeeper Service...
Oct 11 20:19:25 ubuntu zkServer.sh[11047]: ZooKeeper JMX enabled by default
Oct 11 20:19:25 ubuntu zkServer.sh[11047]: Using config: /opt/zk1/zookeeper-3.4.10/bin/../conf/zoo.cfg
Oct 11 20:19:26 ubuntu zkServer.sh[11047]: Starting zookeeper ... STARTED
Oct 11 20:19:30 ubuntu systemd[1]: Started Zookeeper Service.
重启后端服务,发现错误解决了。
3.预防
因为某些原因会导致服务异常关闭就会出现同样的问题,可以在重启服务器时可以将zookeeper下data和datalog 目录下的文件清除。
执行一下脚本
rm -rf /opt/zk1/zookeeper-3.4.10/data/*
rm -rf /opt/zk1/zookeeper-3.4.10/datalog/*
systemctl restart zk.service
除此以外还有其他情况会导致此问题
1.防火墙没有关闭。
ubuntu
centOS
2.端口被占用。
netstat -anp | grep 端口号
kill -9 id
3.等问题
更多推荐
ZK的Error contacting service. It is probably not running问题
发布评论