我的 Spark 的 Worker 无法连接 Master.Akka 有问题吗?

编程入门行业动态更新时间:2024-10-10 23:17:10

本文介绍了我的 Spark 的 Worker 无法连接 Master.Akka 有问题吗?的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！问题描述

我想将 Spark Standlone 模式安装到我的两个虚拟机的集群中.使用 spark-0.9.1-bin-hadoop1 版本，我在每个 vm 中都成功执行了 spark-shell.我按照官方文档制作了一个vm(ip:xx.xx.xx.223) 作为 Master 和 Worker 并使另一个 (ip:xx.xx.xx.224) 仅作为 Worker.但是224-ip vm无法连接223-ip vm.以下是223(Master)的主日志:

I want to install Spark Standlone mode to a Cluster with my two virtual machines. With the version of spark-0.9.1-bin-hadoop1, I execute spark-shell successfully in each vm. I follow the offical document to make one vm(ip:xx.xx.xx.223) as both Master and Worker and to make the other(ip:xx.xx.xx.224) as Worker only. But the 224-ip vm cannot connect the 223-ip vm. Followed is 223(Master)'s master log:

[@tc-52-223 logs]# tail -100f spark-root-org.apache.spark.deploy.master.Master-1-tc-52-223.out Spark Command: /usr/local/jdk/bin/java -cp :/data/test/spark-0.9.1-bin-hadoop1/conf:/data/test/spark-0.9.1-bin-hadoop1/assembly/target/scala-2.10/spark-assembly_2.10-0.9.1-hadoop1.0.4.jar -Dspark.akka.logLifecycleEvents=true -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.master.Master --ip 10.11.52.223 --port 7077 --webui-port 8080 log4j:WARN No appenders could be found for logger (akka.event.slf4j.Slf4jLogger). log4j:WARN Please initialize the log4j system properly. log4j:WARN See logging.apache/log4j/1.2/faq.html#noconfig for more info. 14/04/14 22:17:03 INFO Master: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 14/04/14 22:17:03 INFO Master: Starting Spark master at spark://10.11.52.223:7077 14/04/14 22:17:03 INFO MasterWebUI: Started Master web UI at tc-52-223:8080 14/04/14 22:17:03 INFO Master: I have been elected leader! New state: ALIVE 14/04/14 22:17:06 INFO Master: Registering worker tc-52-223:20599 with 1 cores, 4.0 GB RAM 14/04/14 22:17:06 INFO Master: Registering worker tc_52_224:21371 with 1 cores, 4.0 GB RAM 14/04/14 22:17:06 INFO RemoteActorRefProvider$RemoteDeadLetterActorRef: Message [org.apache.spark.deploy.DeployMessages$RegisteredWorker] from Actor[akka://sparkMaster/user/Master#1972530850] to Actor[akka://sparkMaster/deadLetters] was not delivered. [1] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 14/04/14 22:17:26 INFO Master: Registering worker tc_52_224:21371 with 1 cores, 4.0 GB RAM 14/04/14 22:17:26 INFO RemoteActorRefProvider$RemoteDeadLetterActorRef: Message [org.apache.spark.deploy.DeployMessages$RegisterWorkerFailed] from Actor[akka://sparkMaster/user/Master#1972530850] to Actor[akka://sparkMaster/deadLetters] was not delivered. [2] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 14/04/14 22:17:46 INFO Master: Registering worker tc_52_224:21371 with 1 cores, 4.0 GB RAM 14/04/14 22:17:46 INFO RemoteActorRefProvider$RemoteDeadLetterActorRef: Message [org.apache.spark.deploy.DeployMessages$RegisterWorkerFailed] from Actor[akka://sparkMaster/user/Master#1972530850] to Actor[akka://sparkMaster/deadLetters] was not delivered. [3] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 14/04/14 22:18:06 INFO Master: akka.tcp://sparkWorker@tc_52_224:21371 got disassociated, removing it. 14/04/14 22:18:06 INFO Master: akka.tcp://sparkWorker@tc_52_224:21371 got disassociated, removing it. 14/04/14 22:18:06 INFO LocalActorRef: Message [akka.remote.transport.ActorTransportAdapter$DisassociateUnderlying] from Actor[akka://sparkMaster/deadLetters] to Actor[akka://sparkMaster/system/transports/akkaprotocolmanager.tcp0/akkaProtocol-tcp%3A%2F%2FsparkMaster%4010.11.52.224%3A61550-1#646150938] was not delivered. [4] dead letters encountered. This logging can be turned off or adjusted with configuration settings 'akka.log-dead-letters' and 'akka.log-dead-letters-during-shutdown'. 14/04/14 22:18:06 INFO Master: akka.tcp://sparkWorker@tc_52_224:21371 got disassociated, removing it. 14/04/14 22:18:06 ERROR EndpointWriter: AssociationError [akka.tcp://sparkMaster@10.11.52.223:7077] -> [akka.tcp://sparkWorker@tc_52_224:21371]: Error [Association failed with [akka.tcp://sparkWorker@tc_52_224:21371]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkWorker@tc_52_224:21371] Caused by: akka.remote.transportty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: tc_52_224/10.11.52.224:21371 ] 14/04/14 22:18:06 INFO Master: akka.tcp://sparkWorker@tc_52_224:21371 got disassociated, removing it. 14/04/14 22:18:06 ERROR EndpointWriter: AssociationError [akka.tcp://sparkMaster@10.11.52.223:7077] -> [akka.tcp://sparkWorker@tc_52_224:21371]: Error [Association failed with [akka.tcp://sparkWorker@tc_52_224:21371]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkWorker@tc_52_224:21371] Caused by: akka.remote.transportty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: tc_52_224/10.11.52.224:21371 ] 14/04/14 22:18:06 ERROR EndpointWriter: AssociationError [akka.tcp://sparkMaster@10.11.52.223:7077] -> [akka.tcp://sparkWorker@tc_52_224:21371]: Error [Association failed with [akka.tcp://sparkWorker@tc_52_224:21371]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkWorker@tc_52_224:21371] Caused by: akka.remote.transportty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: tc_52_224/10.11.52.224:21371 ] 14/04/14 22:18:06 INFO Master: akka.tcp://sparkWorker@tc_52_224:21371 got disassociated, removing it. 14/04/14 22:19:03 WARN Master: Removing worker-20140414221705-tc_52_224-21371 because we got no heartbeat in 60 seconds 14/04/14 22:19:03 INFO Master: Removing worker worker-20140414221705-tc_52_224-21371 on tc_52_224:21371

以下是 223(Worker) 的工作日志:

Followed is 223(Worker)'s worker log:

14/04/14 22:17:06 INFO Worker: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 14/04/14 22:17:06 INFO Worker: Starting Spark worker tc-52-223:20599 with 1 cores, 4.0 GB RAM 14/04/14 22:17:06 INFO Worker: Spark home: /data/test/spark-0.9.1-bin-hadoop1 14/04/14 22:17:06 INFO WorkerWebUI: Started Worker web UI at tc-52-223:8081 14/04/14 22:17:06 INFO Worker: Connecting to master spark://xx.xx.52.223:7077... 14/04/14 22:17:06 INFO Worker: Successfully registered with master spark://xx.xx.52.223:7077

以下是224(Worker)的工作日志:

Followed is 224(Worker)'s work log:

Spark Command: /usr/local/jdk/bin/java -cp :/data/test/spark-0.9.1-bin-hadoop1/conf:/data/test/spark-0.9.1-bin-hadoop1/assembly/target/scala-2.10/spark-assembly_2.10-0.9.1-hadoop1.0.4.jar -Dspark.akka.logLifecycleEvents=true -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.worker.Worker spark://10.11.52.223:7077 --webui-port 8081 ======================================== log4j:WARN No appenders could be found for logger (akka.event.slf4j.Slf4jLogger). log4j:WARN Please initialize the log4j system properly. log4j:WARN See logging.apache/log4j/1.2/faq.html#noconfig for more info. 14/04/14 22:17:06 INFO Worker: Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 14/04/14 22:17:06 INFO Worker: Starting Spark worker tc_52_224:21371 with 1 cores, 4.0 GB RAM 14/04/14 22:17:06 INFO Worker: Spark home: /data/test/spark-0.9.1-bin-hadoop1 14/04/14 22:17:06 INFO WorkerWebUI: Started Worker web UI at tc_52_224:8081 14/04/14 22:17:06 INFO Worker: Connecting to master spark://xx.xx.52.223:7077... 14/04/14 22:17:26 INFO Worker: Connecting to master spark://xx.xx.52.223:7077... 14/04/14 22:17:46 INFO Worker: Connecting to master spark://xx.xx.52.223:7077... 14/04/14 22:18:06 ERROR Worker: All masters are unresponsive! Giving up.

以下是我的 spark-env.sh:

Followed is my spark-env.sh:

JAVA_HOME=/usr/local/jdk export SPARK_MASTER_IP=tc-52-223 export SPARK_WORKER_CORES=1 export SPARK_WORKER_INSTANCES=1 export SPARK_MASTER_PORT=7077 export SPARK_WORKER_MEMORY=4g export MASTER=spark://${SPARK_MASTER_IP}:${SPARK_MASTER_PORT} export SPARK_LOCAL_IP=tc-52-223

我在谷歌上搜索了很多解决方案，但都无法奏效.请帮帮我.

I have googled many solutions, but they cant work. Please help me.

推荐答案

我不确定这是否与我遇到的问题相同，但您可能想尝试将 SPARK_MASTER_IP 设置为与什么 spark 相同绑定到.在您的示例中，它看起来像是 10.11.52.223 而不是 tc-52-223.

I'm not sure if this is the same issue I encountered but you may want to try setting SPARK_MASTER_IP the same as what spark binds to. In your example is looks like it would be 10.11.52.223 and not tc-52-223.

它应该与您在 8080 上访问主节点 Web UI 时看到的相同.例如:Spark Master at spark://ec2-XX-XX-XXX-XXXpute-1.amazonaws:7077

It should be the same as what you see when you visit the master node web UI on 8080. Something like: Spark Master at spark://ec2-XX-XX-XXX-XXXpute-1.amazonaws:7077