YARN集群模式下Spark容器的入口点是什么？(What is the entry point of Spark container in YARN cluster mode?)

YARN集群模式下Spark容器的入口点是什么？(What is the entry point of Spark container in YARN cluster mode?)

在Yarn集群模式（Spark 1.2.0+）中运行Spark作业时，Spark执行程序的主要入口点是什么？

我正在寻找的是Executor入口点的Scala类名（它将是执行从属机器上的任务之一的进程）。

What is the main entry point of a Spark executor when a Spark job is being run in Yarn cluster mode (for Spark 1.2.0+)?

What I am looking for is the Scala class name for the entry point of an Executor (which will be the process executing one of the tasks on a slave machine).

最满意答案

我想你要问的是org.apache.spark.executor.Executor或者org.apache.spark.executor.Executor $ TaskRunner 。 最终将运行任务的是TaskRunner 。

它与部署模式（ client与cluster ）或使用的集群管理器无关，即Hadoop YARN或Spark Standalone或Apache Mesos。

I think what you're asking about is org.apache.spark.executor.Executor or perhaps org.apache.spark.executor.Executor$TaskRunner. It is TaskRunner that will ultimately run a task.

It is regardless of the deploy mode (client vs cluster) or the cluster manager used, i.e. Hadoop YARN or Spark Standalone or Apache Mesos.