问题描述
限时送ChatGPT账号..我正在尝试在独立的 Flink 集群上运行 Python Flink 应用程序.该应用程序在单节点集群上运行良好,但在多节点集群上会引发以下错误.java.lang.Exception: 用户定义的open()"方法导致异常:复制文件时出错
.请帮我解决这个问题.谢谢
I am trying to run a Python Flink Application on the standalone Flink cluster. The application works fine on a single node cluster but it throws the following error on a multi-node cluster. java.lang.Exception: The user defined 'open()' method caused an exception: An error occurred while copying the file
. Please help me resolve this problem. Thank you
我尝试执行的应用程序具有以下代码.
The application I am trying to execute has the following code.
from flink.plan.Environment import get_environment
from flink.plan.Constants import INT, STRING, WriteMode
env = get_environment()
data = env.from_elements("Hello")
data.map(lambda x: list(x)).output()
env.execute()
推荐答案
您必须在flink-conf.yaml"中配置python.dc.tmp.dir"以指向分布式文件系统(如 HDFS).该目录用于分发python脚本.
You have to configure "python.dc.tmp.dir" in "flink-conf.yaml" to point to a distributed filesystem (like HDFS). This directory is used to distributed the python scripts.
这篇关于无法在集群上运行 python flink 应用程序的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
更多推荐
[db:关键词]
发布评论