问题描述
限时送ChatGPT账号..我正在我的 Google Dataflow 程序中读取一堆配置文件,并想知道什么是暂存它们的最佳方式.目前我是这样做的,系统找不到它们.
I am reading a bunch configuration files in my Google Dataflow program and wonder what is the best way to stage them. Currently I do it this way and the system cannot find them.
FileReader filereader1 = new FileReader("config_1.csv");
FileReader filereader2 = new FileReader("config_2.csv");
config_1.csv
和 config_2.csv
存储在 ./target/classes/org/model/examples/
我的运行脚本如下所示:
My running script looks like this:
mvn compile exec:java -Dexec.mainClass=org.model.examples.MyPipeline \
-Dexec.args="--runner=DataflowRunner \
--project=mortgage-data-warehouse
--gcpTempLocation=gs://my-project-bucket/tmp \
--inputFile=gs://my-project-bucket/Data/input.txt \
--filesToStage=./target/classes/org/datamodel/examples/config_1.csv, ./target/classes/org/datamodel/examples/config_2.csv" \
-Pdataflow-runner
我有错误
java.io.FileNotFoundException: config_1.csv(系统找不到指定的文件)
java.io.FileNotFoundException: config_1.csv (The system cannot find the file specified)
我想知道这是否是设置 --filesToStage
的正确方法.
I wonder if this is the proper way to set --filesToStage
.
推荐答案
对于小的配置文件,最好从资源文件夹中读取文件,例如本link 并避免使用 --filesToStage
For small configuration files, it is better to read files from resource folder such as what has been written by this link and avoid the complication of using --filesToStage
这篇关于如何使用 Google Cloud Dataflow 暂存其他文件?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
更多推荐
[db:关键词]
发布评论