当我运行Dataflow作业时,它将使用我的小程序包(setup.py或requirements.txt)并将其上载以在Dataflow实例上运行.
When I run Dataflow job, it takes my small package (setup.py or requirements.txt) and uploads it to run on the Dataflow instances.
但是Dataflow实例上实际正在运行什么?我最近有一个堆栈跟踪:
But what is actually running on the Dataflow instance? I got a stacktrace recently:
File "/usr/lib/python2.7/httplib.py", line 1073, in _send_request self.endheaders(body) File "/usr/lib/python2.7/httplib.py", line 1035, in endheaders self._send_output(message_body) File "/usr/lib/python2.7/httplib.py", line 877, in _send_output msg += message_body TypeError: must be str, not unicode [while running 'write to datastore/Convert to Mutation']但是从理论上讲,如果我正在执行str += unicode,则表明我可能没有运行此 Python修补程序?您能否指向这些作业正在运行的docker映像,这样我才能知道我正在使用的Python版本,并确保我在这里没有打错树?
But in theory, if I'm doing str += unicode, it implies I might not be running this Python patch? Can you point to the docker image that these jobs are running, so I can know what version of Python I'm working with, and make sure I'm not barking up the wrong tree here?
云控制台向我显示了实例模板,该实例模板似乎指向 dataflow-dataflow-owned-resource-20170308-rc02 ,但似乎我无权查看.来源在线吗?
The cloud console shows me the instance template, which seems to point to dataflow-dataflow-owned-resource-20170308-rc02, but it seems I don't have permission to look at it. Is the source for it online anywhere?
推荐答案尚未测试(也许有一种更简单的方法),但是类似的方法可能会解决问题:
Haven't tested (and maybe there is an easier way), but something like this might do the trick:
从控制台然后,您应该找到想要的东西.
Then you should find what you are after.
更多推荐
Google Cloud Dataflow实例的图像
发布评论