我希望能够创建EMR群集,并让这些群集将消息发送回中央队列。为了这个工作,我需要在每个主节点上运行某种代理。这些代理中的每一个都必须在这个消息中标识它自己,以便接收者知道该消息是关于哪个集群的。
主节点知道它是否是id(j-的 * ** * * )?如果没有,那么是否还有其他一些识别信息可以让消息接收者推断此ID?
我已经查看了/ home / hadoop / conf,我没有找到任何有用的东西。我在/mnt/var/log/instance-controller/instance-controller.log中发现了这个id,但看起来很难grep。我想知道哪里的实例控制器可能首先得到这个id。
解决方案您可以看看 / mnt / var / lib / info / 以查找有关您的EMR群集设置的大量信息。更具体地说, /mnt/var/lib/info/job-flow.json 包含jobFlowId或ClusterID。
您可以使用预先安装的json解析器( jq )来获取作业流ID。
cat /mnt/var/lib/info/job-flow.json | jq -r.jobFlowId(根据@Marboni更新)
I want to be able to create EMR clusters and for those clusters to send messages back to some central queue. In order for this to work, I need to have some sort of agent running on each master node. Each one of those agents will have to identify itself in this message so that the recipient knows which cluster the message is about.
Does the master node know it's id (j-*****)? If not, then is there some other piece of identifying information that could allow the message recipient to infer this id?
I've taken a look through the config files in /home/hadoop/conf, and I haven't found anything useful. I found the id in /mnt/var/log/instance-controller/instance-controller.log, but it looks like it'll be difficult to grep for. I'm wondering where instance-controller might get that id from in the first place.
解决方案You may look at /mnt/var/lib/info/ on Master node to find lot of info about your EMR cluster setup. More specifically /mnt/var/lib/info/job-flow.json contains the jobFlowId or ClusterID.
You can use the pre-installed json parser (jq) to get the jobflow id.
cat /mnt/var/lib/info/job-flow.json | jq -r ".jobFlowId"(updated as per @Marboni)
更多推荐
EMR主节点是否知道其集群ID?
发布评论