在 docker 容器上运行一个 hadoop 集群

本文介绍了在 docker 容器上运行一个 hadoop 集群的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！ 问题描述

我想运行一个多节点 hadoop 集群，每个节点都在不同主机上的 docker 容器内.此图像 - github/sequenceiq/hadoop-docker 可以很好地开始hadoop 在伪分布式模式下，修改它以使每个节点位于单独的 ec2 主机上的不同容器中的最简单方法是什么?

I want to run a multi-node hadoop cluster, with each node inside a docker container on a different host. This image - github/sequenceiq/hadoop-docker works well to start hadoop in a pseudo distributed mode, what is the easiest way to modify this to have each node in a different container on a separate ec2 host?

推荐答案

我使用两个容器在两个不同的 ubuntu 主机上运行主节点和从节点.我使用 weave 在容器之间建立了网络.我已经在 docker hub 帐户 div4 上添加了容器的图像.我以同样的方式安装了 hadoop，因为它安装在不同的主机上.我在此处添加了带有 coomands 的两个图像以在它们上运行 haddop:

I did this with two containers running master and slave nodes on two different ubuntu hosts. I did the networking between containers using weave. I have added the images of the containers on docker hub account div4. I installed hadoop in the same way, as its installed on different hosts. I have added the two images with coomands to run haddop on them here:

registry.hub.docker/u/div4/hadoop_master/registry.hub.docker/u/div4/hadoop_slave/.