小丸子学Hadoop系列之——部署Hbase集群

集群"/>

小丸子学Hadoop系列之——部署Hbase集群

0.集群规划
主机名 ip地址 安装的软件 运行的进程
AI-OPT-HBS01 10.46.52.30 hadoop,hbase namenode,zkfc,resourcemanager
AI-OPT-HBS02 10.46.52.31 hadoop namenode,zkfc,resourcemanager
AI-OPT-HBS03 10.46.52.32 hadoop,hbase datanode
AI-OPT-HBS04 10.46.52.33 hadoop,zookeeper,hbase datanode,nodemanager,journalnode
AI-OPT-HBS05 10.46.52.34 hadoop,zookeeper,hbase datanode,nodemanager,journalnode
AI-OPT-HBS06 10.46.52.35 hadoop,zookeeper,hbase datanode,nodemanager,journalnode


1.安装1.创建用户以及配置主机名
mkdir -p /aifs01/users
useradd -u 351   -g hadoop -G ibss -d /aifs01/users/hdpusr01 -m hdpusr01
passwd hdpusr01     --stdin <<< Linuhdp_0805
echo -e "\n. ~puwadm/wprofile\n" >> ~hdpusr01/.bash_profile 

useradd -u 352   -g hbase -G ibss -d /aifs01/users/hbsusr01 -m hbsusr01
passwd hbsusr01     --stdin <<< Linuhbs_0805
echo -e "\n. ~puwadm/wprofile\n" >> ~hbsusr01/.bash_profile 

vi /etc/hosts            
10.46.52.30 AI-OPT-HBS01
10.46.52.31 AI-OPT-HBS02
10.46.52.32 AI-OPT-HBS03
10.46.52.33 AI-OPT-HBS04
10.46.52.34 AI-OPT-HBS05
10.46.52.35 AI-OPT-HBS06

2.安装Hadoop集群
具体步骤请参考《小丸子学Hadoop系列之——部署Hadoop集群》
/

3.安装HBase集群
3.1 安装HBase集群（AI-OPT-HBS01 - AI-OPT-HBS06）
--选择一台主机操作（AI-OPT-HBS01）
[root@AI-OPT-HBS01 hbsusr01]# tar xvf /opt/software/hbase-1.1.2.tgz
[root@AI-OPT-HBS01 hbsusr01]# mv hbase-1.1.2 hbase
[root@AI-OPT-HBS01 hbsusr01]# chown -R hbsusr01:hbase hbase

--设置集群节点文件
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01/hbase/conf> vi regionservers
AI-OPT-HBS03
AI-OPT-HBS04
AI-OPT-HBS05
AI-OPT-HBS06

--设置hbase配置文件
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01/hbase/conf> vi hbase-site.xml 
     1:
     2:
     3:         hbase.rootdir
     4:         hdfs://AI-OPT-HBS01:8920/hbase
     5:
     6:
     7:
     8:         hbase.master.port
     9:         60900
    10:
    11:
    12:
    13:         hbase.regionserver.port
    14:         60920
    15:
    16:
    17:
    18:         hbase.cluster.distributed
    19:         true
    20:
    21:
    22:
    23:         hbase.zookeeper.quorum
    24:         AI-OPT-HBS04,AI-OPT-HBS05,AI-OPT-HBS06
    25:
    26:
    27:
    28:         hbase.tmp.dir
    29:         /aifs01/users/hbsusr01/hbase/tmp
    30:
    31:
    32:
    33:         hbase.zookeeper.peerport
    34:         29888
    35:
    36:
    37:
    38:         hbase.zookeeper.leaderport
    39:         39888
    40:
    41:
    42:
    43:         hbase.zookeeper.property.clientPort
    44:         29181
    45:
    46:
    47:
    48:         hbase.rest.port
    49:         8980
    50:
    51:

--设置hbase环境
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01/hbase/conf> vi hbase-env.sh 
export HBASE_CONF_DIR=/aifs01/users/hbsusr01/hbase/conf
export HBASE_PID_DIR=/aifs01/users/hbsusr01/hbase/pid
export HBASE_LOG_DIR=/aifs01/users/hbsusr01/hbase/logs

# export HBASE_MANAGES_ZK=true
export HBASE_MANAGES_ZK=false

--创建相关目录
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> mkdir -p hbase/{conf,pid,logs,tmp}

--拷贝文件到其他节点并解压
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> tar cvf hbase.tar hbase
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS03:/aifs01/users/hbsusr01
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS04:/aifs01/users/hbsusr01
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS05:/aifs01/users/hbsusr01
hbsusr01@AI-OPT-HBS01:/aifs01/users/hbsusr01> scp hbase.tar AI-OPT-HBS06:/aifs01/users/hbsusr01
                    
--配置ssh互信
su - hbsusr01
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
chmod 600 .ssh/authorized_keys

--选择一台master操作(AI-OPT-HBS01)
ssh AI-OPT-HBS02 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS03 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS04 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS05 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 
ssh AI-OPT-HBS06 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 

scp ~/.ssh/authorized_keys AI-OPT-HBS02:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS03:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS04:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS05:/aifs01/users/hbsusr01/.ssh/authorized_keys
scp ~/.ssh/authorized_keys AI-OPT-HBS06:/aifs01/users/hbsusr01/.ssh/authorized_keys

--验证配置是否生效
ssh AI-OPT-HBS01 date
ssh AI-OPT-HBS02 date
ssh AI-OPT-HBS03 date
ssh AI-OPT-HBS04 date
ssh AI-OPT-HBS05 date
ssh AI-OPT-HBS06 date

--解压hadoop包（AI-OPT-HBS03 - AI-OPT-HBS06）
tar xvf hbase.tar

--启动hbase
start-hbase.sh
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=hbsusr01, access=WRITE, inode="/":hdpusr01:supergroup:drwxr-xr-x

解决方法：hadoop fs添加目录和权限
hdpusr01@AI-OPT-HBS01:/aifs01/users/hdpusr01/hadoop/bin> ./hadoop  fs -mkdir /hbase
hdpusr01@AI-OPT-HBS01:/aifs01/users/hdpusr01/hadoop/bin> ./hadoop fs -chown hbsusr01:hbase /hbase

至此,Hbase集群搭建完毕。

来自 “ ITPUB博客 ” ，链接：/，如需转载，请注明出处，否则将追究法律责任。

转载于:/