redis cluster初探之部署

redis cluster初探之部署"/>

redis cluster初探之部署

一、简介

在3.0版本之前，redis通过哨兵实现主从的高可用，在3.0版本之后，redis官方推出了高可用的redis集群解决方案。

重点知识：数据分区
分布式数据库是将数据根据分区规则划分到多个节点上，每个节点负责存储一部分数据；常用的分区规则有哈希分区与顺序分区两种，两种方式各有优势，redis集群使用的是哈希分区的虚拟槽分区方式
redis集群采用的虚拟槽分区，所有的键通过计算（slot=CRC16（key）&16383）映射到0～16383整数槽位内，每一个节点负责一部分的槽位。

二、redis安装

前提：节点间公钥推送 互免密

2.1、配置ansible hosts

ansible所在节点：192.168.143.130

vim /etc/ansible/hosts 
[redis-cluster]
192.168.143.131
192.168.143.132

2.2、在ansible所在节点安装ruby环境

yum install rubygem-psych ruby-irb ruby-devel rubygem-rdoc rubygem-io-console rubygems rubygem-abrt rebygem-json ruby-libs rubygem-bigdecimal -y

2.3、在ansible所在节点下载redis

Redis：4.0.14
Redis-gem：3.3.5 /

2.4、在ansible所在节点安装redis-gem（redis的ruby客户端）

[root@node1 local]# gem install redis
Successfully installed redis-3.3.5
Parsing documentation for redis-3.3.5
Installing ri documentation for redis-3.3.5
1 gem installed

2.5、在ansible所在节点安装redis节点（集群模式）

解压

cd /usr/local/
tar -xzvf redis-4.0.14.tar.gz -C .

编译安装

cd redis-4.0.14/
make && make install

准备工作

mkdir redis-cluster/
cp redis-4.0.14/src/redis-server redis-cluster/
cp redis-4.0.14/src/redis-cli redis-cluster/

准备公共配置文件

vim redis-cluster/redis_comm.conf 
#集群相关的配置
cluster-enabled yes
cluster-node-timeout 15000
cluster-require-full-coverage no

在所有节点创建redis集群工作目录

ansible redis-cluster -m file -a 'path=/home/redis/redis_cluster/redis_node state=directory'

将redis-server和redis-cli放在集群各节点的/usr/bin目录下

ansible redis-cluster -m copy -a 'src=/usr/local/redis-cluster/redis-server dest=/usr/bin/redis-server mode=755'
ansible redis-cluster -m copy -a 'src=/usr/local/redis-cluster/redis-cli dest=/usr/bin/redis-cli mode=755'

将公共配置文件放在集群各节点的/home/redis/redis_cluster/redis_node下

ansible redis-cluster -m copy -a 'src=/usr/local/redis-cluster/redis_comm.conf dest=/home/redis/redis_cluster/redis_node/redis_comm.conf mode=644'`

创建redis节点脚本（redis_node_build.sh ）

#!/bin/bash
#
#Redis can open up to 16
redis_node=16
redis_workdir="/home/redis/redis_cluster/redis_node"
start_port=9001
redis_conf="redis.conf"
#Initialize the redis process
rm -rf ${redis_workdir}/90*
ps -ef|grep redis-server|grep -v grep
if [ $? -eq 0 ];thenkillall redis-serverecho "The node has been reset successfully"
elseecho "Node initialization succeeded"
Fi
#Modify kernel parameters: maximum listening queue length per port
conn_num=`sysctl -a|grep net.core.somaxconn|awk -F "=" '{print $2}'`
if [ $conn_num -ne 16384 ];thensed -i 's/net.core.somaxconn//g' /etc/sysctl.confecho "net.core.somaxconn=16384" >> /etc/sysctl.confsysctl -psysctl -w net.core.somaxconn=16384
elseecho "net.core.somaxconn = ${conn_num}"
fi
#Modify the maximum number of file descriptors that a process can open
echo 'ulimit -n 16384' >> /etc/profile
source /etc/profile
#Create a redis node
for ((i=0;i<$redis_node;i++));
donext_port=$(expr $start_port + $i)#Create an node directorymkdir -p $redis_workdir/$next_portcd $redis_workdir/$next_port#Prepare each node configuration fileecho "include $redis_workdir/redis_comm.conf" >> $redis_confecho "port ${next_port}" >> $redis_confecho "pidfile ${redis_workdir}/${next_port}/redis.pid" >> $redis_confecho "dir ${redis_workdir}/${next_port}" >> $redis_confecho "logfile ${redis_workdir}/${next_port}/redis.log" >> $redis_conf#Run the redis node/usr/bin/redis-server redis.conf
done

将创建redis节点的脚本放在集群各节点的/home/redis/redis_cluster/redis_node下

ansible redis-cluster -m copy -a 'src=/usr/local/redis-cluster/redis_node_build.sh dest=/home/redis/redis_cluster/redis_node/redis_node_build.sh  mode=755'

在各节点执行创建redis节点脚本

ansible redis-cluster -m shell -a 'cd /home/redis/redis_cluster/redis_node/;./redis_node_build.sh &> /dev/null'

查看进程

三、redis-cluster安装

3.1、准备

 cp /usr/local/redis-4.0.14/src/redis-trib.rb /usr/local/redis-cluster/

3.2、生成集群ip列表（全主集群）

cd /usr/local/redis-cluster/
vim redis_cluster_build.py
#!/usr/bin/python
#
ret = "/usr/local/redis-cluster/redis-trib.rb create --replicas 0 "
ip=open('./iplist-redis')
for i in ip:for j in range(16):ret += i.replace('\n','')+":"+str(9001+j)+" "
print ret

3.3、创建集群

/usr/local/redis-cluster/redis-trib.rb create --replicas 0 192.168.143.131:9001 192.168.143.131:9002 192.168.143.131:9003 192.168.143.131:9004 192.168.143.131:9005 192.168.143.131:9006 192.168.143.131:9007 192.168.143.131:9008 192.168.143.131:9009 192.168.143.131:9010 192.168.143.131:9011 192.168.143.131:9012 192.168.143.131:9013 192.168.143.131:9014 192.168.143.131:9015 192.168.143.131:9016 192.168.143.132:9001 192.168.143.132:9002 192.168.143.132:9003 192.168.143.132:9004 192.168.143.132:9005 192.168.143.132:9006 192.168.143.132:9007 192.168.143.132:9008 192.168.143.132:9009 192.168.143.132:9010 192.168.143.132:9011 192.168.143.132:9012 192.168.143.132:9013 192.168.143.132:9014 192.168.143.132:9015 192.168.143.132:9016 

Ps：create参数是redis-trib.rb命令创建集群的参数 --replicas 1参数意思是主从各占节点总数的一半，节点列表的第一个节点是master，第二个节点是第一个节点的slave，依次往后；--replicas 0参数为创建全master集群，意思是节点列表的所有节点都是master

3.4、查看集群状态

[root@node2 ~]# redis-cli -h 192.168.143.131 -p 9001
192.168.143.131:9001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:32
cluster_size:32
cluster_current_epoch:32
cluster_my_epoch:1
cluster_stats_messages_ping_sent:1198
cluster_stats_messages_pong_sent:1174
cluster_stats_messages_sent:2372
cluster_stats_messages_ping_received:1143
cluster_stats_messages_pong_received:1198
cluster_stats_messages_meet_received:31
cluster_stats_messages_received:2372

参考文章：
Redis-cluster docs：
replicas参数：
Redis-cluster command：
Redis-cluster-create scripts：

转载于: