Redis 集群搭建

柔光的暖阳◎ 2022-02-14 00:37 182阅读 0赞

**1、搭建环境**

centos 7.6、redis-5.0.4；

说明：redis 集群要求至少需要 3个redis主节点，每个主节点有1个从节点，共6个redis实例。在一台服务器搭建redis集群，通过端口区分各个redis实例；

[root@syztoo local]# cat /etc/redhat-release 
    CentOS Linux release 7.6.1810 (Core) 
    [root@syztoo local]# ls
    aegis  bin  etc  games  include  java  lib  lib64  libexec  maven  nginx  python3  redis-5.0.4.tar.gz  sbin  share  src  tomcat

**2、安装redis**

[root@syztoo local]# tar -zxvf redis-5.0.4.tar.gz
    [root@syztoo local]# mv redis-5.0.4 redis
    [root@syztoo local]# cd redis
    [root@syztoo redis]# make PREFIX=/usr/local/redis install

**3.创建各节点的redis实例**

创建各节点的主目录：

[root@syztoo redis]# mkdir -pv /usr/local/redis-cluster/{7001,7002,7003,7004,7005,7006}
    mkdir: created directory ‘/usr/local/redis-cluster’
    mkdir: created directory ‘/usr/local/redis-cluster/7001’
    mkdir: created directory ‘/usr/local/redis-cluster/7002’
    mkdir: created directory ‘/usr/local/redis-cluster/7003’
    mkdir: created directory ‘/usr/local/redis-cluster/7004’
    mkdir: created directory ‘/usr/local/redis-cluster/7005’
    mkdir: created directory ‘/usr/local/redis-cluster/7006’

将 redis-server 和 redis.conf 文件复制到各节点的主目录下：

[root@syztoo redis]# echo /usr/local/redis-cluster/{7001,7002,7003,7004,7005,7006} | xargs -n 1 cp -v redis.conf src/redis-server
    ‘redis.conf’ -> ‘/usr/local/redis-cluster/7001/redis.conf’
    ‘src/redis-server’ -> ‘/usr/local/redis-cluster/7001/redis-server’
    ‘redis.conf’ -> ‘/usr/local/redis-cluster/7002/redis.conf’
    ‘src/redis-server’ -> ‘/usr/local/redis-cluster/7002/redis-server’
    ‘redis.conf’ -> ‘/usr/local/redis-cluster/7003/redis.conf’
    ‘src/redis-server’ -> ‘/usr/local/redis-cluster/7003/redis-server’
    ‘redis.conf’ -> ‘/usr/local/redis-cluster/7004/redis.conf’
    ‘src/redis-server’ -> ‘/usr/local/redis-cluster/7004/redis-server’
    ‘redis.conf’ -> ‘/usr/local/redis-cluster/7005/redis.conf’
    ‘src/redis-server’ -> ‘/usr/local/redis-cluster/7005/redis-server’
    ‘redis.conf’ -> ‘/usr/local/redis-cluster/7006/redis.conf’
    ‘src/redis-server’ -> ‘/usr/local/redis-cluster/7006/redis-server’

将redis-cli 复制到 redis-cluster 目录下：

[root@syztoo redis]# cp src/redis-cli /usr/local/redis-cluster/

**4、修改各redis实例配置文件**

[root@syztoo redis]# cd ../redis-cluster
    [root@syztoo redis-cluster]# sed -i -e '/^daemonize/s/no/yes/' -e '/port 6379/c\port 7001' -e '/^# cluster-enabled/s/# //' 7001/redis.conf
    [root@syztoo redis-cluster]# sed -i -e '/^daemonize/s/no/yes/' -e '/port 6379/c\port 7002' -e '/^# cluster-enabled/s/# //' 7002/redis.conf
    [root@syztoo redis-cluster]# sed -i -e '/^daemonize/s/no/yes/' -e '/port 6379/c\port 7003' -e '/^# cluster-enabled/s/# //' 7003/redis.conf
    [root@syztoo redis-cluster]# sed -i -e '/^daemonize/s/no/yes/' -e '/port 6379/c\port 7004' -e '/^# cluster-enabled/s/# //' 7004/redis.conf
    [root@syztoo redis-cluster]# sed -i -e '/^daemonize/s/no/yes/' -e '/port 6379/c\port 7005' -e '/^# cluster-enabled/s/# //' 7005/redis.conf
    [root@syztoo redis-cluster]# sed -i -e '/^daemonize/s/no/yes/' -e '/port 6379/c\port 7006' -e '/^# cluster-enabled/s/# //' 7006/redis.conf

**5、启动各redis实例**

注意：启动各redis实例，要cd到各redis实例主目录下操作，不然服务起不起来；可编辑一键启动脚本，并给予执行权限。

[root@syztoo redis-cluster]# cat startup.sh 
    #!/bin/bash
    
    cd 7001
    ./redis-server redis.conf > /dev/null 2>&1
    cd ../7002
    ./redis-server redis.conf > /dev/null 2>&1
    cd ../7003
    ./redis-server redis.conf > /dev/null 2>&1
    cd ../7004
    ./redis-server redis.conf > /dev/null 2>&1
    cd ../7005
    ./redis-server redis.conf > /dev/null 2>&1
    cd ../7006
    ./redis-server redis.conf > /dev/null 2>&1

[root@syztoo redis-cluster]# chmod +x startup.sh

[root@syztoo redis-cluster]# ./startup.sh
    [root@syztoo redis-cluster]# ps -ef | grep redis
    root     30805     1  0 16:22 ?        00:00:00 ./redis-server 127.0.0.1:7001 [cluster]
    root     30810     1  0 16:22 ?        00:00:00 ./redis-server 127.0.0.1:7002 [cluster]
    root     30815     1  0 16:22 ?        00:00:00 ./redis-server 127.0.0.1:7003 [cluster]
    root     30820     1  0 16:22 ?        00:00:00 ./redis-server 127.0.0.1:7004 [cluster]
    root     30825     1  0 16:22 ?        00:00:00 ./redis-server 127.0.0.1:7005 [cluster]
    root     30830     1  0 16:22 ?        00:00:00 ./redis-server 127.0.0.1:7006 [cluster]
    root     30835 26407  0 16:23 pts/0    00:00:00 grep --color=auto redis

**6、创建redis集群**

[root@syztoo redis-cluster]# ./redis-cli --cluster create 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006 --cluster-replicas 1
    >>> Performing hash slots allocation on 6 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
    Adding replica 127.0.0.1:7006 to 127.0.0.1:7002
    Adding replica 127.0.0.1:7004 to 127.0.0.1:7003
    >>> Trying to optimize slaves allocation for anti-affinity
    [WARNING] Some slaves are in the same host as their master
    M: b72a202e2c55b5084d7cacc98455ea467f45ac4f 127.0.0.1:7001
       slots:[0-5460] (5461 slots) master
    M: d3b0a4facc3f62a3fe4b36edc02f1e3aa645cb8c 127.0.0.1:7002
       slots:[5461-10922] (5462 slots) master
    M: 8e4c69be01c192330ad6895735ec22fa2040282d 127.0.0.1:7003
       slots:[10923-16383] (5461 slots) master
    S: 9d5a2086a775a166da2afb36adbd6cda0226297a 127.0.0.1:7004
       replicates 8e4c69be01c192330ad6895735ec22fa2040282d
    S: 5bfbbe1e21ac0d32662a0b7d147b6481d780c95b 127.0.0.1:7005
       replicates b72a202e2c55b5084d7cacc98455ea467f45ac4f
    S: 59153f5129ec95a6fd0d86081b3840d3f79bf753 127.0.0.1:7006
       replicates d3b0a4facc3f62a3fe4b36edc02f1e3aa645cb8c
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    ......
    >>> Performing Cluster Check (using node 127.0.0.1:7001)
    M: b72a202e2c55b5084d7cacc98455ea467f45ac4f 127.0.0.1:7001
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    M: 8e4c69be01c192330ad6895735ec22fa2040282d 127.0.0.1:7003
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
    S: 59153f5129ec95a6fd0d86081b3840d3f79bf753 127.0.0.1:7006
       slots: (0 slots) slave
       replicates d3b0a4facc3f62a3fe4b36edc02f1e3aa645cb8c
    M: d3b0a4facc3f62a3fe4b36edc02f1e3aa645cb8c 127.0.0.1:7002
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    S: 9d5a2086a775a166da2afb36adbd6cda0226297a 127.0.0.1:7004
       slots: (0 slots) slave
       replicates 8e4c69be01c192330ad6895735ec22fa2040282d
    S: 5bfbbe1e21ac0d32662a0b7d147b6481d780c95b 127.0.0.1:7005
       slots: (0 slots) slave
       replicates b72a202e2c55b5084d7cacc98455ea467f45ac4f
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

至此，redis集群创建成功，显示7001，7002，7003端口下的redis实例为主节点，7004，7005，7006端口下的redis实例为从节点；

**7、关于redis集群的端口**

[root@syztoo redis-cluster]# netstat -tulnp
    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
    tcp        0      0 127.0.0.1:17001         0.0.0.0:*               LISTEN      30805/./redis-serve 
    tcp        0      0 127.0.0.1:17002         0.0.0.0:*               LISTEN      30810/./redis-serve 
    tcp        0      0 127.0.0.1:17003         0.0.0.0:*               LISTEN      30815/./redis-serve 
    tcp        0      0 127.0.0.1:17004         0.0.0.0:*               LISTEN      30820/./redis-serve 
    tcp        0      0 127.0.0.1:17005         0.0.0.0:*               LISTEN      30825/./redis-serve 
    tcp        0      0 127.0.0.1:17006         0.0.0.0:*               LISTEN      30830/./redis-serve 
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      3194/sshd           
    tcp        0      0 127.0.0.1:7001          0.0.0.0:*               LISTEN      30805/./redis-serve 
    tcp        0      0 127.0.0.1:7002          0.0.0.0:*               LISTEN      30810/./redis-serve 
    tcp        0      0 127.0.0.1:7003          0.0.0.0:*               LISTEN      30815/./redis-serve 
    tcp        0      0 127.0.0.1:7004          0.0.0.0:*               LISTEN      30820/./redis-serve 
    tcp        0      0 127.0.0.1:7005          0.0.0.0:*               LISTEN      30825/./redis-serve 
    tcp        0      0 127.0.0.1:7006          0.0.0.0:*               LISTEN      30830/./redis-serve 
    udp        0      0 0.0.0.0:68              0.0.0.0:*                           2751/dhclient       
    udp        0      0 127.0.0.1:323           0.0.0.0:*                           2470/chronyd        
    udp6       0      0 ::1:323                 :::*                                2470/chronyd

每个Redis集群中的节点都需要打开两个TCP连接。一个连接用于正常的给Client提供服务，比如6379，还有一个额外的端口（通过在这个端口号上加10000）作为数据端口，比如16379。额外的端口用于集群总线，这是一个用二进制协议的点对点通信信道。这个集群总线（Cluster bus）用于节点的失败侦测、配置更新、故障转移授权，等等。客户端从来都不应该尝试和这些集群总线端口通信，它们只应该和正常的Redis命令端口进行通信。注意，确保在你的防火墙中开放着两个端口，否则，Redis集群节点之间将无法通信。

命令端口和集群总线端口的偏移量总是10000。

注意，要想集群正常工作，集群中的每个节点需要做到以下两点：

> 1.  正常的客户端通信端口（通常是6379）必须对所有的客户端都开放，换言之，所有的客户端都可以访问；
> 2.  集群总线端口（客户端通信端口 + 10000）必须对集群中的其它节点开放，换言之，其它任意节点都可以访问；

**8、关于redis集群配置文件参数**

>  *  **cluster-enabled <yes/no>:** 如果是yes，表示启用集群，否则以单例模式启动；
>  *  **cluster-config-file <filename>:** 可选，这不是一个用户可编辑的配置文件，这个文件是Redis集群节点自动持久化每次配置的改变，为了在启动的时候重新读取它；
>  *  **cluster-node-timeout <milliseconds>:** 超时时间，集群节点不可用的最大时间。如果一个master节点不可到达超过了指定时间，则认为它失败了。注意，每一个在指定时间内不能到达大多数master节点的节点将停止接受查询请求；
>  *  **cluster-slave-validity-factor <factor>:** 如果设置为0，则一个slave将总是尝试故障转移一个master。如果设置为一个正数，那么最大失去连接的时间是node timeout乘以这个factor；
>  *  **cluster-migration-barrier <count>:** 一个master和slave保持连接的最小数量（即：最少与多少个slave保持连接），也就是说至少与其它多少slave保持连接的slave才有资格成为master；
>  *  **cluster-require-full-coverage <yes/no>:** 如果设置为yes，这也是默认值，如果key space没有达到百分之多少时停止接受写请求。如果设置为no，将仍然接受查询请求，即使它只是请求部分key；