使用docker部署redis-sentinel 和redis-cluster

亦凉 2022-03-15 03:40 262阅读 0赞

写这篇文章的目的并不是为了介绍redis以及它的高可用方案,主要是通过redis各种高可用方案的简单安装,让读者能够通过真实的集群环境更好的体会redis的高可用方案。对redis感兴趣的同学经常会遇到这样的一个问题，没有足够的虚拟机,因此有时候想要自己部署redis各种集群环境时总是显得力不从心,而使用官方的安装包又非常麻烦,因此本文利用docker-compose来快速安装redis，当然，前提是你要会简单使用docker和docker-compose哦。

话不多说，咱们直接进入正题,下面所有的操作都是在我的一台虚拟机(192.168.174.129)上完成的,读者在使用时可以把配置中涉及到的ip改成自己的实际ip.

# **        一、单实例的redis** #

创建  /home/ubuntu/redis/standalone/docker-compose.yml，如下

version: '2'
    services:
      redis:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6379:6379

$: cd /home/ubuntu/redis/standalone/

$: docker-compose up -d

然后,一个redis实例就启动好了,就是这么easy。此时只需要使用你的redis client访问192.168.174.129的6379端口就可以愉快的使用redis了，可以简单的验证一下,如下,还是在/home/ubuntu/redis/standalone目录下执行如下命令

$: docker-compose  exec redis redis-cli

进入redis-cli的命令行，然后就可以在里面验证各种redis命令了

![20190228182423551.png][]

# **        二、redis一主多从(读写分离)** #

创建  /home/ubuntu/redis/repl/docker-compose.yml，如下

version: '2'
    services:
      master:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6380:6379
    
      slave-1:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6381:6379
        command: ["redis-server","--slaveof","192.168.174.129","6380"]
    
      slave-2:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6382:6379
        command: ["redis-server","--slaveof","192.168.174.129","6380"]
    
      slave-3:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6383:6379
        command: ["redis-server","--slaveof","192.168.174.129","6382"]

注：slave-3是slave-2的从服务器,slave-2和slave-1是master的从服务器,这样举例主要是为了体现嵌套树状复制结构 。

# **    三、redis-sentinel** #

**  **创建  /home/ubuntu/redis/sentinel/docker-compose.yml，如下

version: '2'
    services:
      repl-0:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6380:6379       
        networks:
          - sentinel
        command: ["redis-server","--replica-announce-ip","192.168.174.129","--replica-announce-port","6380"]
    
      repl-1:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6381:6379       
        networks:
          - sentinel
        command: ["redis-server","--slaveof","repl-0","6379","--replica-announce-ip","192.168.174.129","--replica-announce-port","6381"]
    
      repl-2:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6382:6379       
        networks:
          - sentinel
        command: ["redis-server","--slaveof","repl-0","6379","--replica-announce-ip","192.168.174.129","--replica-announce-port","6382"]
    
      repl-3:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        ports:
          - 6383:6379       
        networks:
          - sentinel
        ports:
          - 6383:6379
        command: ["redis-server","--slaveof","repl-2","6379","--replica-announce-ip","192.168.174.129","--replica-announce-port","6383"]
        
      sentinel-1:
        build: ./build
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        networks:
          - sentinel 
        ports:
          - 26380:26379
        command: ["redis-server","/data/sentinel.conf","--sentinel","announce-ip","192.168.174.129","--sentinel","announce-port" , "26380"]
    
      sentinel-2:
        build: ./build
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        networks:
          - sentinel 
        ports:
          - 26381:26379
        command: ["redis-server","/data/sentinel.conf","--sentinel","announce-ip","192.168.174.129","--sentinel","announce-port" , "26381"]
        
      sentinel-3:
        build: ./build
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
        networks:
          - sentinel  
        ports:
          - 26382:26379
        command: ["redis-server","/data/sentinel.conf","--sentinel","announce-ip","192.168.174.129","--sentinel","announce-port" , "26382"]
    
    networks:
      sentinel: 
         driver: bridge

注：与非docker环境相比,我们添加了announce-ip和announce-port这些命令,这是因为默认情况下主从服务器之间以及各个sentinel实例之间使用的ip是容器内部自动分配的ip,这样在切换主从服务或sentinel之间通信时无法联通，因此务必需要添加对应的命令。

同时在/home/ubuntu/redis/sentinel/build目录下创建 Dockerfile和sentinel.conf

**Dockerfile:**

FROM redis:5.0-rc-alpine
    COPY ./sentinel.conf /data/sentinel.conf
    EXPOSE 26379

**sentinel.conf**

sentinel monitor mymaster 192.168.174.129 6380 2
    sentinel down-after-milliseconds mymaster 10000
    sentinel failover-timeout mymaster 180000
    sentinel parallel-syncs mymaster 1

# **    四、redis-cluster** #

创建  /home/ubuntu/redis/cluster/docker-compose.yml，如下

version: '2'
    services:
      cluster-a1:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
          - "./conf.d/cluster-6371.conf:/data/redis.conf" 
        network_mode: host
        command: ["redis-server","/data/redis.conf"]
    
      cluster-a2:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
          - "./conf.d/cluster-6372.conf:/data/redis.conf" 
        network_mode: host
        command: ["redis-server","/data/redis.conf"]
    
      cluster-b1:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
          - "./conf.d/cluster-6381.conf:/data/redis.conf" 
        network_mode: host
        command: ["redis-server","/data/redis.conf"]
    
      cluster-b2:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
          - "./conf.d/cluster-6382.conf:/data/redis.conf"
        network_mode: host
        command: ["redis-server","/data/redis.conf"]
    
      cluster-c1:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
          - "./conf.d/cluster-6391.conf:/data/redis.conf" 
        network_mode: host
        command: ["redis-server","/data/redis.conf"]
        
      cluster-c2:
        image: redis:5.0-rc-alpine
        restart: always
        volumes:
          - "/etc/timezone:/etc/timezone:ro"
          - "/etc/localtime:/etc/localtime:ro"
          - "./conf.d/cluster-6392.conf:/data/redis.conf"
        network_mode: host
        command: ["redis-server","/data/redis.conf"]

同时在 /home/ubuntu/redis/cluster/conf.d目录下创建cluster-6371.conf 、 cluster-6372.conf  、cluster-6381.conf  、cluster-6382.conf  、cluster-6391.conf 、cluster-6392.conf ，这六个配置文件唯一的区别只是第一行的port不一样,每个配置文件的port与文件名上的数字一致,如下面是cluster-6371.conf

port 6371
    cluster-enabled yes
    cluster-node-timeout 15000
    cluster-config-file "nodes.conf"
    pidfile /var/run/redis.pid
    logfile "cluster.log"
    dbfilename dump-cluster.rdb
    appendfilename "appendonly-cluster.aof"

接下来,在/home/ubuntu/redis/cluster目录下进行如下操作启动所有redis实例

$: docker-compose up -d

\-- 查看容器的状态

$: docker-compose ps

![20190301145407908.png][]

**(1) 将所有实例加入集群**

$: docker-compose exec cluster-c1 redis-cli -p 6381 -h 192.168.174.129

进入redis客户端命令行

192.168.174.129:6381> cluster meet 192.168.174.129 6371

192.168.174.129:6381> cluster meet 192.168.174.129 6372

192.168.174.129:6381> cluster meet 192.168.174.129 6381

192.168.174.129:6381> cluster meet 192.168.174.129 6382

192.168.174.129:6381> cluster meet 192.168.174.129 6391

192.168.174.129:6381> cluster meet 192.168.174.129 6392

检查一下效果

192.168.174.129:6381> cluster  nodes

![20190301114054136.png][]

注1：上面的第一列是node\_id，这个在步骤(3)中会用到,比如16a7e8a557ba01615c5b15e967c769f983913b85是192.168.174.129:6392的node\_id。

注2：当前的6个node都是master，还没有分配主从关系，具体见步骤(3)

**（2）分配虚拟槽**

可以看到配置的六个节点都已经加入到了集群中，但是其现在还不能使用，因为还没有将16384个槽分配到集群节点中。虚拟槽的分配可以使用redis-cli分别连接到6371，6381和6391端口的节点中，然后分别执行如下命令：

$:  docker-compose exec cluster-c1 redis-cli -p **6371** \-h 192.168.174.129 cluster addslots \{0..5000\}

$:  docker-compose exec cluster-c1 redis-cli -p **6381** \-h 192.168.174.129 cluster addslots \{5001..10000\}

$:  docker-compose exec cluster-c1 redis-cli -p **6391** \-h 192.168.174.129 cluster addslots \{10001..16383\}

\-- 检查当前集群的状态

$: docker-compose exec cluster-c1 redis-cli -p 6381 -h 192.168.174.129 cluster info

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NvbmdfamF2YQ_size_16_color_FFFFFF_t_70][]

**（3）设置主从关系**

通过上面的步骤，我们把16384个槽分配给了6371、6381、6391这三个端口对应的redis实例,因此这三个实例必须是master，至少为每一个master再分配一个slave，如下命令 : cluster replicate \[master\_node\_id\],

\-- 将6372对应的实例设置为6371的slave

$:  docker-compose exec cluster-c1 redis-cli -p 6372 -h 192.168.174.129 cluster replicate  16a7e8a557ba01615c5b15e967c769f983913b85

\-- 将6382对应的实例设置为6381的slave

$:  docker-compose exec cluster-c1 redis-cli -p 6382 -h 192.168.174.129 cluster replicate  3921bc89204a900360de770fe42f931787431e7c

\-- 将6392对应的实例设置为6391的slave

$:  docker-compose exec cluster-c1 redis-cli -p 6392 -h 192.168.174.129 cluster replicate fad1e9b5115c876dbbaf3ba4c500ddfca0508065

\-- 检查当前的节点主从关系

$:  docker-compose exec cluster-c1 redis-cli -p 6382 -h 192.168.174.129 cluster nodes

![20190301145138351.png][]

**（4）使用redis-cli连接集群**

$:  docker-compose exec cluster-c1 redis-cli -p 6382 -h 192.168.174.129 -c

进入redis-cli控制台

192.168.174.129:6382>  get k1

![20190301150123920.png][]

发现跳转到6391对应的实例上,你也可以试着停掉其中的一个master，然后观察一下他的slave是否会自动转为master。

还有更多redis-cluster的特性，都可以在这个集群上进行测试验证，小伙伴们还等什么，赶紧动手吧。

注： 细心的同学会发现，我们使用redis-cli时都是使用了"docker-compose exec **cluster-c1**  redis-cli "，其实我们也可以使用其他的redis实例,比如 "docker-compose exec **cluster-a2 ** redis-cli ",我们只是为了使用redis实例中内置的redis-cli命令,如果你本地已经装了redis-cli,也是可以直接使用redis-cli。

# 五、总结 #

关于redis主从复制、sentinel高可用以及redis-cluster，网上有很多优秀的文章对他们进行了非常详细的介绍，我在这里不在赘述，只是简单说一下我个人不太成熟的理解吧。redis主从复制可以实现读写分离,但是在master挂掉之后没有现成的机制重新选举，redis的写功能就不可用了，所以引出了sentinel，sentinel可以在master挂掉之后可以选一个slave替代挂掉的master，所以sentinel增强了高可用性,但是没有解决redis水平扩展的问题,当master的内存不足时，除了增加机器内存,没有现成的办法扩容，因此引入了redis-cluster，cluster通过哈希槽让各个master节点只负责一部分的哈希槽（hash slo)，然后每个master有多个slave作为数据备份,一旦master不行了，可以让slave顶上去。

sentinel和cluster都不能很好的使用读写分离,slave节点只是作为备份节点，因此slave节点不能替master分担读的压力，同时由于redis使用的是异步复制,不能保证强一致性。

有一些不错的帖子,大家可以参考一些

[https://my.oschina.net/zhangxufeng/blog/905611][https_my.oschina.net_zhangxufeng_blog_905611]

[https://dbaplus.cn/news-158-2182-1.html][https_dbaplus.cn_news-158-2182-1.html]

redis镜像地址： [https://hub.docker.com/\_/redis][https_hub.docker.com_redis]

官方地址: [https://redis.io/][https_redis.io]

英文不好的同学可以参考 [http://www.redis.cn/][http_www.redis.cn]

docker安装请参考  [https://blog.csdn.net/song\_java/article/details/88061162][https_blog.csdn.net_song_java_article_details_88061162]

[20190228182423551.png]: /images/20220315/5dc45b6be3064f0c8997c24718d1236f.png
[20190301145407908.png]: /images/20220315/293348ae680b452a9bf30eebe9ec4a81.png
[20190301114054136.png]: /images/20220315/541c214aee9f43b4aa3de444f975f676.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NvbmdfamF2YQ_size_16_color_FFFFFF_t_70]: /images/20220315/79c9ec91a1584954bdf77434854139ab.png
[20190301145138351.png]: /images/20220315/0899acb695c74236b898bf18fc4ccd14.png
[20190301150123920.png]: /images/20220315/547178285329486893bad7ef745a58f5.png
[https_my.oschina.net_zhangxufeng_blog_905611]: https://my.oschina.net/zhangxufeng/blog/905611
[https_dbaplus.cn_news-158-2182-1.html]: https://dbaplus.cn/news-158-2182-1.html
[https_hub.docker.com_redis]: https://hub.docker.com/_/redis
[https_redis.io]: https://redis.io/
[http_www.redis.cn]: http://www.redis.cn/
[https_blog.csdn.net_song_java_article_details_88061162]: https://blog.csdn.net/song_java/article/details/88061162