通过docker-compose快速搭建mongodb副本集

约定不等于承诺〃 2023-05-28 05:06 32阅读 0赞

# 转  [https://monkeywie.github.io/2020/03/10/mongodb-replica-set/?utm\_source=tuicool&utm\_medium=referral][https_monkeywie.github.io_2020_03_10_mongodb-replica-set_utm_source_tuicool_utm_medium_referral] #

## 前言 ##

mongoDB 提供的副本集是将数据同步至多个节点，提供了数据冗余备份和节点故障的情况下可以自动转移的高可用特性，架构图如下：

有时候需要要在本地搭建一套 mongoDB 副本集环境来做测试，如果用虚拟机的话还是比较麻烦的，这里记录下如何用 docker-compose 快速搭建 mongoDB 副本集。

## 构建镜像 ##

1.  准备秘钥用于节点之间的认证

<table> 
 <tbody> 
  <tr> 
   <td> <pre><code>1
</code></pre> </td> 
   <td> <pre><code>openssl rand -base64 756 &gt; auth.key
</code></pre> </td> 
  </tr> 
 </tbody> 
</table>

1.  编写 Dockerfile

调整 mongo 官方提供的镜像，像里面添加秘钥文件

<table> 
 <tbody> 
  <tr> 
   <td> <pre><code>1
2
3
4
5
6
</code></pre> </td> 
   <td> <pre><code>FROM mongo:3.4.10
#将秘钥文件复制到镜像中
COPY auth.key /app/auth.key
RUN chown -R mongodb:mongodb /app/auth.key
#设置秘钥文件权限，这一步非常关键
RUN chmod 600 /app/auth.key
</code></pre> </td> 
  </tr> 
 </tbody> 
</table>

1.  构建镜像

<table> 
 <tbody> 
  <tr> 
   <td> <pre><code>1
</code></pre> </td> 
   <td> <pre><code>docker build -t mongo-replset .
</code></pre> </td> 
  </tr> 
 </tbody> 
</table>

1.  编写 docker-compose

开启三个容器，组成一主两从的副本集集群，另外再开启一个容器监听和初始化副本集，这里我的实现方式并不优雅，就是一直重试直到前面三个 mongod 进程启动完成(官方有个[解决方案][Link 1]来检测依赖服务的启动，不过有点麻烦这里就略过了)，启动完成之后执行副本集初始化命令：`mongo mongodb://root:123@mongo-1:27011/admin --eval 'rs.initiate({ _id: "rs", members: [{_id:1,host:"mongo-1:27011"},{_id:2,host:"mongo-2:27012"},{_id:3,host:"mongo-3:27013"}]})'`

<table> 
 <tbody> 
  <tr> 
   <td> <pre><code>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
</code></pre> </td> 
   <td> <pre><code>version: "3.1"
services:
  mongo-1:
    image: mongo-replset
    hostname: mongo-1
    restart: always
    ports:
      - 27011:27011
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: 123
    command:
      - --port
      - "27011"
      - --replSet
      - rs
      - --keyFile
      - /app/auth.key
  mongo-2:
    image: mongo-replset
    hostname: mongo-2
    restart: always
    ports:
      - 27012:27012
    command:
      - --port
      - "27012"
      - --replSet
      - rs
      - --keyFile
      - /app/auth.key
  mongo-3:
    image: mongo-replset
    hostname: mongo-3
    restart: always
    ports:
      - 27013:27013
    command:
      - --port
      - "27013"
      - --replSet
      - rs
      - --keyFile
      - /app/auth.key
  mongo-init:
    image: mongo:3.4.10
    depends_on:
      - mongo-1
      - mongo-2
      - mongo-3
    restart: on-failure:5
    command:
      - mongo
      - mongodb://root:123@mongo-1:27011/admin
      - --eval
      - 'rs.initiate({ _id: "rs", members: [{_id:1,host:"mongo-1:27011"},{_id:2,host:"mongo-2:27012"},{_id:3,host:"mongo-3:27013"}]})'
</code></pre> </td> 
  </tr> 
 </tbody> 
</table>

1.  运行

<table> 
 <tbody> 
  <tr> 
   <td> <pre><code>1
2
3
4
5
</code></pre> </td> 
   <td> <pre><code>docker-compose up -d
Creating mongo-replica_mongo-1_1 ... done
Creating mongo-replica_mongo-3_1 ... done
Creating mongo-replica_mongo-2_1 ... done
Creating mongo-replica_mongo-init_1 ... done
</code></pre> </td> 
  </tr> 
 </tbody> 
</table>

1.  查看副本集状态

<table> 
 <tbody> 
  <tr> 
   <td> <pre><code>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
</code></pre> </td> 
   <td> <pre><code>docker-compose exec mongo-1 mongo mongodb://root:123@mongo-1:27011/admin --eval 'rs.status()'
MongoDB shell version v3.4.10
connecting to: mongodb://root:123@mongo-1:27011/admin
MongoDB server version: 3.4.10
{
        "set" : "rs",
        "date" : ISODate("2020-03-10T07:01:59.764Z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "heartbeatIntervalMillis" : NumberLong(2000),
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1583823710, 1),
                        "t" : NumberLong(1)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1583823710, 1),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1583823710, 1),
                        "t" : NumberLong(1)
                }
        },
        "members" : [
                {
                        "_id" : 1,
                        "name" : "mongo-1:27011",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 263,
                        "optime" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-03-10T07:01:50Z"),
                        "electionTime" : Timestamp(1583823469, 1),
                        "electionDate" : ISODate("2020-03-10T06:57:49Z"),
                        "configVersion" : 1,
                        "self" : true
                },
                {
                        "_id" : 2,
                        "name" : "mongo-2:27012",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 261,
                        "optime" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-03-10T07:01:50Z"),
                        "optimeDurableDate" : ISODate("2020-03-10T07:01:50Z"),
                        "lastHeartbeat" : ISODate("2020-03-10T07:01:59.120Z"),
                        "lastHeartbeatRecv" : ISODate("2020-03-10T07:01:58.109Z"),
                        "pingMs" : NumberLong(0),
                        "syncingTo" : "mongo-1:27011",
                        "configVersion" : 1
                },
                {
                        "_id" : 3,
                        "name" : "mongo-3:27013",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 261,
                        "optime" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1583823710, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-03-10T07:01:50Z"),
                        "optimeDurableDate" : ISODate("2020-03-10T07:01:50Z"),
                        "lastHeartbeat" : ISODate("2020-03-10T07:01:59.120Z"),
                        "lastHeartbeatRecv" : ISODate("2020-03-10T07:01:58.103Z"),
                        "pingMs" : NumberLong(0),
                        "syncingTo" : "mongo-1:27011",
                        "configVersion" : 1
                }
        ],
        "ok" : 1
}
</code></pre> </td> 
  </tr> 
 </tbody> 
</table>

# #

增加节点  
在新的服务器上部署并启动容器，部署的时候需要将mongodb.key和.env文件一起复制过去，端口根据实际情况修改映射  
docker-compsoe.yaml文件  
version: '3.1'

services:  
  mongodb1:  
    image: mongo  
    restart: always  
    container\_name: mongo4  
    volumes:  
      - ./data/db/mongo4:/data/db  
      - ./mongodb.key:/data/mongodb.key  
    ports:  
      - 27020:27017  
    environment:  
      MONGO\_INITDB\_ROOT\_USERNAME: $\{MONGO\_INITDB\_ROOT\_USERNAME\}  
      MONGO\_INITDB\_ROOT\_PASSWORD: $\{MONGO\_INITDB\_ROOT\_PASSWORD\}  
    command: mongod --replSet mongos --keyFile /data/mongodb.key  
    entrypoint:  
      - bash  
      - -c  
      - |  
        chmod 400 /data/mongodb.key  
        chown 999:999 /data/mongodb.key  
        exec docker-entrypoint.sh $$@  
  
在Primary节点数据库中，进入shell添加新创建的容器，服务器IP和端口  
mongos:PRIMARY> rs.add("192.168.1.111:27020")  
原文链接：https://blog.csdn.net/biao0309/java/article/details/87641272

# #

# **python读取mongo读写分离**   https://api.mongodb.com/python/current/examples/high\_availability.html #

>>> from pymongo import ReadPreference  
>>> client.read\_preference  
SecondaryPreferred(tag\_sets=None)  
>>> db = client.get\_database('test', read\_preference=ReadPreference.SECONDARY)  
>>> db.read\_preference  
Secondary(tag\_sets=None)  
>>> coll = db.get\_collection('test', read\_preference=ReadPreference.PRIMARY)  
>>> coll.read\_preference  
Primary()

利用python测试mongodb副本集数据同步延迟  
转  [https://blog.51cto.com/hnr520/1874506][https_blog.51cto.com_hnr520_1874506]  
本文主要介绍python如何连接mongodb副本集及读写分离配置，mongodb副本集数据同步延迟测试。

一、python连接mongodb副本集

1.连接副本集

在新版的驱动中直接使用MongoClient连接

如下：

from pymongo import MongoClient  
conn = MongoClient(\['192.168.3.11:27017', '192.168.3.12:27017', '192.168.3.13:27017'\])  
2.读写分离配置

from pymongo import ReadPreference  
db = conn.get\_database('hnrtest', read\_preference=ReadPreference.SECONDARY\_PREFERRED)  
副本集ReadPreference有5个选项：

PRIMARY：默认选项，从primary节点读取数据

PRIMARY\_PREFERRED：优先从primary节点读取，如果没有primary节点，则从集群中可用的secondary节点读取

SECONDARY：从secondary节点读取数据

SECONDARY\_PREFERRED：优先从secondary节点读取，如果没有可用的secondary节点，则从primary节点读取

NEAREST：从集群中可用的节点读取数据

二、测试mongodb延迟脚本

1.原理

往集群中插入一批数据，然后从secondary节点读取，直到读到数据总数与插入的总数相等。

2.脚本内容

\#!/usr/bin/python  
\# -\*- coding: utf-8 -\*-  
import pymongo  
from pymongo import MongoClient  
from pymongo import ReadPreference  
import time  
import argparse  
\# 定义执行时长装饰器函数

def warps(\*args):  
    def deco(func):  
        def \_deco(\*args, \*\*kwargs):  
            \# 记录开始时间  
            start = time.clock()  
            \# 回调原函数  
            func(\*args, \*\*kwargs)  
            \# 记录结束时间  
            end = time.clock()  
            \# 计算执行时长  
            delat = end - start  
            \# 转换成ms输出  
            print("delay:%sms" % (int(delat\*1000)))  
        return \_deco  
    return deco

\# 连接副本集  
conn = MongoClient(\['192.168.3.11:27017', '192.168.3.12:27017', '192.168.3.13:27017'\])  
\# 读写分离  
db = conn.get\_database('hnrtest', read\_preference=ReadPreference.SECONDARY\_PREFERRED)  
\# 定义连接的集合  
collection = db.student  
\# 创建插入数据函数

def data\_insert(num):  
    try:  
        for i in range(1, num):  
            collection.insert(\{"name": "student"+str(i), "age": (i % 100), "city": "FuZhou"\})  
    except Exception as e:  
        print("insert data:", e)  
\# 创建查询数据函数，引用装饰器函数

@warps()  
def data\_select(num):  
    try:  
        count = collection.find().count()  
        while count != num - 1:  
            count = collection.find().count()  
    except Exception as e:  
        print("select data:", e)  
\# 创建删除数据函数

def data\_delete():  
    try:  
        collection.remove(\{\})  
    except Exception as e:  
        print("delete data:", e)  
\# 创建计算延迟时长函数

def data\_delay(num):  
    data\_insert(num)  
    data\_select(num)

if \_\_name\_\_ == '\_\_main\_\_':  
    \# 定义脚本需要传入插入的数据量，默认值为1000，通过-n传入参数  
    parser = argparse.ArgumentParser(description='insert data to mongodb number')  
    parser.add\_argument('-n', action='store', dest='num', type=int, required=False, default=1000)  
    given\_args = parser.parse\_args()  
    num = given\_args.num  
    data\_delete()  
    data\_delay(num)

# #

## 后记 ##

使用 docker-compose 搭建的 mongodb 副本集就完成了，以后可以基于此快速搭建一套 mongodb 副本集环境进行测试。

[https_monkeywie.github.io_2020_03_10_mongodb-replica-set_utm_source_tuicool_utm_medium_referral]: https://monkeywie.github.io/2020/03/10/mongodb-replica-set/?utm_source=tuicool&utm_medium=referral
[Link 1]: https://docs.docker.com/compose/startup-order/
[https_blog.51cto.com_hnr520_1874506]: https://blog.51cto.com/hnr520/1874506