MongoDB Advance----> AutoSharding and Replication

深碍√TFBOYSˉ_ 2022-06-18 06:36 150阅读 0赞

[MongoDB Advance----> AutoSharding and Replication][MongoDB Advance----_ AutoSharding and Replication]

转自：[http://blog.csdn.net/crazyjixiang/article/details/6623418][MongoDB Advance----_ AutoSharding and Replication]

MongoDB提供了Replica pairs模式启动数据库，以这种方式启动后，数据库会自动协商谁是master谁是slave。一旦一个数据库服务器断电，另外一个会自动接管，并从那一刻起为master，万一另一个将来也出错了，那么master状态会转回第一个服务器。

MongoDB包括一个自动分片模块”mongos“ 从而可以构建一个大的水平可扩展的数据库集群，可以动态的添加服务器，自动建立一个水平扩展的数据库集群系统，将数据库分表存储在sharding的各个节点上

这里我使用3台服务器做测试。

分别为

10.X.X.21.163

10.X.X.21.164

10.X.X.228

................. (如果有服务没起来 ，根据经验可能是你Data下的文件锁问题，或者是命名不对)

准备工作:

每台机器先创建数据目录

Server 1

[view plain][] [print][view plain] [?][view plain]

1.  ^\_^\[root@:/usr/local/mongodb\]\#mkdir -p data/shard11 
2.  ^\_^\[root@:/usr/local/mongodb\]\#mkdir -p data/shard21

Server 2

[view plain][] [print][view plain] [?][view plain]

1.  ^\_^\[root@:/usr/local/mongodb\]\#mkdir -p data/shard12 
2.  ^\_^\[root@:/usr/local/mongodb\]\#mkdir -p data/shard22

Server 3

[view plain][] [print][view plain] [?][view plain]

1.  ^\_^\[root@:/usr/local/mongodb\]\#mkdir -p data/shard13 
2.  ^\_^\[root@:/usr/local/mongodb\]\#mkdir -p data/shard23

然后针对每个服务器进行Shard1 Replica Sets:

Server1:

[view plain][] [print][view plain] [?][view plain]

1.  ./mongod --shardsvr --replSet shard1 --port 27017 --dbpath ../data/shard11 --oplogSize 100 --logpath ../data/shard11.log --logappend --fork.

Server2:

[view plain][] [print][view plain] [?][view plain]

1.  ./mongod --shardsvr --replSet shard1 --port 27017 --dbpath ../data/shard12 --oplogSize 100 --logpath ../data/shard12.log --logappend --fork.

Server3:

[view plain][] [print][view plain] [?][view plain]

1.  ./mongod --shardsvr --replSet shard1 --port 27017 --dbpath ../data/shard13 --oplogSize 100 --logpath ../data/shard13.log --logappend --fork.

初始化Replica set:

[view plain][] [print][view plain] [?][view plain]

1.  > config=\{\_id:'shard1',members:\[ 
2.  ... \{\_id:0,host:'10.X.X.228:27017'\}, 
3.  ... \{\_id:1,host:'10.X.X.163:27017'\}, 
4.  ... \{\_id:2,host:'10.X.X.164:27017'\}\] 
5.  ... \}

[view plain][] [print][view plain] [?][view plain]

1.  rs.initiate(config);

再给Shard2配置Replica Sets

Server1:

[view plain][] [print][view plain] [?][view plain]

1.  ./mongod --shardsvr --replSet shard2 --port 27018 --dbpath ../data/shard21 --oplogSize 100 --logpath ../data/shard21.log --logappend --fork.

Server2:

[view plain][] [print][view plain] [?][view plain]

1.  ./mongod --shardsvr --replSet shard2 --port 27018 --dbpath ../data/shard22 --oplogSize 100 --logpath ../data/shard22.log --logappend --fork.

Server3:

[view plain][] [print][view plain] [?][view plain]

1.  ./mongod --shardsvr --replSet shard2 --port 27018 --dbpath ../data/shard23 --oplogSize 100 --logpath ../data/shard23.log --logappend --fork.

经过第一次初始化后运行客户端会出现：

![0_1318666774Mma3.gif][]

如果要用27018的 需要指定

./mongo 10.X.X.228:27018

再次初始化Replica set:

[view plain][] [print][view plain] [?][view plain]

1.  > config=\{\_id:'shard1',members:\[ 
2.  ... \{\_id:0,host:'10.X.X.228:27018'\}, 
3.  ... \{\_id:1,host:'10.X.X.163:27018'\}, 
4.  ... \{\_id:2,host:'10.X.X.164:27018'\}\] 
5.  ... \}

[view plain][] [print][view plain] [?][view plain]

1.  rs.initiate(config);

现在有2 个 Replica sets 和2个shards

.......................

再配置三台Config Server

[view plain][] [print][view plain] [?][view plain]

1.  mkdir -p data/config 
2.  ./mongod --configsvr --dbpath ../data/config --port 20000 --logpath ../data/config.log -- logappend --fork.

每台Server都这样运行一遍（哎。好多配置啊。。。。。）

再配置Mongos（同样每台机器都要运行一遍）

[view plain][] [print][view plain] [?][view plain]

1.  ./mongos --configdb 10.X.X.228:20000,10.X.X.163:20000,10.X.X.164:20000 --port 30000 - -chunkSize 5 --logpath ../data/mongos.log --logappend --fork

再Config Shard Cluster

连接到mongos，并且切换到admin

./mongo 10.X.X.228:30000/admin

>db

admin

再加入Shards

[view plain][] [print][view plain] [?][view plain]

1.  db.runCommand(\{addshard:"shard1/10.7.3.228:27017,10.10.21.163:27017,10.10.21.164:27017",name:"s1",maxsize:20480\}); 
2.  db.runCommand(\{addshard:"shard2/10.7.3.228:27018,10.10.21.163:27018,10.10.21.164:27018",name:"s2",maxsize:20480\});

enable一个数据库

[view plain][] [print][view plain] [?][view plain]

1.  db.runCommand(\{ enablesharding:"test" \})

数据集分片

db.runCommand(\{ shardcollection: "test.users", key: \{ \_id:1 \}\})

列出已经加入的Shards

[view plain][] [print][view plain] [?][view plain]

1.  > db.runCommand(\{listshards:1\})

查看Sharding信息

[view plain][] [print][view plain] [?][view plain]

1.  printShardingStatus ()

查看Shard存储信息(需要先use test)

[view plain][] [print][view plain] [?][view plain]

1.  db.users.stats()

[view plain][] [print][view plain] [?][view plain]

1.  > use test 
2.  switched to db test 
3.  > db.users.stats() 
4.  \{ 
5.  "sharded" : **true**, 
6.  "ns" : "test.users", 
7.  "count" : 0, 
8.  "size" : 0, 
9.  "avgObjSize" : NaN, 
10. "storageSize" : 8192, 
11. "nindexes" : 1, 
12. "nchunks" : 1, 
13. "shards" : \{ 
14. "s1" : \{ 
15. "ns" : "test.users", 
16. "count" : 0, 
17. "size" : 0, 
18. "storageSize" : 8192, 
19. "numExtents" : 1, 
20. "nindexes" : 1, 
21. "lastExtentSize" : 8192, 
22. "paddingFactor" : 1, 
23. "flags" : 1, 
24. "totalIndexSize" : 8192, 
25. "indexSizes" : \{ 
26. "\_id\_" : 8192 
27.  \}, 
28. "ok" : 1 
29.  \} 
30.  \}, 
31. "ok" : 1 
32. \}

下面对分片进行测试：

[view plain][] [print][view plain] [?][view plain]

1.  \#include <iostream>
2.  \#include <mongo/client/dbclient.h>
3.  **using****namespace** std; 
4.  **using****namespace** mongo; 
5.  
6.  \#define INIT\_TIME \\
7.  **struct** timeval time1,time2; \\ 
8.  
9.  \#define START\_TIME \\
10.  gettimeofday(&time1,NULL); \\ 
11. 
12. \#define STOP\_TIME \\
13.  gettimeofday(&time2,NULL); \\ 
14. 
15. \#define PRINT\_TIME \\
16.  cout<<"Time:"<<time2.tv\_sec-time1.tv\_sec<<":"<<time2.tv\_usec-time1.tv\_usec<<endl; 
17. 
18. **int** main() \{ 
19.  srand(time(NULL)); 
20. **char** ar\[26+1\]; 
21.  DBClientConnection conn; 
22.  conn.connect("10.7.3.228:30000"); 
23.  cout<<"MongoDB Connected OK!"<<endl; 
24. **int** count=10000000; 
25.  INIT\_TIME; 
26.  START\_TIME; 
27. //insert
28. \#if 1
29. **while** (count--) \{ //loop insert 1000W
30. **for** (**int** i=0; i<26; i++) \{ 
31.  ar\[i\] = rand()%26+97; 
32.  \} 
33.  ar\[26\]='\\0'; 
34.  BSONObj p = BSON("NewsId"<<ar); 
35.  conn.insert("test.users",p); 
36.  \} 
37. \#endif
38. 
39. //Query
40. \#if 0
41.  cout<<"Count:"<<conn.count("News.News\_key")<<endl; 
42.  BSONObj emptyObj; 
43.  auto\_ptr<DBClientCursor> cursor = conn.query("News.News\_key",emptyObj); 
44. **while** (cursor->more()) \{ 
45.  BSONObj p = cursor->next(); 
46.  cout<<p.getStringField("NewsId")<<endl; 
47. **if** (p.getStringField("NewsId")=="bfatckiyxlougsxrffsnylsfuo")\{ 
48.  cout<<"find"<<endl; 
49. **break**; 
50.  \} 
51.  \} 
52. \#endif
53.  STOP\_TIME; 
54.  PRINT\_TIME; 
55. **return** 0; 
56. \} 
57.

[MongoDB Advance----_ AutoSharding and Replication]: http://blog.csdn.net/crazyjixiang/article/details/6623418
[view plain]: http://blog.csdn.net/crazyjixiang/article/details/6623418#
[0_1318666774Mma3.gif]: /images/20220618/878ba4782214496780a9c338e2409dde.png