MongoDB分片群集
阿新 • • 發佈：2018-07-18
大全 請求 test mongodb分片 水平 reclaim === roc reat MongoDB分片群集
什麽是分片
高數據量和吞吐量的數據庫應用會對單機的性能造成較大的壓力，大的查詢量會把單機CPU耗盡，大的數據量對單機的儲存壓力大，最終會消耗系統的內存而將壓力轉移到磁盤IO上
MongoDB分片的優勢
分片為應對高吞吐量與大數據量提供了方法
- 使用分片減少了對需要處理的請求，因此，通過水平拓展，群集可以提高自己的儲存量和吞吐量。
- 使用分片減少了每個分片存儲的數據
分片服務器分為以下幾個服務器組成
-
路由服務器 （router）
-
配置服務器 （configsvr）
-
分片服務器 （shardsvr）
我們通過路由服務器發送請求給配置服務器，配置服務器調用下面的分片服務器
結構圖如下
MongoDB分片群集的部署
====（註：分片群集在3.2和3.2以上的版本不同3.2版本在做分片服務器的時候可以一個分片服務器對應一臺數據庫，但是在3.2以上的版本它要求每個分片服務器必須做群集）====
這邊我們用的是3.2版本的源代碼包
實驗步驟：做配置服務器
---------------------安裝MongoDB 3.2---------------------------- #安裝開發包 yum install openssl-devel -y #解壓源代碼包 tar zxvf mongodb-linux-x86_64-3.2.1.tgz -C /opt/ #重命名並移動到/usr/local文件裏面 mv mongodb-linux-x86_64-3.2.1/ /usr/local/mongodb #創建數據存放位置 mkdir -p /data/mongodb/mongodb{1,2,3,4} #創建日誌存放位置 mkdir /data/mongodb/logs 創建以log為結尾的日誌存放 touch /data/mongodb/logs/mongodb{1,2,3,4}.log #給日誌文件最大全權限 chmod -R 777 /data/mongodb/logs/*.log #修改最大並發連接 ulimit -n 25000 ulimit -u 25000
上面是開始安裝MongoDB，下面開始配置，配置服務器的配置文件
cd /usr/local/mongodb/bin/
vim mongodb1.conf
#配置服務器端口號
port=37017
#數據文件儲存位置
dbpath=/data/mongodb/mongodb1
#日誌文件儲存位置
logpath=/data/mongodb/logs/mongodb1.log
logappend=true
fork=true
#同時承受的並發連接數
maxConns=5000
storageEngine=mmapv1
#配置為配置服務器
configsvr=true某節點內存不足時，從其他節點分配內存
sysctl -w vm.zone_reclaim_mode=0
echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
#建立軟連接便於管理
ln -s /usr/local/mongodb/bin/mongo /usr/bin/mongo
ln -s /usr/local/mongodb/bin/mongod /usr/bin/mongod
#開啟配置服務器
mongod -f /usr/local/mongodb/bin/mongodb1.conf分片服務器 1和分片服務器 2同樣配置改端口號數據文件日誌文件位置加入shardsvr=true刪除原有的配置服務器configsvr=true
#端口號改為47017
port=37017
#數據文件儲存位置
dbpath=/data/mongodb/mongodb2
#日誌文件儲存位置
logpath=/data/mongodb/logs/mongodb.log
logappend=true
fork=true
#同時承受的並發連接數
maxConns=5000
storageEngine=mmapv1
#配置為分片服務器
shardsvr=true
#分別開啟分片服務器1和2啟動路由服務器
cd /usr/local/mongodb/bin/
#啟動mongos 路由服務器的端口號為27017日誌存放目錄在/usr/local/mongodb/bin/下面的route.log裏面
配置服務IP：端口號 --chunkSize 1 塊大小為1
./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.235.204:37017 --chunkSize 1
直接啟動mongo路由服務器
[root@localhost bin]# mongo
MongoDB shell version: 3.2.1
connecting to: test
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
Server has startup warnings:
2018-07-18T14:52:06.622+0800 I CONTROL [main] ** WARNING: You are running this process as the root user, which is not recommended.
2018-07-18T14:52:06.622+0800 I CONTROL [main]
mongos>
#查看數據庫
mongos> show dbs
config 0.031GB
mongos> sh.status() #shards下為空，沒有分片服務器
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
shards:
active mongoses:
"3.2.1" : 1
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
#添加分片服務器
mongos> sh.addShard("192.168.32.207:47018")
{ "shardAdded" : "shard0000", "ok" : 1 }
mongos> sh.addShard("192.168.32.207:47017")
{ "shardAdded" : "shard0001", "ok" : 1 }
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
shards: #下面兩個是分片服務器
{ "_id" : "shard0000", "host" : "192.168.32.207:47018" }
{ "_id" : "shard0001", "host" : "192.168.32.207:47017" }
active mongoses:
"3.2.1" : 1
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:實驗分片功能
添加兩個分片服務器後，數據庫和集合還未啟用分片使用mongoimport命令導入sales.txt數據到kgc數據庫的sales表
mongos> db.createCollection(‘users‘)
{ "ok" : 1 }
mongos> show collections
system.indexes
users
mongos> exit
bye
[root@localhost bin]# mongo
MongoDB shell version: 3.2.1
connecting to: test
Server has startup warnings:
2018-07-18T14:52:06.622+0800 I CONTROL [main] ** WARNING: You are running this process as the root user, which is not recommended.
2018-07-18T14:52:06.622+0800 I CONTROL [main]
mongos> show dbs
config 0.031GB
kgc 0.078GB
mongos> for (var i=1;i<=10000;i++)db.users.insert({"id":1,"name":"jaap"+i}) #插入10000條信息
WriteResult({ "nInserted" : 1 })
mongos> show tables
system.indexes
users
mongos> db.users.find() #查看信息
{ "_id" : ObjectId("5b4ef398b83d741f7c50491a"), "id" : 1, "name" : "zhang" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491b"), "id" : 1, "name" : "jaap1" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491c"), "id" : 1, "name" : "jaap2" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491d"), "id" : 1, "name" : "jaap3" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491e"), "id" : 1, "name" : "jaap4" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491f"), "id" : 1, "name" : "jaap5" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504920"), "id" : 1, "name" : "jaap6" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504921"), "id" : 1, "name" : "jaap7" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504922"), "id" : 1, "name" : "jaap8" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504923"), "id" : 1, "name" : "jaap9" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504924"), "id" : 1, "name" : "jaap10" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504925"), "id" : 1, "name" : "jaap11" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504926"), "id" : 1, "name" : "jaap12" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504927"), "id" : 1, "name" : "jaap13" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504928"), "id" : 1, "name" : "jaap14" }
{ "_id" : ObjectId("5b4ef407b83d741f7c504929"), "id" : 1, "name" : "jaap15" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492a"), "id" : 1, "name" : "jaap16" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492b"), "id" : 1, "name" : "jaap17" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492c"), "id" : 1, "name" : "jaap18" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50492d"), "id" : 1, "name" : "jaap19" }
Type "it" for more
mongos> db.users.find().limit(5) #查看指定信息
{ "_id" : ObjectId("5b4ef398b83d741f7c50491a"), "id" : 1, "name" : "zhang" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491b"), "id" : 1, "name" : "jaap1" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491c"), "id" : 1, "name" : "jaap2" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491d"), "id" : 1, "name" : "jaap3" }
{ "_id" : ObjectId("5b4ef407b83d741f7c50491e"), "id" : 1, "name" : "jaap4" }
mongos> sh.status() #查看數據庫分片信息
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
shards:
{ "_id" : "shard0000", "host" : "192.168.32.207:47018" }
{ "_id" : "shard0001", "host" : "192.168.32.207:47017" }
active mongoses:
"3.2.1" : 1
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "kgc", "primary" : "shard0000", "partitioned" : false }啟用分片
mongos> sh.enableSharding("kgc")
{ "ok" : 1 }
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
shards:
{ "_id" : "shard0000", "host" : "192.168.32.207:47018" }
{ "_id" : "shard0001", "host" : "192.168.32.207:47017" }
active mongoses:
"3.2.1" : 1
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "kgc", "primary" : "shard0000", "partitioned" : true
#對users表中id創建主鍵索引
mongos> db.users.createIndex({"id":1})
#表分片
mongos> sh.shardCollection("kgc.users",{"id":1})
#查看分片情況
mongos> sh.status()
--- Sharding Status ---
sharding version: {
"_id" : 1,
"minCompatibleVersion" : 5,
"currentVersion" : 6,
"clusterId" : ObjectId("5b4ee3961c914459f603260f")
}
shards:
{ "_id" : "shard0000", "host" : "192.168.32.207:47018" }
{ "_id" : "shard0001", "host" : "192.168.32.207:47017" }
active mongoses:
"3.2.1" : 1
balancer:
Currently enabled: yes
Currently running: no
Failed balancer rounds in last 5 attempts: 0
Migration Results for the last 24 hours:
No recent migrations
databases:
{ "_id" : "kgc", "primary" : "shard0000", "partitioned" : true }
kgc.users
shard key: { "id" : 1 }
unique: false
balancing: true
chunks:
shard0000 1
{ "id" : { "$minKey" : 1 } } -->> { "id" : { "$maxKey" : 1 } } on : shard0000 Timestamp(1, 0)添加標簽
#添加分片服務器
mongod -f mongodb4.conf
mongo
mongos> sh.addShard("192.168.235.205:47019")
mongos> sh.status()
chunks:
shard0000 4
shard0001 4
shard0002 3
#刪除分片節點
mongos> use admin
mongos> db.runCommand({"removeshard":"192.168.235.205:47019"})
MongoDB分片群集