ES(elasticsearch)備份還原腳本
阿新 • • 發佈：2018-05-31
ES備份還原在作es backup和es restore（導入）時，需要停es集群中的data節點，只保留master節點。
以下圖說明，星號標識的為master節點，其他為data節點。
停兩個節點之前，可以在頁面上確認紅色標示的docs值，再停兩個data節點，檢查運行著的節點其docs值與停之前一致。
以下圖說明，星號標識的為master節點，其他為data節點。
停兩個節點之前，可以在頁面上確認紅色標示的docs值，再停兩個data節點，檢查運行著的節點其docs值與停之前一致。
備份腳本es_backup.sh
#!/bin/bash filename=`date +%Y%m%d%H` backesFile=es$filename.tar.gz mkdir -p /xor/elasticsearch/es_snapshot mkdir -p /xor/elasticsearch/es_backup/es_dump es_backup_dir=/xor/elasticsearch/es_backup es_snapshot_dir=/xor/elasticsearch/es_snapshot chmod -R 777 $es_snapshot_dir curl -XPUT localhost:9200/_snapshot/es_snapshot -d ‘{"type":"fs","settings":{"location":"/xor/elasticsearch/es_snapshot"}}‘ cd $es_backup_dir/es_dump curl -XDELETE localhost:9200/_snapshot/es_snapshot/$filename? echo -e ‘\nsleep 5\n‘ sleep 10 curl -XPUT localhost:9200/_snapshot/es_snapshot/$filename?wait_for_completion=true echo -e ‘\nsleep 5\n‘ sleep 5 cp $es_snapshot_dir/* $es_backup_dir/es_dump -rf cd .. tar czf $backesFile es_dump/ rm es_dump -rf cd $es_snapshot_dir rm * -rf echo -e ‘\nes data backup succeed!\n‘
還原腳本es_restore.sh
#!/bin/bash filename=$1 backesFile=es$filename.tar.gz mkdir -p /xor/data2/packages/aqua/backup/es/es_snapshot mkdir -p /xor/data2/packages/aqua/backup/es/es_backup/es_dump es_backup_dir=/xor/data2/packages/aqua/backup/es/es_backup es_snapshot_dir=/xor/data2/packages/aqua/backup/es/es_snapshot chmod -R 777 $es_snapshot_dir curl -XPUT localhost:9200/_snapshot/es_snapshot -d ‘{"type":"fs","settings":{"location":"/xor/data2/packages/aqua/backup/es/es_snapshot"}}‘ cd $es_backup_dir tar zxvf $backesFile rm $es_snapshot_dir/* -rf cp $es_backup_dir/es_dump/* $es_snapshot_dir -rf curl -XPOST localhost:9200/_all/_close echo -e ‘\nsleep 5\n‘ sleep 5 curl -XPOST localhost:9200/_snapshot/es_snapshot/$filename/_restore? rm es_dump -rf echo -e ‘\n es data restore succeed!‘
es_delete_by_query.sh
用於刪除es的數據。腳本需要傳四個參數,參數依次為表名、時間字段名、開始時間、結束時間,示例如下:
./es_delete_by_query.sh vhsession view_begin_time 2018-01-10 00:00:00
2018-01-10 18:00:00
，時間區間包含上下界，時間字段也可以只傳日期
#!/bin/sh # example: sh ./es_delete_by_query.sh "bundle" "licensing_window_start" "2018-01-05 15:30:00" "2018-01-05 15:30:00" # example: sh ./es_delete_by_query.sh "bundle" "licensing_window_start" "2018-01-05" "2018-01-06" doc_type=$1 datefield=$2 start_time=$3 end_time=$4 if [ ! -n "$start_time" ]; then echo "the start_time is empty,please input again...." exit 1 else tt=`date -d "$start_time" +"%FT%T+0800"` if [ 0 -eq $? ] then start_time=$tt else exit 1 fi fi if [ ! -n "$end_time" ]; then echo "the end_time is empty,please input again...." exit 1 else tt=`date -d "$end_time" +"%FT%T+0800"` if [ 0 -eq $? ] then end_time=$tt else exit 1 fi fi echo "args: doc_type = ${doc_type},datefield = ${datefield}, start_time = ${start_time}, end_time = ${end_time}" echo "query count response : " curl -XPOST http://localhost:9200/aquapaassearch/${doc_type}/_count?pretty -d ‘ { "query": { "filtered": { "filter": { "bool": { "must": { "range": { "‘${datefield}‘": { "from": "‘${start_time}‘", "to": "‘${end_time}‘", "include_lower": true, "include_upper": true } } } } } } } }‘ read -p "please input ‘1‘(otherwise not) to confirm if you need delete:" flag if [ "1" = "$flag" ]; then echo "delete response : " curl -XDELETE http://localhost:9200/aquapaassearch/${doc_type}/_query?pretty -d ‘ { "query": { "filtered": { "filter": { "bool": { "must": { "range": { "‘${datefield}‘": { "from": "‘${start_time}‘", "to": "‘${end_time}‘", "include_lower": true, "include_upper": true } } } } } } } }‘ echo "delete sucess!!!" else echo "confirmed just query count!!!" fi
ES(elasticsearch)備份還原腳本