1. 程式人生 > >ES(elasticsearch)備份還原腳本

ES(elasticsearch)備份還原腳本

ES備份還原

在作es backup和es restore(導入)時,需要停es集群中的data節點,只保留master節點。
以下圖說明,星號標識的為master節點,其他為data節點。
停兩個節點之前,可以在頁面上確認紅色標示的docs值,再停兩個data節點,檢查運行著的節點其docs值與停之前一致。
技術分享圖片

備份腳本es_backup.sh

#!/bin/bash
filename=`date +%Y%m%d%H`
backesFile=es$filename.tar.gz
mkdir -p  /xor/elasticsearch/es_snapshot
mkdir -p  /xor/elasticsearch/es_backup/es_dump
es_backup_dir=/xor/elasticsearch/es_backup
es_snapshot_dir=/xor/elasticsearch/es_snapshot
chmod -R 777 $es_snapshot_dir
curl -XPUT localhost:9200/_snapshot/es_snapshot -d ‘{"type":"fs","settings":{"location":"/xor/elasticsearch/es_snapshot"}}‘
cd $es_backup_dir/es_dump
curl -XDELETE localhost:9200/_snapshot/es_snapshot/$filename?
echo -e ‘\nsleep 5\n‘
sleep 10
curl -XPUT localhost:9200/_snapshot/es_snapshot/$filename?wait_for_completion=true
echo -e ‘\nsleep 5\n‘
sleep 5
cp $es_snapshot_dir/* $es_backup_dir/es_dump -rf
cd ..
tar czf $backesFile  es_dump/
rm es_dump -rf
cd $es_snapshot_dir
rm  * -rf 
echo -e ‘\nes data backup succeed!\n‘

還原腳本es_restore.sh

#!/bin/bash
filename=$1
backesFile=es$filename.tar.gz
mkdir -p  /xor/data2/packages/aqua/backup/es/es_snapshot
mkdir -p  /xor/data2/packages/aqua/backup/es/es_backup/es_dump
es_backup_dir=/xor/data2/packages/aqua/backup/es/es_backup
es_snapshot_dir=/xor/data2/packages/aqua/backup/es/es_snapshot
chmod -R 777 $es_snapshot_dir
curl -XPUT localhost:9200/_snapshot/es_snapshot -d ‘{"type":"fs","settings":{"location":"/xor/data2/packages/aqua/backup/es/es_snapshot"}}‘
cd $es_backup_dir
tar zxvf $backesFile
rm $es_snapshot_dir/* -rf
cp $es_backup_dir/es_dump/* $es_snapshot_dir -rf
curl -XPOST localhost:9200/_all/_close
echo -e ‘\nsleep 5\n‘
sleep 5
curl -XPOST localhost:9200/_snapshot/es_snapshot/$filename/_restore? 
rm es_dump -rf
echo -e ‘\n es data restore succeed!‘

es_delete_by_query.sh
用於刪除es的數據。腳本需要傳四個參數,參數依次為表名、時間字段名、開始時間、結束時間,示例如下:
./es_delete_by_query.sh vhsession view_begin_time 2018-01-10 00:00:00
2018-01-10 18:00:00
,時間區間包含上下界,時間字段也可以只傳日期

#!/bin/sh
# example: sh  ./es_delete_by_query.sh "bundle" "licensing_window_start" "2018-01-05 15:30:00" "2018-01-05 15:30:00"
# example: sh  ./es_delete_by_query.sh "bundle" "licensing_window_start" "2018-01-05" "2018-01-06"

doc_type=$1
datefield=$2
start_time=$3
end_time=$4

if [ ! -n "$start_time" ]; then
  echo "the start_time is empty,please input again...."
  exit 1
  else
    tt=`date -d "$start_time" +"%FT%T+0800"`
    if [ 0 -eq $? ]
      then
       start_time=$tt
      else
        exit 1
    fi       
fi

if [ ! -n "$end_time" ]; then
   echo "the end_time is empty,please input again...."
   exit 1
   else
   tt=`date -d "$end_time" +"%FT%T+0800"`
    if [ 0 -eq $? ]
      then
       end_time=$tt
      else
        exit 1
    fi   
fi

echo "args: doc_type = ${doc_type},datefield = ${datefield}, start_time = ${start_time}, end_time = ${end_time}"
echo "query count response : "
curl  -XPOST http://localhost:9200/aquapaassearch/${doc_type}/_count?pretty -d ‘
{
        "query": {
                "filtered": {
                        "filter": {
                                "bool": {
                                        "must": {
                                                "range": {
                                                        "‘${datefield}‘": {
                                                                "from": "‘${start_time}‘",
                                                                "to": "‘${end_time}‘",
                                                                "include_lower": true,
                                                                "include_upper": true
                                                        }
                                                }
                                        }
                                }
                        }
                }
        }
}‘ 

read -p "please input ‘1‘(otherwise not) to confirm if you need delete:" flag

if [ "1" = "$flag" ]; then
echo "delete response : "
curl  -XDELETE http://localhost:9200/aquapaassearch/${doc_type}/_query?pretty -d ‘
{
        "query": {
                "filtered": {
                        "filter": {
                                "bool": {
                                        "must": {
                                                "range": {
                                                        "‘${datefield}‘": {
                                                                "from": "‘${start_time}‘",
                                                                "to": "‘${end_time}‘",
                                                                "include_lower": true,
                                                                "include_upper": true
                                                        }
                                                }
                                        }
                                }
                        }
                }
        }
}‘
    echo "delete sucess!!!"
 else
   echo "confirmed just query count!!!"
fi

ES(elasticsearch)備份還原腳本