elasticsearch 數據遷移
阿新 • • 發佈:2018-06-01
elasticsearch 數據遷移現需要將某集群下一個索引下的所有數據遷移到另一個集群上, elasticsearch-dump, Elasticsearch-Exporter 試了一下都不好使,只能老實的寫代碼來實現
import os import sys import pyes import datetime index_list = [ ["alias-offer", "offer"] ] ES_URL = "http://ip1:9200/" NEW_ES_URL = "http://ip2:9200/" def main(): for _index, _type in index_list: print "開始索引:"+_index conn = pyes.es.ES(ES_URL) print "已經連接上原地址" search = pyes.query.MatchAllQuery().search(bulk_read=1000) hits = conn.search(search, _index, _type, scan=True, scroll="30m", model=lambda _,hit: hit) print "原地址返回數據" conn2 = pyes.es.ES(NEW_ES_URL) print "連接到新地址" count = 0 for hit in hits: conn2.index(hit[‘_source‘], _index, _type, hit[‘_id‘], bulk=True) count += 1 if count % 1000 == 0: print (datetime.datetime.now().strftime(‘%Y-%m-%d %H:%M:%S‘),count) conn2.flush_bulk(forced=True) conn2.flush_bulk(forced=True) conn2 = None conn = None if __name__ == ‘__main__‘: main() print "數據導入結束"
雖然慢,但能用,基本滿足需要!
elasticsearch 數據遷移