1. 程式人生 > 資料庫 >Python小程式掃描清理Redis中的key

Python小程式掃描清理Redis中的key

場景

專案中很多地方使用Redis,有的用於快取,有的直接做為儲存,有的key設定有過期,有的key沒有過期時間。
隨著時間增長,Redis儲存資料越來越多,消耗記憶體不斷增長;
無論測試或生產環境,總記憶體是有限的;
有的key可能臨時或測試使用的;
於是有了清理Redis key的需求。

Redis命令

檢視key個數:
dbsize
info keyspace

檢視記憶體情況:
info memory

萬用字元掃描key:
SCAN cursor [MATCH pattern] [COUNT count]

Python小程式

Linux伺服器一般都自帶Python,這裡用Python編寫2個小程式,分別用於掃描key和掃描並刪除key。

掃描key:

#!/usr/bin/env python
# Scan keys in Redis.
# Author: cdfive
from redis import Redis
import time


def RedisScan(host, port, password, db, cursor, pattern, count):
    start_time = time.time()
    client = Redis(host=host, port=port, password=password, db=db)
    counts, other_cursor_counts = 0, 0
    while True:
        cursor, keys = client.scan(cursor, pattern, count)
        length = len(keys)
        if length > 0:
            for key in keys:
                counts += 1
                print("[%s][%s]%s,cursor=%s" % (
                    int(time.time() - start_time), counts, key.decode("utf-8"), cursor))
        else:
            other_cursor_counts += 1
            print("[%s][other_curosr]other_cursor_counts=%s,cursor=%s" % (
                int(time.time() - start_time), other_cursor_counts, cursor))

        if cursor == 0:
            break
    client.close()
    print("[%s]counts=%s,cursor=%s,other_cursor_counts=%s"
          % (int(time.time() - start_time), counts, cursor, other_cursor_counts))


RedisScan("localhost", 6379, "123456", 0, 0, "*xxx*", 1000)

掃描並刪除key:

#!/usr/bin/env python
# Scan and delete keys in Redis.
# Author: cdfive
from redis import Redis
import time


def RedisScanDelete(host, port, password, db, cursor, pattern, count, batch_delete_size):
    start_time = time.time()
    client = Redis(host=host, port=port, password=password, db=db)
    counts, other_cursor_counts = 0, 0
    delete_keys = []
    while True:
        cursor, keys = client.scan(cursor, pattern, count)
        length = len(keys)
        if length > 0:
            index = 0
            for key in keys:
                index += 1
                counts += 1
                print("[%s][%s]%s,cursor=%s" % (
                    int(time.time() - start_time), counts, key.decode("utf-8"), cursor))

                delete_keys.append(key)
                if (len(delete_keys) >= batch_delete_size or index >= length) and len(delete_keys) > 0:
                    client.delete(*delete_keys)
                    delete_keys = []
        else:
            other_cursor_counts += 1
            print("[%s][other_curosr]other_cursor_counts=%s,cursor=%s" % (
                int(time.time() - start_time), other_cursor_counts, cursor))

        if cursor == 0:
            break
    client.close()
    print("[%s]counts=%s,cursor=%s,other_cursor_counts=%s"
          % (int(time.time() - start_time), counts, cursor, other_cursor_counts))


RedisScanDelete("localhost", 6379, "123456", 0, 0, "*xxx*", 1000, 100)

注:RedisScanDelete最後1個引數為batch_delete_size,用於通過pipeline批量刪除key,提高刪除效率。