redis cluster pipeline問題排查
阿新 • • 發佈:2018-12-25
問題背景:
問題程式碼:
r = StrictRedis('10.20.23.45', 3901)
print r.get('7551A63C3F2C0EA261AAE2B509ABC782172FE56DF64F64B6CB0B355E5A9D9FB7:u_feed_lt_ad_type_show:0')
print r.get('1803D8B45F7371F0DE0F98D392B07814B09E9601F2DDE57A0EA154685E85E16B:u_lt_search_showclk:0')
p = r.pipeline()
p.get('7551A63C3F2C0EA261AAE2B509ABC782172FE56DF64F64B6CB0B355E5A9D9FB7:u_feed_lt_ad_type_show:0' )
p.get('1803D8B45F7371F0DE0F98D392B07814B09E9601F2DDE57A0EA154685E85E16B:u_lt_search_showclk:0')
print p.execute()
出現報錯:redis.exceptions.ResponseError: CROSSSLOT Keys in request don’t hash to the same slot
異常資訊:這兩個key在同一個redis例項的不同slot上面。
- 在該redis例項分開get可以查詢到值
- 兩個key倒在同一個redis的不同slot
問題排查
- 趕緊google一發:大部分資訊都描述的是cluster環境下,批量get會由同一個slot的限定。
- 如果是cluster對mget的限定,我覺得可以理解,但是pipeline按照本人的理解,應該是打包多個命令,一次請求redis,然後在redis單獨執行,回到問題背景描述1,正常邏輯應該是執行沒問題的。
- 基於上面的分析,我認為pipeline的兩個get操作被壓縮了,可能在客戶端。
- 原始碼分析,python客戶端pipeline有個transaction引數預設為True,在execute的時候,會變成MULTI命令。
def execute(self, raise_on_error=True):
"Execute all the commands in the current pipeline"
stack = self.command_stack
if not stack:
return []
if self.scripts:
self.load_scripts()
if self.transaction or self.explicit_transaction:
# transaction引數
execute = self._execute_transaction
else:
execute = self._execute_pipeline
conn = self.connection
if not conn:
conn = self.connection_pool.get_connection('MULTI',
self.shard_hint)
# assign to self.connection so reset() releases the connection
# back to the pool after we're done
self.connection = conn
try:
# 請求redis
return execute(conn, stack, raise_on_error)
except (ConnectionError, TimeoutError) as e:
conn.disconnect()
if not conn.retry_on_timeout and isinstance(e, TimeoutError):
raise
# if we were watching a variable, the watch is no longer valid
# since this connection has died. raise a WatchError, which
# indicates the user should retry his transaction. If this is more
# than a temporary failure, the WATCH that the user next issues
# will fail, propegating the real ConnectionError
if self.watching:
raise WatchError("A ConnectionError occured on while watching "
"one or more keys")
# otherwise, it's safe to retry since the transaction isn't
# predicated on any state
return execute(conn, stack, raise_on_error)
finally:
self.reset()