業務背景

訂單服務和訊息盒子服務使用典型的生產者消費者模式，
訂單狀態變更, 產生訊息, 發往SQS佇列, 
訊息盒子服務通過訂閱SQS佇列消費訊息，更新DB中訂單訊息的狀態。

出現死鎖問題

訊息盒子服務多個執行緒消費訂單狀態訊息, update DB中對應的訂單記錄, 出現死鎖。
update語句如下:

update msgbox_message set record_status = -1 where record_status = 0 and gmt_create >= now() - INTERVAL 3 MONTH and msg_key = ‘SO146213662’ and target = ‘201307438’.

看起來是很普通的update語句, 不應該出現死鎖。

msgbox_message 表結構如下 (精簡版)

CREATE TABLE msgbox_message (
	id BIGINT ( 20 ) UNSIGNED AUTO_INCREMENT,
	gmt_create TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
	record_status TINYINT ( 3 ),
	msg_key VARCHAR ( 64 ),
	target VARCHAR ( 32 ) COMMENT 'uid',
	target_type TINYINT ( 3 ),
	url VARCHAR ( 255 ),
	PRIMARY KEY (id),
	KEY target(target, target_type),
	KEY msg_key(msg_key),
	KEY gmt_create(gmt_create)
 ) ENGINE = INNODB
 

死鎖日誌分析

show engine innodb status的顯示結果如下(刪除了不必要的部分):

LATEST DETECTED DEADLOCK
2019-09-26 15:20:13
(1) TRANSACTION: TRANSACTION 149683649,thread id 659442, query id 956107029 IP1ops_write updating
update msgbox_message set record_status = -1 where record_status = 0 and gmt_create >= now() - INTERVAL 3 MONTH and msg_key = ‘SO146213662

’ and target = ‘201307438’
WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 118 page no 1399014 n bits 640index target_idxof table cf_msgbox.msgbox_message;
trx id 149683649 lock_mode X locks rec but not gap waiting Record lock,
heap no 539 PHYSICAL RECORD: n_fields 4; compact format; info bits 0 ——(索引樹的葉子節點資料)
0: len 9; hex 323031333037343338; asc201307438;;
1: len 1; hex 00; asc ;;
2: len 8; hex 0000000004a135cb; asc 5 ;; ——(4a135cb是 16 進位制的主鍵 ID 值77673931)
3: len 4; hex 5d8af714; asc ] ;;
..
(2) TRANSACTION: TRANSACTION 149683648,thread id 660492, query id 956107024 IP2ops_write updating
update msgbox_message set record_status = -1 where record_status = 0 and gmt_create >= now() - INTERVAL 3 MONTH and msg_key = ‘SOxxxxxxx’ and target = ‘201307438’
HOLDS THE LOCK(S):
RECORD LOCKS space id 118 page no 1399014 n bits 640index target_idxof table cf_msgbox.msgbox_message;
trx id 149683648 lock_mode X locks rec but not gap Record lock,
heap no 539 PHYSICAL RECORD: n_fields 4; compact format; info bits 0 ——(索引樹的葉子節點資料)
0: len 9; hex 323031333037343338; asc201307438;;
1: len 1; hex 00; asc ;;
2: len 8; hex 0000000004a135cb; asc 5 ;; ——(4a135cb是 16 進位制的主鍵 ID 值77673931)
3: len 4; hex 5d8af714; asc ] ;;
WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 118 page no 1936298 n bits 152index PRIMARYof table cf_msgbox.msgbox_message;
trx id 149683648 lock_mode X locks rec but not gap waiting Record lock,
heap no 13 PHYSICAL RECORD: ——(索引樹的葉子節點資料)
n_fields 18; compact format; info bits 0
0: len 8; hex 0000000004a135cb; asc 5 ;;—— (4a135cb是 16 進位制的主鍵 ID 值77673931)
1: len 4; hex 5d8af714; asc ] ;;
…….. 省略一些欄位
WE ROLL BACK TRANSACTION (2)

我把部分關鍵的資訊重點標註了出來，其中thread id是mysql的兩個內部執行緒, 
兩個IP地址就是訊息盒子服務機器的IP地址, 兩條update語句只有where條件中的msg_key值不同。

• 事務 1 在等待 target_idx 索引樹 (index target_idx) 中葉子節點 (heap no 539 PHYSICAL RECORD) 的 X 鎖(互斥鎖)—標記為 1 號鎖