基於snowflake演算法實現發號器
(2)snowflake的結構如下(每部分用-分開):
0 -
0000000000
0000000000
0000000000
0000000000
000 -
000 -
00000 -
0000000000
00
一共加起來剛好64位,為一個Long型。(轉換成字串長度為18)
snowflake生成的ID整體上按照時間自增排序,並且整個分散式系統內不會產生ID碰撞(由datacenter和workerId作區分),並且效率較高。
2、演算法變形:
(1)long型別最大值是9,223,372,036,854,775,807(2^63 -1),即19位十進位制數。取前13位作為毫秒數,1位作為毫秒內序列號,2位作為機器編號,3位作為資料庫表尾號。這個演算法單機每毫秒內理論最多可以生產10個id(每秒內理論最多可以生產1w個id),完全滿足業務需求。
(2)結構如下(每部分用 - 分開):
0000000000
000 - 0 - 00 -000
加起來正好19位。生成的ID整體上按照時間自增排序,並且整個分散式系統內不會產生ID碰撞。
3、清分系統中的id樣例:
(1)伺服器分配機器編碼
#server_host=server_number( 0 ~ 59 用於線上機器, 60 ~ 79 使用者staging機器, 80 ~ 99 用於線下機器)
yf-pay-clear21= 0
yf-pay-clear22= 1
yf-pay-clear23= 2
yf-pay-clear24= 3
yf-pay-clear25= 4
yf-pay-clear26= 5
yf-pay-clear27= 6
yf-pay-clear28= 7
yf-pay-clear29= 8
yf-pay-clear30= 9
yf-pay-clear-staging01= 60
jiabaozhen-clear.office.mos= 81
zhuangyudeMacBook-Pro.local= 82
yf-pay-clear-test01.corp.sankuai.com= 83
|
(2)現清分系統中,tradeFlowId和summaryTradeFlowId前4位為YYMM,即1611開頭的19位數。
新的snowflake id生成器生成的id是1613開頭的19位數。如1613024787541981316,1613024787541為當前時間 (毫秒+偏移量1345400000000),9為序列號,81為機器編碼,316為資料庫尾號。
(3)現清分系統中,batchid為26開頭的10位數。
batchid不涉及資料庫尾號,所以可以減少id的位數,保證batchid大於26開頭的10位數就可以。生成id如1478484776931281,1478484776931為當前時間,2為序列號,81為機器編碼。
三、程式碼描述:
1、加鎖實現:
public class IdWorker {
// 按照清分系統現在的id生成速度,2016-10-01之前,最大id小於1609040000000。
// 時間戳1473000000000對應北京時間2016-9-4 22:40:00
// twepoch = 1609040000000 - 1473000000000 = 136040000000,所以 2016-9-4以後的時間戳 + 136040000000 肯定大於 1609040000000,只要在2016-10-01之前上線服務,生成的id就不會與目前庫中id衝突。
private final long twepoch = 136040000000L;
// 機器編號,十進位制兩位,0-99
private final int workerIdBits = 2;
private final int maxWorkerId = 99;
// 毫秒內序列號
private final int sequenceBits = 1;
private final int sequenceMask = 10;
// 資料庫表尾號0-999,共3位
private final int tableIndexBits = 3;
// 十進位制偏移量
private final int sequenceShift = tableIndexBits;
private final int workerIdShift = sequenceBits + tableIndexBits;
private final int timestampLeftShift = workerIdBits + sequenceBits + tableIndexBits;
private final Random random = new Random();
private int workerId;
private int sequence = 0;
private long lastTimestamp = -1L;
public IdWorker(int workerId) {
if (workerId > maxWorkerId || workerId < 0) {
throw new IllegalArgumentException(String.format("worker Id can't be greater than %d or less than 0", maxWorkerId));
}
this.workerId = workerId;
}
public long nextId() {
lock.lock();
// 偏移的倍數
long timestampShiftValue;
long workerShiftValue;
long sequenceShiftValue;
// 隨機獲取資料庫表尾號
int tableIndex;
try {
long timestamp = timeGen();
if (timestamp < lastTimestamp) {
throw new RuntimeException(String.format("Clock moved backwards. Refusing to generate id for %d milliseconds", lastTimestamp - timestamp));
}
if (lastTimestamp == timestamp) {
sequence = (sequence + 1) % sequenceMask;
if (sequence == 0) {
timestamp = tilNextMillis(lastTimestamp);
}
} else {
sequence = 0;
}
lastTimestamp = timestamp;
timestampShiftValue = new Double(Math.pow(10, timestampLeftShift)).longValue();
workerShiftValue = new Double(Math.pow(10, workerIdShift)).longValue();
sequenceShiftValue = new Double(Math.pow(10, sequenceShift)).longValue();
tableIndex = random.nextInt(1000);
} finally {
lock.unlock();
}
return (timestamp + twepoch) * timestampShiftValue + workerId * workerShiftValue + sequence * sequenceShiftValue + tableIndex;
}
protected long tilNextMillis(long lastTimestamp) {
long timestamp = timeGen();
while (timestamp <= lastTimestamp) {
timestamp = timeGen();
}
return timestamp;
}
protected long timeGen() {
return System.currentTimeMillis() / 10;
}
}
2、無鎖實現:
public class IdWorker2 {
// 時間基準
private final long twepoch = 136040000000L;
private final long workerIdBits = 5L;
private final long datacenterIdBits = 3L;
private final long maxWorkerId = -1L ^ (-1L << workerIdBits);
private final long maxDatacenterId = -1L ^ (-1L << datacenterIdBits);
private final long sequenceBits = 12L;
private final long workerIdShift = sequenceBits;
private final long datacenterIdShift = sequenceBits + workerIdBits;
private final long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits;
private final long sequenceMask = -1L ^ (-1L << sequenceBits);
private long workerId;
private long datacenterId;
private AtomicLong sequence = new AtomicLong(0);
private volatile long lastTimestamp = -1L;
public IdWorker2(long workerId, long datacenterId) {
if (workerId > maxWorkerId || workerId < 0) {
throw new IllegalArgumentException(String.format("worker Id can't be greater than %d or less than 0", maxWorkerId));
}
if (datacenterId > maxDatacenterId || datacenterId < 0) {
throw new IllegalArgumentException(String.format("datacenter Id can't be greater than %d or less than 0", maxDatacenterId));
}
this.workerId = workerId;
this.datacenterId = datacenterId;
}
public long nextId() {
long currentSeq, seq, timestamp;
timestamp = timeGen();
if (timestamp < lastTimestamp) {
throw new RuntimeException(String.format("Clock moved backwards. Refusing to generate id for %d milliseconds",
lastTimestamp - timestamp));
}
for (; ; ) {
if (lastTimestamp == timestamp) {
currentSeq = sequence.incrementAndGet();
if (currentSeq > sequenceMask) { //當前毫秒的已經用完,計數器清0,等到下一個毫秒
timestamp = tilNextMillis(lastTimestamp);
sequence.compareAndSet(currentSeq, -1);
continue;
} else {
seq = currentSeq & sequenceMask;
break;
}
}
lastTimestamp = timestamp;
}
return ((timestamp + twepoch) << timestampLeftShift) | (datacenterId << datacenterIdShift) | (workerId << workerIdShift) | seq;
}
protected long tilNextMillis(long lastTimestamp) {
long timestamp = timeGen();
while (timestamp <= lastTimestamp) {
timestamp = timeGen();
}
return timestamp;
}
protected long timeGen() {
return System.currentTimeMillis();
}
}
四、結果分析
1、對比與選擇
通過對比加鎖和無鎖兩種演算法,發現併發數為100-10000的區間中,兩者差別甚微。由於twitter選取的是加鎖演算法,我們也選擇經過考證的加鎖演算法。
2、演算法特點分析
- 演算法支援單機qps為10000(自己mac上跑的結果),目前業務情況是單臺機器qps為50左右。
- 單臺機器生成的id為單調遞增。
- 演算法支援的時間截止日期為2262年(取long型最高13位9223372036850作為時間戳)。
- 演算法支援99臺機器。
- 針對業務需求,機器號需要額外獲取。
3、異常情況
(1)獲取機器號出錯。
- 服務啟動時獲取機器編號,獲取失敗則服務啟動失敗。
(2)在獲取當前 Timestamp 時, 如果獲取到的時間戳比前一個已生成 ID 的 Timestamp 還要小怎麼辦?
- 繼續獲取當前機器的時間, 直到獲取到更大的 Timestamp 才能繼續工作;
- 把 NTP 配置成不會向後調整的模式,也就是說, NTP 糾正時間時, 不會向後回撥機器時鐘。