可實現的全域性唯一有序ID生成策略
在部落格園搜素全域性唯一有序ID,羅列出來的文章大致講述了以下幾個問題,常見的生成全域性唯一id的常見方法 :使用資料庫自動增長序列實現 ; 使用UUID實現; 使用redis實現; 使用Twitter的snowflake演算法實現;使用資料庫+本地快取實現。作為一個記錄性質的部落格,簡單總結一下。
在實際的生產場景中,經常會出現如下的情況比方說訂單號:D channelNo 流水號 樣例PSDK1600000001, PSDK1600000002, PSDK1600000003... 這種具有業務意義的全域性唯一id且有序自增。先來看一下使用比較多的Twitter的snowflake演算法,snowflake生成的ID整體上按照時間自增排序,並且整個分散式系統內不會產生ID碰撞(由datacenter和workerId作區分),並且效率較高。經測試snowflake每秒能夠產生26萬個ID。一個簡單的實現如下:
/** * Twitter的分散式自增ID雪花演算法snowflake **/ public class SnowFlake { /** * 起始的時間戳 */ private final static long START_STMP = 1480166465631L; /** * 每一部分佔用的位數 */ private final static long SEQUENCE_BIT = 12; //序列號佔用的位數 private final static long MACHINE_BIT = 5; //機器標識佔用的位數 private final static long DATACENTER_BIT = 5;//資料中心佔用的位數 /** * 每一部分的最大值 */ private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT); private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT); private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT); /** * 每一部分向左的位移 */ private final static long MACHINE_LEFT = SEQUENCE_BIT; private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT; private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT; private long datacenterId; //資料中心 private long machineId; //機器標識 private long sequence = 0L; //序列號 private long lastStmp = -1L;//上一次時間戳 public SnowFlake(long datacenterId, long machineId) { if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) { throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0"); } if (machineId > MAX_MACHINE_NUM || machineId < 0) { throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0"); } this.datacenterId = datacenterId; this.machineId = machineId; } /** * 產生下一個ID * * @return */ public synchronized long nextId() { long currStmp = getNewstmp(); if (currStmp < lastStmp) { throw new RuntimeException("Clock moved backwards. Refusing to generate id"); } if (currStmp == lastStmp) { //相同毫秒內,序列號自增 sequence = (sequence + 1) & MAX_SEQUENCE; //同一毫秒的序列數已經達到最大 if (sequence == 0L) { currStmp = getNextMill(); } } else { //不同毫秒內,序列號置為0 sequence = 0L; } lastStmp = currStmp; return (currStmp - START_STMP) << TIMESTMP_LEFT //時間戳部分 | datacenterId << DATACENTER_LEFT //資料中心部分 | machineId << MACHINE_LEFT //機器標識部分 | sequence; //序列號部分 } private long getNextMill() { long mill = getNewstmp(); while (mill <= lastStmp) { mill = getNewstmp(); } return mill; } private long getNewstmp() { return System.currentTimeMillis(); } public static void main(String[] args) { SnowFlake snowFlake = new SnowFlake(1, 1); long start = System.currentTimeMillis(); for (int i = 0; i < 10; i++) { System.out.println(snowFlake.nextId()); } System.out.println(System.currentTimeMillis() - start); } }
演算法中引入了時間因子,所以可以保證生成的id唯一且有序,但是滿足不了業務欄位+流水號有序自增的要求。如果在此基礎上再配合使用資料庫本地快取自然也是可以實現的,不過複雜化了。上述程式碼執行兩次結果如下:
385063405393940480 385063405393940481 385063405393940482 385063405393940483 385063405393940484 385063405393940485 385063405393940486 385063405393940487 385063405398134784 385063405398134785 385064572152844288 385064572152844289 385064572152844290 385064572152844291 385064572152844292 385064572152844293 385064572152844294 385064572152844295 385064572152844296 385064572152844297
簡單的方法就是我們放棄自己造輪子的思想。mongodb中資料的基本單元稱為document,在一個特定集合內部需要唯一的標識文件,因此mongdb中儲存的文件都由一個‘_id’鍵,這個鍵的值可以是任意型別的ObjectId,要求不同的機器都能用全域性唯一的同種方法方便的生成它。因此不能使用自增主鍵,ObjectId 底層也是借鑑了雪花演算法,使用12位元組的儲存空間 |0|1|2|3|4|5|6 |7|8|9|10|11| |時間戳 |機器ID|PID|計數器| 前四個位元組時間戳是從標準紀元開始的時間戳,單位為秒 。時間戳保證秒級唯一,機器ID保證設計時考慮分散式,避免時鐘同步,PID保證同一臺伺服器執行多個mongod例項時的唯一性,最後的計數器保證同一秒內的唯一性。
mongo在spring boot中的引入和配置,此處不再介紹。
建立model類
package com.slowcity.admin.generate.dbmodel; import java.io.Serializable; public class BaseSequence implements Serializable{ private static final long serialVersionUID = 475722757687764546L; private String id; private String name; private Long sequence; public String getId() { return id; } public void setId(String id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public Long getSequence() { return sequence; } public void setSequence(Long sequence) { this.sequence = sequence; } @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((id == null) ? 0 : id.hashCode()); result = prime * result + ((name == null) ? 0 : name.hashCode()); result = prime * result + ((sequence == null) ? 0 : sequence.hashCode()); return result; } @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; BaseSequence other = (BaseSequence) obj; if (id == null) { if (other.id != null) return false; } else if (!id.equals(other.id)) return false; if (name == null) { if (other.name != null) return false; } else if (!name.equals(other.name)) return false; if (sequence == null) { if (other.sequence != null) return false; } else if (!sequence.equals(other.sequence)) return false; return true; } @Override public String toString() { return "BaseSequence [id=" + id + ", name=" + name + ", sequence=" + sequence + "]"; } }
public class DigitalTaskSequence extends BaseSequence{ private static final long serialVersionUID = -7287622688931253780L; }
import org.springframework.data.annotation.Id; import org.springframework.data.mongodb.core.mapping.Document; import org.springframework.stereotype.Component; @Component @Document(collection = "dm_id_task") public class DigitalTaskSequenceMG extends DigitalTaskSequence { private static final long serialVersionUID = -425011291271386371L; @Id @Override public String getId() { return super.getId(); } }
service
import java.util.List; import com.slowcity.admin.generate.dbmodel.BaseSequence; public interface SequenceGenericService { public String generateId(Class<? extends BaseSequence> clazz); List<BaseSequence> initAllId(); }
import java.util.ArrayList; import java.util.List; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; import com.slowcity.admin.generate.dbmodel.BaseSequence; import com.slowcity.admin.generate.dbmodel.DigitalTaskSequenceMG; import com.slowcity.admin.generate.repository.SequenceGenericRepository; import com.slowcity.admin.generate.service.SequenceGenericService; @Service @Transactional public class SequenceGenericServiceImpl implements SequenceGenericService { private static final Logger log = LoggerFactory.getLogger(SequenceGenericServiceImpl.class); private SequenceGenericRepository sequenceGenericRepository; public SequenceGenericServiceImpl(SequenceGenericRepository sequenceGenericRepository) { this.sequenceGenericRepository = sequenceGenericRepository; } @Override public String generateId(Class<? extends BaseSequence> clazz) { String id = sequenceGenericRepository.generateId(clazz); log.info("{} generate {}", clazz.getName(), id); return id; } @Override public List<BaseSequence> initAllId() { List<BaseSequence> baseSequenceList = new ArrayList<>(), baseSequenceResultList = new ArrayList<>(); DigitalTaskSequenceMG digitalTaskSequenceMG = new DigitalTaskSequenceMG(); digitalTaskSequenceMG.setName("sequence"); digitalTaskSequenceMG.setSequence(1210000000000000000L); //1210可以代表業務號 000000000000000代表自增流水號
baseSequenceList.add(digitalTaskSequenceMG); for (BaseSequence baseSequence:baseSequenceList) { BaseSequence resultSequence = sequenceGenericRepository.initAllId(baseSequence); if(resultSequence != null){ baseSequenceResultList.add(resultSequence); } } return baseSequenceResultList; } }
資料實現層
import com.slowcity.admin.generate.dbmodel.BaseSequence; public interface SequenceGenericRepository { public String generateId(Class<? extends BaseSequence> clazz); BaseSequence initAllId(BaseSequence Sequence); }
import java.util.List; import java.util.Map; import java.util.stream.Collectors; import org.springframework.data.mongodb.core.MongoTemplate; import org.springframework.data.mongodb.core.query.Criteria; import org.springframework.data.mongodb.core.query.Query; import org.springframework.data.mongodb.core.query.Update; import org.springframework.stereotype.Component; import com.slowcity.admin.generate.dbmodel.BaseSequence; import com.slowcity.admin.generate.repository.SequenceGenericRepository; @Component public class SequenceMongoGenericRepository implements SequenceGenericRepository { private Map<Class,Class<? extends BaseSequence>> baseSequenceMap; private MongoTemplate mongoTemplate; public SequenceMongoGenericRepository(List<BaseSequence> baseSequences, MongoTemplate mongoTemplate){ baseSequenceMap = baseSequences.stream() .collect(Collectors.toMap(baseSequence -> baseSequence.getClass().getSuperclass(), BaseSequence::getClass)); this.mongoTemplate = mongoTemplate; } @Override public String generateId(Class<? extends BaseSequence> clazz) { Class<? extends BaseSequence> childClazz = baseSequenceMap.get(clazz); if(childClazz != null) { Query query = new Query(Criteria.where("name").is("sequence")); Update update = new Update().inc("sequence", 1); Object dbm = mongoTemplate.findAndModify(query, update, childClazz); if(dbm != null) { BaseSequence bs = (BaseSequence)dbm; return String.valueOf(bs.getSequence()); } } return null; } @Override public BaseSequence initAllId(BaseSequence Sequence) { Query query = new Query(Criteria.where("name").is("sequence")); Class clazz = Sequence.getClass(); List<? extends BaseSequence> list = mongoTemplate.find(query,clazz); if(list.isEmpty()){ mongoTemplate.save(Sequence); return Sequence; } return null; } }
controller
import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMethod; import org.springframework.web.bind.annotation.RestController; import com.slowcity.admin.generate.dbmodel.BaseSequence; import com.slowcity.admin.generate.dbmodel.DigitalTaskSequence; import com.slowcity.admin.generate.service.SequenceGenericService; /** * id生成器 * @author moona * */ @RestController @RequestMapping("/generateId/task") public class TaskGenerateIdController { @Autowired private SequenceGenericService sequenceGenericService; @RequestMapping(value = "/taskId", method = RequestMethod.GET) public String generateTaskId() { return sequenceGenericService.generateId(DigitalTaskSequence.class); } @RequestMapping(value = "/init", method = RequestMethod.GET) public List<BaseSequence> generateTaskIdinit() { return sequenceGenericService.initAllId(); } }
執行初始化呼叫方法
對應資料庫
開始測試生成id
第一次呼叫:
第2次呼叫
第10次呼叫
此時再檢視資料庫,序列已經到1210000000000000011 下次呼叫直接取值了。真正做到了了分散式滿足業務的自增全域性唯一索引。mongo底層是原子性的,所以也不會出現併發的問題。如果將id生成策略部署成單臺機器服務,則可以滿足不同服務不同業務的需求,真正做到可定製可擴充套件。儘可放心使用。
【end】
&n