Redis穿透問題解決方案
快取穿透
快取穿透是指使用者查詢資料,在資料庫沒有,自然在快取中也不會有。這樣就導致使用者查詢的時候,在快取中找不到,每次都要去資料庫再查詢一遍,然後返回空。這樣請求就繞過快取直接查資料庫,這也是經常提的快取命中率問題。
解決的辦法就是:如果查詢資料庫也為空,直接設定一個預設值存放到快取,這樣第二次到緩衝中獲取就有值了,而不會繼續訪問資料庫,這種辦法最簡單粗暴。
把空結果,也給快取起來,這樣下次同樣的請求就可以直接返回空了,即可以避免當查詢的值為空時引起的快取穿透。同時也可以單獨設定個快取區域儲存空值,對要查詢的key進行預先校驗,然後再放行給後面的正常快取處理邏輯。
查詢查不到的資料,在快取中沒有,而直接走了資料庫! 反反覆覆的去這麼做就崩潰了哦
4沒有,redis中沒有,然後去DB查詢,會導致雪崩效應。稱之為 穿透效應。
穿透 產生的原因:客戶端隨機生成不同的key,在redis快取中沒有該資料,資料庫也沒有該資料。這樣的話可能導致一直髮生jdbc連線
解決方案:
1、通過閘道器判斷客戶端傳入對應key的規則,不符合資料庫查詢規則,直接返回空
2、如果使用的key資料庫查詢不到的話,直接在redis中存一份null結果。
在存入id為4的資料庫的時候,直接清除對應redis為4的快取(此時是空哈)
廢話不多說,上程式碼:
pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.toov5.architect</groupId> <artifactId>architect</artifactId> <version>0.0.1-SNAPSHOT</version> <parent> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-parent</artifactId> <version>2.0.0.RELEASE</version> </parent> <dependencies> <!-- SpringBoot 對lombok 支援 --> <dependency> <groupId>org.projectlombok</groupId> <artifactId>lombok</artifactId> </dependency> <!-- SpringBoot web 核心元件 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-tomcat</artifactId> </dependency> <!-- SpringBoot 外部tomcat支援 --> <dependency> <groupId>org.apache.tomcat.embed</groupId> <artifactId>tomcat-embed-jasper</artifactId> </dependency> <!-- springboot-log4j --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-log4j</artifactId> <version>1.3.8.RELEASE</version> </dependency> <!-- springboot-aop 技術 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-aop</artifactId> </dependency> <!-- https://mvnrepository.com/artifact/commons-lang/commons-lang --> <dependency> <groupId>commons-lang</groupId> <artifactId>commons-lang</artifactId> <version>2.6</version> </dependency> <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient --> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclient</artifactId> </dependency> <!-- https://mvnrepository.com/artifact/com.alibaba/fastjson --> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.47</version> </dependency> <dependency> <groupId>javax.servlet</groupId> <artifactId>jstl</artifactId> </dependency> <dependency> <groupId>taglibs</groupId> <artifactId>standard</artifactId> <version>1.1.2</version> </dependency> <!--開啟 cache 快取 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-cache</artifactId> </dependency> <!-- ehcache快取 --> <dependency> <groupId>net.sf.ehcache</groupId> <artifactId>ehcache</artifactId> <version>2.9.1</version><!--$NO-MVN-MAN-VER$ --> </dependency> <dependency> <groupId>org.mybatis.spring.boot</groupId> <artifactId>mybatis-spring-boot-starter</artifactId> <version>1.1.1</version> </dependency> <!-- mysql 依賴 --> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> </dependency> <!-- redis 依賴 --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis</artifactId> </dependency> </dependencies> </project>
service:
package com.toov5.service; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.cache.ehcache.EhCacheCacheManager; import org.springframework.stereotype.Component; import net.sf.ehcache.Cache; import net.sf.ehcache.Element; @Component public class EhCacheUtils { // @Autowired // private CacheManager cacheManager; @Autowired private EhCacheCacheManager ehCacheCacheManager; // 新增本地快取 (相同的key 會直接覆蓋) public void put(String cacheName, String key, Object value) { Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName); Element element = new Element(key, value); cache.put(element); } // 獲取本地快取 public Object get(String cacheName, String key) { Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName); Element element = cache.get(key); return element == null ? null : element.getObjectValue(); } public void remove(String cacheName, String key) { Cache cache = ehCacheCacheManager.getCacheManager().getCache(cacheName); cache.remove(key); } }
package com.toov5.service; import java.util.Set; import java.util.concurrent.TimeUnit; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.redis.core.StringRedisTemplate; import org.springframework.stereotype.Component; @Component public class RedisService { @Autowired private StringRedisTemplate stringRedisTemplate; //這樣該方法支援多種資料型別 public void set(String key , Object object, Long time){ //開啟事務許可權 stringRedisTemplate.setEnableTransactionSupport(true); try { //開啟事務 stringRedisTemplate.multi(); String argString =(String)object; //強轉下 stringRedisTemplate.opsForValue().set(key, argString); //成功就提交 stringRedisTemplate.exec(); } catch (Exception e) { //失敗了就回滾 stringRedisTemplate.discard(); } if (object instanceof String ) { //判斷下是String型別不 String argString =(String)object; //強轉下 //存放String型別的 stringRedisTemplate.opsForValue().set(key, argString); } //如果存放Set型別 if (object instanceof Set) { Set<String> valueSet =(Set<String>)object; for(String string:valueSet){ stringRedisTemplate.opsForSet().add(key, string); //此處點選下原始碼看下 第二個引數可以放好多 } } //設定有效期 if (time != null) { stringRedisTemplate.expire(key, time, TimeUnit.SECONDS); } } //做個封裝 public void setString(String key, Object object){ String argString =(String)object; //強轉下 //存放String型別的 stringRedisTemplate.opsForValue().set(key, argString); } public void setSet(String key, Object object){ Set<String> valueSet =(Set<String>)object; for(String string:valueSet){ stringRedisTemplate.opsForSet().add(key, string); //此處點選下原始碼看下 第二個引數可以放好多 } } public String getString(String key){ return stringRedisTemplate.opsForValue().get(key); } }
package com.toov5.service; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import org.springframework.web.bind.annotation.RequestMapping; import com.toov5.entity.Users; import com.toov5.mapper.UserMapper; import io.netty.util.internal.StringUtil; @Service public class SnowslideService { @Autowired private UserMapper userMapper; @Autowired private RedisService redisService; private Lock lock = new ReentrantLock(); public String getUser01(Long id){ //定義key, key以當前的類名+方法名+id+引數值 String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName() + "-id:" + id; //1查詢redis String username = redisService.getString(key); if (!StringUtil.isNullOrEmpty(username)) { return username; } String resultUsaerName = null; try { //開啟鎖 lock.lock(); Users user = userMapper.getUser(id); if (username == null) { return null; } resultUsaerName =user.getName(); redisService.setString(key, resultUsaerName); } catch (Exception e) { // TODO: handle exception }finally { //釋放鎖 lock.unlock(); } //3直接返回 return resultUsaerName; } //穿透解決方案 public String getUser02(Long id){ //定義key, key以當前的類名+方法名+id+引數值 String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName() + "-id:" + id; //1查詢redis System.out.println("查詢redis快取"+"key"+key+".resultUserName"); String username = redisService.getString(key); if (!StringUtil.isNullOrEmpty(username)) { return username; } String resultUsaerName = null; //如果資料庫中,沒有對應的資料資訊的時候 System.out.println("查詢資料庫:id"+id); Users user = userMapper.getUser(id); if (user == null) { resultUsaerName="${null}"; //做個標記 客戶端識別到後 提示下吧 }else { resultUsaerName=user.getName(); } System.out.println("寫入redis快取"+"key"+key+".resultUserName"+resultUsaerName); redisService.setString(key, resultUsaerName); //3直接返回 return resultUsaerName; } }
package com.toov5.service; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Component; import com.alibaba.fastjson.JSONObject; import com.toov5.entity.Users; import com.toov5.mapper.UserMapper; import io.netty.util.internal.StringUtil; @Component public class UserService { @Autowired private EhCacheUtils ehCacheUtils; @Autowired private RedisService redisService; @Autowired private UserMapper userMapper; //定義個全域性的cache名字 private String cachename ="userCache"; public Users getUser(Long id){ //先查詢一級快取 key以當前的類名+方法名+id+引數值 String key = this.getClass().getName() + "-" + Thread.currentThread().getStackTrace()[1].getMethodName() + "-id:" + id; //查詢一級快取資料有對應值的存在 如果有 返回 Users user = (Users)ehCacheUtils.get(cachename, key); if (user != null) { System.out.println("key"+key+",直接從一級快取獲取資料"+user.toString()); return user; } //一級快取沒有對應的值存在,接著查詢二級快取 // redis存物件的方式 json格式 然後反序列號 String userJson = redisService.getString(key); //如果rdis快取中有這個對應的值,修改一級快取 最下面的會有的 相同會覆蓋的 if (!StringUtil.isNullOrEmpty(userJson)) { //有 轉成json JSONObject jsonObject = new JSONObject();//用的fastjson Users resultUser = jsonObject.parseObject(userJson,Users.class); ehCacheUtils.put(cachename, key, resultUser); return resultUser; } //都沒有 查詢DB Users user1 = userMapper.getUser(id); if (user1 == null) { return null; } //保證兩級快取有效期相同!? 一級快取時間-二級快取執行的時間 //一級快取時間 等於 二級快取剩下的時間 //存放到二級快取 redis中 redisService.setString(key, new JSONObject().toJSONString(user1)); //存放到一級快取 Ehchache ehCacheUtils.put(cachename, key, user1); return user1; } }
mapper
package com.toov5.mapper; import java.util.List; import org.apache.ibatis.annotations.Param; import org.apache.ibatis.annotations.Select; import org.springframework.cache.annotation.CacheConfig; import org.springframework.cache.annotation.Cacheable; import com.toov5.entity.Users; //引入的jar包後就有了這個註解了 非常好用 (配置快取的基本資訊) @CacheConfig(cacheNames={"userCache"}) //快取的名字 整個類的 public interface UserMapper { @Select("SELECT ID ,NAME,AGE FROM users where id=#{id}") @Cacheable //讓這個方法實現快取 查詢完畢後 存入到快取中 不是每個方法都需要快取呀!save()就不用了吧 Users getUser(@Param("id") Long id); }
entity
package com.toov5.entity; import java.io.Serializable; import lombok.Data; @Data public class Users implements Serializable{ private String name; private Integer age; }
controller
package com.toov5.controller; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; import com.toov5.service.SnowslideService; @RestController public class UserRedisController { @Autowired private SnowslideService snowslideService; @RequestMapping("/getUser02") public String getUser02(Long id){ return snowslideService.getUser02(id); } }
package com.toov5.controller; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RestController; import com.toov5.entity.Users; import com.toov5.service.UserService; @RestController public class IndexController { @Autowired private UserService userService; @RequestMapping("/userId") public Users getUserId(Long id){ return userService.getUser(id); } }
啟動類
package com.toov5.app; import org.mybatis.spring.annotation.MapperScan; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.cache.annotation.EnableCaching; @EnableCaching //開啟快取 @MapperScan(basePackages={"com.toov5.mapper"}) @SpringBootApplication(scanBasePackages={"com.toov5.*"}) public class app { public static void main(String[] args) { SpringApplication.run(app.class, args); } }
yml
###埠號配置 server: port: 8080 ###資料庫配置 spring: datasource: url: jdbc:mysql://localhost:3306/test username: root password: root driver-class-name: com.mysql.jdbc.Driver test-while-idle: true test-on-borrow: true validation-query: SELECT 1 FROM DUAL time-between-eviction-runs-millis: 300000 min-evictable-idle-time-millis: 1800000 # 快取配置讀取 cache: type: ehcache ehcache: config: classpath:app1_ehcache.xml redis: database: 0 jedis: pool: max-active: 8 max-wait: -1 max-idle: 8 min-idle: 0 timeout: 10000 cluster: nodes: - 192.168.91.5:9001 - 192.168.91.5:9002 - 192.168.91.5:9003 - 192.168.91.5:9004 - 192.168.91.5:9005 - 192.168.91.5:9006
<?xml version="1.0" encoding="UTF-8"?> <ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"> <diskStore path="java.io.tmpdir/ehcache-rmi-4000" /> <!-- 預設快取 --> <defaultCache maxElementsInMemory="1000" eternal="true" timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true" diskSpoolBufferSizeMB="30" maxElementsOnDisk="10000000" diskPersistent="true" diskExpiryThreadIntervalSeconds="120" memoryStoreEvictionPolicy="LRU"> </defaultCache> <!-- demo快取 --><!-- name="userCache" 對應我們在 @CacheConfig(cacheNames={"userCache"}) !!!!! --> <!--Ehcache底層也是用Map集合實現的 --> <cache name="userCache" maxElementsInMemory="1000" eternal="false" timeToIdleSeconds="120" timeToLiveSeconds="120" overflowToDisk="true" diskSpoolBufferSizeMB="30" maxElementsOnDisk="10000000" diskPersistent="false" diskExpiryThreadIntervalSeconds="120" memoryStoreEvictionPolicy="LRU"> <!-- LRU快取策略 --> <cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory" /> <!-- 用於在初始化快取,以及自動設定 --> <bootstrapCacheLoaderFactory class="net.sf.ehcache.distribution.RMIBootstrapCacheLoaderFactory" /> </cache> </ehcache>
再加一個攔截
執行結果:
把空結果,也給快取起來,這樣下次同樣的請求就可以直接返回空了,即可以避免當查詢的值為空時引起的快取穿透。同時也可以單獨設定個快取區域儲存空值,對要查詢的key進行預先校驗,然後再放行給後面的正常快取處理邏輯。
注意:再給對應的ip存放真值的時候,需要先清除對應的之前的空快取。
補充熱點key
熱點key:某個key訪問非常頻繁,當key失效的時候有打量執行緒來構建快取,導致負載增加,系統崩潰。
解決辦法:
①使用鎖,單機用synchronized,lock等,分散式用分散式鎖。
②快取過期時間不設定,而是設定在key對應的value裡。如果檢測到存的時間超過過期時間則非同步更新快取。
③在value設定一個比過期時間t0小的過期時間值t1,當t1過期的時候,延長t1並做更新快取操作。