基於Spring Boot自建分散式基礎應用
目前剛入職了一家公司,要求替換當前系統(單體應用)以滿足每日十萬單量和一定系統使用者負載以及保證開發質量和效率。由我來設計一套基礎架構和建設基礎開發測試運維環境,github地址。
出於本公司開發現狀及成本考慮,我摒棄了市面上流行的Spring Cloud以及Dubbo分散式基礎架構,捨棄了叢集的設計,以Spring Boot和Netty為基礎自建了一套RPC分散式應用架構。可能這裡各位會有疑問,為什麼要捨棄應用的高可用呢?其實這也是跟公司的產品發展有關的,避免過度設計是非常有必要的。下面是整個系統的架構設計圖。
這裡簡單介紹一下,這裡ELK或許並非最好的選擇,可以另外採用zabbix或者prometheus,我只是考慮了後續可能的擴充套件。資料庫採用了兩種儲存引擎,便是為了因對上面所說的每天十萬單的大資料量,可以採用定時指令碼的形式完成資料的轉移。
許可權的設計主要是基於JWT+Filter+Redis來做的。Common工程中的com.imspa.web.auth.Permissions定義了所有需要的permissions:
1 package com.imspa.web.auth; 2 3 /** 4 * @author Pann 5 * @description TODO 6 * @date 2019-08-12 15:09 7 */ 8 public enum Permissions { 9 ALL("/all", "所有許可權"), 10 ROLE_GET("/role/get/**", "許可權獲取"), 11 USER("/user", "使用者列表"), 12 USER_GET("/user/get", "使用者查詢"), 13 RESOURCE("/resource", "資源獲取"), 14 ORDER_GET("/order/get/**","訂單查詢"); 15 16 private String url; 17 private String desc; 18 19 Permissions(String url, String desc) { 20 this.url = url; 21 this.desc = desc; 22 } 23 24 public String getUrl() { 25 return this.url; 26 } 27 28 public String getDesc() { 29 return this.desc; 30 } 31 }
如果你的沒有為你的介面在這裡定義許可權,那麼系統是不會對該介面進行許可權的校驗的。在資料庫中User與Role的設計如下:
1 CREATE TABLE IF NOT EXISTS `t_user` ( 2 `id` VARCHAR(36) NOT NULL, 3 `name` VARCHAR(20) NOT NULL UNIQUE, 4 `password_hash` VARCHAR(255) NOT NULL, 5 `role_id` VARCHAR(36) NOT NULL, 6 `role_name` VARCHAR(20) NOT NULL, 7 `last_login_time` TIMESTAMP(6) NULL, 8 `last_login_client_ip` VARCHAR(15) NULL, 9 `created_time` TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6), 10 `created_by` VARCHAR(36) NOT NULL, 11 `updated_time` TIMESTAMP(6) NULL, 12 `updated_by` VARCHAR(36) NULL, 13 PRIMARY KEY (`id`) 14 ); 15 16 CREATE TABLE IF NOT EXISTS `t_role` ( 17 `id` VARCHAR(36) NOT NULL, 18 `role_name` VARCHAR(20) NOT NULL UNIQUE, 19 `description` VARCHAR(90) NULL, 20 `permissions` TEXT NOT NULL, #其資料格式類似於"/role/get,/user"或者"/all" 21 `created_time` TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6), 22 `created_by` VARCHAR(36) NOT NULL, 23 `updated_time` TIMESTAMP(6) NULL, 24 `updated_by` VARCHAR(36) NULL, 25 PRIMARY KEY (`id`) 26 );
需要注意的是"/all"代表了所有許可權,表示root許可權。我們通過postman呼叫登陸介面可以獲取相應的token:
這個token是半個小時失效的,如果你需要更長一些的話,可以通過com.imspa.web.auth.TokenAuthenticationService進行修改:
1 package com.imspa.web.auth; 2 3 import com.imspa.web.util.WebConstant; 4 import io.jsonwebtoken.Jwts; 5 import io.jsonwebtoken.SignatureAlgorithm; 6 7 import java.util.Date; 8 import java.util.Map; 9 10 /** 11 * @author Pann 12 * @description TODO 13 * @date 2019-08-14 23:24 14 */ 15 public class TokenAuthenticationService { 16 static final long EXPIRATIONTIME = 30 * 60 * 1000; //TODO 17 18 public static String getAuthenticationToken(Map<String, Object> claims) { 19 return "Bearer " + Jwts.builder() 20 .setClaims(claims) 21 .setExpiration(new Date(System.currentTimeMillis() + EXPIRATIONTIME)) 22 .signWith(SignatureAlgorithm.HS512, WebConstant.WEB_SECRET) 23 .compact(); 24 } 25 }
Refresh Token目前還沒有實現,後續我會更新,請關注我的github。如果你跟蹤登陸邏輯程式碼,你可以看到我把role和user都快取到了Redis:
1 public User login(String userName, String password) { 2 UserExample example = new UserExample(); 3 example.createCriteria().andNameEqualTo(userName); 4 5 User user = userMapper.selectByExample(example).get(0); 6 if (null == user) 7 throw new UnauthorizedException("user name not exist"); 8 9 if (!StringUtils.equals(password, user.getPasswordHash())) 10 throw new UnauthorizedException("user name or password wrong"); 11 12 roleService.get(user.getRoleId()); //for role cache 13 14 hashOperations.putAll(RedisConstant.USER_SESSION_INFO_ + user.getName(), hashMapper.toHash(user)); 15 hashOperations.getOperations().expire(RedisConstant.USER_SESSION_INFO_ + user.getName(), 30, TimeUnit.MINUTES); 16 17 return user; 18 }
在Filter中,你可以看到過濾器的一系列邏輯,注意返回http狀態碼401,403和404的區別:
1 package com.imspa.web.auth; 2 3 import com.imspa.web.Exception.ForbiddenException; 4 import com.imspa.web.Exception.UnauthorizedException; 5 import com.imspa.web.pojo.Role; 6 import com.imspa.web.pojo.User; 7 import com.imspa.web.util.RedisConstant; 8 import com.imspa.web.util.WebConstant; 9 import io.jsonwebtoken.Claims; 10 import io.jsonwebtoken.Jwts; 11 import org.apache.commons.lang3.StringUtils; 12 import org.apache.logging.log4j.LogManager; 13 import org.apache.logging.log4j.Logger; 14 import org.springframework.data.redis.core.HashOperations; 15 import org.springframework.data.redis.hash.HashMapper; 16 import org.springframework.util.AntPathMatcher; 17 18 import javax.servlet.Filter; 19 import javax.servlet.FilterChain; 20 import javax.servlet.FilterConfig; 21 import javax.servlet.ServletException; 22 import javax.servlet.ServletOutputStream; 23 import javax.servlet.ServletRequest; 24 import javax.servlet.ServletResponse; 25 import javax.servlet.http.HttpServletRequest; 26 import javax.servlet.http.HttpServletResponse; 27 import java.io.IOException; 28 import java.util.Date; 29 import java.util.HashMap; 30 import java.util.Map; 31 import java.util.Optional; 32 import java.util.concurrent.TimeUnit; 33 34 /** 35 * @author Pann 36 * @description TODO 37 * @date 2019-08-16 14:39 38 */ 39 public class SecurityFilter implements Filter { 40 private static final Logger logger = LogManager.getLogger(SecurityFilter.class); 41 private AntPathMatcher matcher = new AntPathMatcher(); 42 private HashOperations<String, byte[], byte[]> hashOperations; 43 private HashMapper<Object, byte[], byte[]> hashMapper; 44 45 public SecurityFilter(HashOperations<String, byte[], byte[]> hashOperations, HashMapper<Object, byte[], byte[]> hashMapper) { 46 this.hashOperations = hashOperations; 47 this.hashMapper = hashMapper; 48 } 49 50 @Override 51 public void init(FilterConfig filterConfig) throws ServletException { 52 53 } 54 55 @Override 56 public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain chain) throws IOException, ServletException { 57 HttpServletRequest request = (HttpServletRequest) servletRequest; 58 HttpServletResponse response = (HttpServletResponse) servletResponse; 59 60 Optional<String> optional = PermissionUtil.getAllPermissionUrlItem().stream() 61 .filter(permissionItem -> matcher.match(permissionItem, request.getRequestURI())).findFirst(); 62 if (!optional.isPresent()) { //TODO some api not config permission will direct do 63 chain.doFilter(servletRequest, servletResponse); 64 return; 65 } 66 67 try { 68 validateAuthentication(request, optional.get()); 69 flushSessionAndToken(((User) request.getAttribute("userInfo")), response); 70 chain.doFilter(servletRequest, servletResponse); 71 } catch (ForbiddenException e) { 72 logger.debug("occur forbidden exception:{}", e.getMessage()); 73 response.setStatus(403); 74 ServletOutputStream output = response.getOutputStream(); 75 output.print(e.getMessage()); 76 output.flush(); 77 } catch (UnauthorizedException e) { 78 logger.debug("occur unauthorized exception:{}", e.getMessage()); 79 response.setStatus(401); 80 ServletOutputStream output = response.getOutputStream(); 81 output.print(e.getMessage()); 82 output.flush(); 83 } 84 } 85 86 @Override 87 public void destroy() { 88 89 } 90 91 private void validateAuthentication(HttpServletRequest request, String permission) { 92 String authHeader = request.getHeader("Authorization"); 93 if (StringUtils.isEmpty(authHeader)) 94 throw new UnauthorizedException("no auth header"); 95 96 Claims claims; 97 try { 98 claims = Jwts.parser().setSigningKey(WebConstant.WEB_SECRET) 99 .parseClaimsJws(authHeader.replace("Bearer ", "")) 100 .getBody(); 101 } catch (Exception e) { 102 throw new UnauthorizedException(e.getMessage()); 103 } 104 105 String userName = (String) claims.get("user"); 106 String roleId = (String) claims.get("role"); 107 108 if (StringUtils.isEmpty(userName) || StringUtils.isEmpty(roleId)) 109 throw new UnauthorizedException("token error,user:" + userName); 110 111 if (new Date().getTime() > claims.getExpiration().getTime()) 112 throw new UnauthorizedException("token expired,user:" + userName); 113 114 115 User user = (User) hashMapper.fromHash(hashOperations.entries(RedisConstant.USER_SESSION_INFO_ + userName)); 116 if (user == null) 117 throw new UnauthorizedException("session expired,user:" + userName); 118 119 120 if (validateRolePermission(permission, user)) 121 request.setAttribute("userInfo", user); 122 } 123 124 private Boolean validateRolePermission(String permission, User user) { 125 Role role = (Role) hashMapper.fromHash(hashOperations.entries(RedisConstant.ROLE_PERMISSION_MAPPING_ + user.getRoleId())); 126 if (role.getPermissions().contains(Permissions.ALL.getUrl())) 127 return Boolean.TRUE; 128 129 if (role.getPermissions().contains(permission)) 130 return Boolean.TRUE; 131 132 throw new ForbiddenException("do not have permission for this request"); 133 } 134 135 private void flushSessionAndToken(User user, HttpServletResponse response) { 136 hashOperations.getOperations().expire(RedisConstant.USER_SESSION_INFO_ + user.getName(), 30, TimeUnit.MINUTES); 137 138 Map<String, Object> claimsMap = new HashMap<>(); 139 claimsMap.put("user", user.getName()); 140 claimsMap.put("role", user.getRoleId()); 141 response.setHeader("Authorization",TokenAuthenticationService.getAuthenticationToken(claimsMap)); 142 } 143 144 }
下面是RPC的內容,我是用Netty來實現整個RPC的呼叫的,其中包含了心跳檢測,自動重連的過程,基於Spring Boot的實現,配置和使用都還是很方便的。
我們先看一下service端的寫法,我們需要先定義好對外服務的介面,這裡我們在application.yml中定義:
1 service: 2 addr: localhost:8091 3 interfaces: 4 - 'com.imspa.api.OrderRemoteService'
其中service.addr是對外發布的地址,service.interfaces是對外發布的介面的定義。然後便不需要你再定義其他內容了,是不是很方便?其實現你可以根據它的配置類com.imspa.config.RPCServiceConfig來看:
1 package com.imspa.config; 2 3 import com.imspa.rpc.core.RPCRecvExecutor; 4 import com.imspa.rpc.model.RPCInterfacesWrapper; 5 import org.springframework.beans.factory.annotation.Value; 6 import org.springframework.boot.context.properties.ConfigurationProperties; 7 import org.springframework.boot.context.properties.EnableConfigurationProperties; 8 import org.springframework.context.annotation.Bean; 9 import org.springframework.context.annotation.Configuration; 10 11 /** 12 * @author Pann 13 * @description config order server's RPC service method 14 * @date 2019-08-08 14:51 15 */ 16 @Configuration 17 @EnableConfigurationProperties 18 public class RPCServiceConfig { 19 @Value("${service.addr}") 20 private String addr; 21 22 @Bean 23 @ConfigurationProperties(prefix = "service") 24 public RPCInterfacesWrapper serviceContainer() { 25 return new RPCInterfacesWrapper(); 26 } 27 28 @Bean 29 public RPCRecvExecutor recvExecutor() { 30 return new RPCRecvExecutor(addr); 31 } 32 33 }
在client端,我們也僅僅只需要在com.imspa.config.RPCReferenceConfig中配置一下我們這個工程所需要呼叫的service 介面(注意所需要配置的內容哦):
1 package com.imspa.config; 2 3 import com.imspa.api.OrderRemoteService; 4 import com.imspa.rpc.core.RPCSendExecutor; 5 import org.springframework.context.annotation.Bean; 6 import org.springframework.context.annotation.Configuration; 7 8 /** 9 * @author Pann 10 * @Description config this server need's reference bean 11 * @Date 2019-08-08 16:55 12 */ 13 @Configuration 14 public class RPCReferenceConfig { 15 @Bean 16 public RPCSendExecutor orderService() { 17 return new RPCSendExecutor<OrderRemoteService>(OrderRemoteService.class,"localhost:8091"); 18 } 19 20 }
然後你就可以在程式碼裡面正常的使用了
1 package com.imspa.resource.web; 2 3 import com.imspa.api.OrderRemoteService; 4 import com.imspa.api.order.OrderDTO; 5 import com.imspa.api.order.OrderVO; 6 import org.springframework.beans.factory.annotation.Autowired; 7 import org.springframework.web.bind.annotation.GetMapping; 8 import org.springframework.web.bind.annotation.PathVariable; 9 import org.springframework.web.bind.annotation.RequestMapping; 10 import org.springframework.web.bind.annotation.RestController; 11 12 import java.math.BigDecimal; 13 import java.util.Arrays; 14 import java.util.List; 15 16 /** 17 * @author Pann 18 * @Description TODO 19 * @Date 2019-08-08 16:51 20 */ 21 @RestController 22 @RequestMapping("/resource") 23 public class ResourceController { 24 @Autowired 25 private OrderRemoteService orderRemoteService; 26 27 @GetMapping("/get/{id}") 28 public OrderVO get(@PathVariable("id")String id) { 29 OrderDTO orderDTO = orderRemoteService.get(id); 30 return new OrderVO().setOrderId(orderDTO.getOrderId()).setOrderPrice(orderDTO.getOrderPrice()) 31 .setProductId(orderDTO.getProductId()).setProductName(orderDTO.getProductName()) 32 .setStatus(orderDTO.getStatus()).setUserId(orderDTO.getUserId()); 33 } 34 35 @GetMapping() 36 public List<OrderVO> list() { 37 return Arrays.asList(new OrderVO().setOrderId("1").setOrderPrice(new BigDecimal(2.3)).setProductName("西瓜")); 38 } 39 }
以上是本基礎架構的大概內容,還有很多其他的內容和後續更新請關注我的github,筆芯。
&n