使用lua-nginx模組實現請求解析與排程
阿新 • • 發佈:2021-03-01
**系統版本及需求**:
``OS``:CentOS 7.7.1908
``OpenResty``:1.15.8.2
[TOC]
# 描述
lua-nginx-module模組是什麼:
> It is a core component of OpenResty. If you are using this module, then you are essentially using OpenResty.
>
> By leveraging Nginx's subrequests, this module allows the integration of the powerful Lua threads (known as Lua "coroutines") into the Nginx event model.
>
> Unlike Apache's mod_lua and Lighttpd's mod_magnet, Lua code executed using this module can be 100% non-blocking on network traffic as long as the Nginx API for Lua provided by this module is used to handle requests to upstream services such as MySQL, PostgreSQL, Memcached, Redis, or upstream HTTP web services.
OpenResty的核心元件,將lua執行緒整合到nginx模型中,且不會阻塞網路流量。
[lua-nginx-module專案詳細內容](https://github.com/openresty/lua-nginx-module)
可以通過編譯將其安裝為Nginx Module。本文直接安裝OpenResty,通過lua指令碼主要實現以下兩個目標:
- 將一份請求轉發給多個後端,但僅用其中一個後端回覆請求。
- 分析一份請求的引數內容,根據規則將請求分發到不同的後端。
通過以上兩個目標,也很容易衍生出其他的可能性,例如通過此模組實現根據請求使用者的特徵將其排程到不同的伺服器:以此來達到目標(比如灰度、就近訪問、黑白名單等);根據轉發多後端特性,實現完全的真實環境壓力測試等。
# 安裝配置
## 安裝openresty
通過原始碼編譯安裝,具體步驟如下:
```bash
yum install -y pcre-devel openssl-devel gcc curl
mkdir -p /data/pkg/ && cd /data/pkg/
wget https://openresty.org/download/openresty-1.15.8.2.tar.gz
tar xf openresty-1.15.8.2.tar.gz
cd openresty-1.15.8.2
./configure --with-file-aio --with-http_ssl_module --with-http_realip_module --with-http_sub_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_stub_status_module
make -j$nproc
make install
```
編譯時Nginx的大多數選項都支援。如上啟動了一些Module,具體根據你的需求選擇。
預設安裝路徑``/usr/local/openresty``,想更改路徑通過--prefix=/path指定。
## 使用示例
建立一個存放指令碼的目錄:
```bash
cd /usr/local/openresty
mkdir nginx/conf/lua
```
建立一個lua測試指令碼:
vim nginx/conf/lua/hello.lua
```lua
local action = ngx.var.request_method
if(action == "POST") then
ngx.say("Method: POST; Hello world")
elseif(action == "GET") then
ngx.say("Method: GET; Welcome to the web site")
end
```
在Server段配置中啟用lua指令碼:
vim nginx/conf/nginx.conf
在nginx.conf中新增加一個server段,請求路徑以/開頭的則使用lua指令碼進行處理。
```conf
server {
listen 0.0.0.0:8080;
location / {
root html;
index index.html index.htm;
}
location ~* ^/(.*)$ {
content_by_lua_file "conf/lua/hello.lua"; # lua script location
}
}
```
測試效果:
使用./bin/openresty -t命令檢查配置無誤,然後使用./bin/openresty命令啟動服務。
POST和GET請求方式返回不同的相應內容
```bash
curl 127.0.0.1:8080
# 返回資訊
Method: GET; Welcome to the web site
curl -d "" 127.0.0.1:8080
# 返回資訊
Method: POST; Hello world
```
# HTTP請求複製
環境準備妥當,現在通過lua指令碼程式配合openresty實現對於HTTP請求的複製。
當一個請求來到openresty服務時,把此請求轉發給後端的server1、server2等等,但只是用server1或server2的應答訊息回覆這個請求。
一個簡單的示例圖:
![HTTP請求複製](https://img2020.cnblogs.com/blog/2286432/202102/2286432-20210228213500632-2099875475.png)
**需求**:將請求同時傳送到/prod和/test路徑後的真實後端,但只使用/prod的後端回覆client的請求
lua程式碼
vim nginx/conf/lua/copyRequest.lua
```lua
function req_copy()
local resp_prod, resp_test = ngx.location.capture_multi {
{"/prod" .. ngx.var.request_uri, arry},
{"/test" .. ngx.var.request_uri, arry},
}
if resp_prod.status == ngx.HTTP_OK then
local header_list = {"Content-Type", "Content-Encoding", "Accept-Ranges"}
for _, i in ipairs(header_list) do
if resp_prod.header[i] then
ngx.header[i] = resp_prod.header[i]
end
end
ngx.say(resp_prod.body)
else
ngx.say("Upstream server error : " .. resp_prod.status)
end
end
req_copy()
```
nginx新建的server段配置如下,且引入vhosts目錄下配置檔案,nginx/conf/nginx.conf:
```conf
server {
listen 0.0.0.0:8080;
location / {
root html;
index index.html index.htm;
}
# 匹配lua檔案中/prod+原請求的uri(/copy/*)
location ^~ /prod/ {
# 路徑重寫後,/prod後端伺服器收到的路徑為客戶端請求路徑去掉開頭/copy/
rewrite /prod/copy/(.*)$ /$1 break;
proxy_pass http://127.0.0.1:8081;
}
# 匹配lua檔案中/test+原請求的uri(/copy/*)
location ^~ /test/ {
# 路徑重寫後,/prod後端伺服器收到的路徑為客戶端請求路徑去掉開頭/copy/
rewrite /test/copy/(.*)$ /$1 break;
proxy_pass http://127.0.0.1:8082;
}
location ^~ /copy/ {
content_by_lua_file "conf/lua/copyRequest.lua";
}
}
include vhosts/*.conf;
```
建立兩個後端主機,模擬代表不同環境:
```conf
# nginx/conf/vhosts/prod.conf
server {
listen 8081;
server_name localhost;
access_log logs/prod_server.log;
location / {
return 200 "Welcome to prod server";
}
location /api/v1 {
return 200 "API V1";
}
}
# nginx/conf/vhosts/test.conf
server {
listen 8082;
server_name localhost;
access_log logs/test_server.log;
location / {
return 200 "Welcome to test server";
}
}
```
配置更新後重載服務,然後測試訪問:
```sh
> curl 127.0.0.1:8080/copy/
Welcome to prod server
> curl 127.0.0.1:8080/copy/api/v1
API V1
# /prod和/test後端的服務都收到了請求
# 檢視日誌;tail -2 nginx/logs/prod_server.log
127.0.0.1 - - [01/Mar/2020:01:41:12 +0800] "GET / HTTP/1.0" 200 22 "-" "curl/7.29.0"
127.0.0.1 - - [01/Mar/2020:01:41:15 +0800] "GET /api/v1 HTTP/1.0" 200 6 "-" "curl/7.29.0"
# 檢視日誌;tail -2 nginx/logs/test_server.log
127.0.0.1 - - [01/Mar/2020:01:41:12 +0800] "GET / HTTP/1.0" 200 22 "-" "curl/7.29.0"
127.0.0.1 - - [01/Mar/2020:01:41:15 +0800] "GET /api/v1 HTTP/1.0" 200 22 "-" "curl/7.29.0"
```
模擬場景圖示:
![lua-nginx.copyRequest](https://img2020.cnblogs.com/blog/2286432/202102/2286432-20210228213540641-1350456455.png)
1. client訪問nginx服務監聽的``IP:8080/copy/``路徑
2. lua指令碼處理收到的請求,代訪問``/prod和/test``路徑
3. ``/prod和/test``路徑在本地被代理到真實後端
4. 真實後端接收到請求並返回
5. lua指令碼僅處理``/prod(resp_prod)``後端伺服器返回的內容(見lua指令碼中程式碼)
6. 將/prod後端伺服器返回的內容返回給client
# HTTP報文解析
**需求**:將請求報文引數中的userid在某個區間的請求排程到指定的後端服務上。
建立nginx/conf/lua/requestBody.lua程式碼:
```lua
-- post body提交方式為application/x-www-form-urlencoded的內容獲取方法
function urlencodedMethod()
local postBody = {}
for key, val in pairs(args) do
postBody[key] = val
end
local uid = postBody["userid"]
postBody = nil
return tonumber(uid)
end
-- get請求方式為xx.com/?userid=x其params的獲取方式
function uriParameterMethod()
local getParameter = {}, key, val
for key, val in pairs(args) do
if type(val) == "table" then
getParameter[key] = table.concat(val)
else
getParameter[key] = val
end
end
local uid = getParameter["userid"]
getParameter = nil
return tonumber(uid)
end
-- 獲取post body提交的方式;multipart/from-data在此示例中沒有實現對其內容的處理
function contentType()
local conType = ngx.req.get_headers()["Content-Type"]
local conTypeTable = {"application/x-www-form-urlencoded", "multipart/form-data"}
local receiveConType, y
if(type(conType) == "string") then
for y = 1, 2 do
local word = conTypeTable[y]
local from, to, err = ngx.re.find(conType, word, "jo")
if from and to then
receiveConType = string.sub(conType, from, to)
end
end
else
receiveConType = nil
end
return receiveConType
end
-- 迴圈出一些需要的header返回給客戶端
function iterHeaders(resp_content)
local header_list = {"Content-Type", "Content-Encoding", "Accept-Ranges","Access-Control-Allow-Origin", "Access-Control-Allow-Methods","Access-Control-Allow-Headers", "Access-Control-Allow-Credentials"}
for _, i in ipairs(header_list) do
if(resp_content.header[i]) then
ngx.header[i] = resp_content.header[i]
end
end
return resp_content
end
-- 將userid大於等於1小於等於10的請求傳送給/prod路徑的後端
-- 將userid大於等於11小於等於20的請求傳送給/test路徑的後端
-- 將userid非以上兩種的同時傳送給/prod和/test路徑的後端,使用/prod後端回覆請求
function requestTo(uid)
local resp, resp_noReply
if(uid >= 1 and uid <= 10) then
resp = ngx.location.capture_multi {
{"/prod".. ngx.var.request_uri, arry},
}
elseif(uid >= 11 and uid <= 20) then
resp = ngx.location.capture_multi {
{"/test".. ngx.var.request_uri, arry},
}
else
resp, resp_noReply = ngx.location.capture_multi {
{"/prod" .. ngx.var.request_uri, arry},
{"/test" .. ngx.var.request_uri, arry},
}
end
local res
if(resp.status == ngx.HTTP_OK) then
res = iterHeaders(resp)
else
res = "Upstream server err : " .. reps_content.status
end
ngx.say(res.body)
end
-- 處理主函式
function main_func()
ngx.req.read_body()
local action = ngx.var.request_method
if(action == "POST") then
arry = {method = ngx.HTTP_POST, body = ngx.req.read_body()}
args = ngx.req.get_post_args()
elseif(action == "GET") then
args = ngx.req.get_uri_args()
arry = {method = ngx.HTTP_GET}
end
local u
if(action == "POST") then
if args then
local getContentType = contentType()
if(getContentType == "application/x-www-form-urlencoded") then
u = urlencodedMethod()
end
end
elseif(action == "GET") then
if args then
u = uriParameterMethod()
end
end
if(u == nil) then
ngx.say("Request parameter cannot be empty