WebSocket協議學習
websocket協議規定了客戶端和服務端socket連線和通訊時的規則，一是連線握手時的認證，二是通訊時的資料報文解析。其整個流程的簡單分析如下：
（websocket簡介參見：https://www.zhihu.com/question/20215561/answer/40316953）
1.websocket伺服器和客戶端連線
socket服務端
#coding: utf-8 import socket soc = socket.socket(socket.AF_INET,socket.SOCK_STREAM) soc.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,View Code1) soc.bind(('127.0.0.1',8080)) soc.listen(5) client,address = soc.accept() msg = client.recv(8096) print msg
websocket客戶端
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edgeView Code"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> </head> <body> <script> var web = new WebSocket("ws://127.0.0.1:8080") </script> </body> </html>
執行後可以看到客戶端發過來的請求資訊如下，比普通的http請求頭多了一個Sec-WebSocket-Key，用來進行握手認證
GET / HTTP/1.1 Host: 127.0.0.1:8080 User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:63.0) Gecko/20100101 Firefox/63.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2 Accept-Encoding: gzip, deflate Sec-WebSocket-Version: 13 Origin: http://localhost:63342 Sec-WebSocket-Extensions: permessage-deflate Sec-WebSocket-Key: lOfBaOFgUccUfIKUDD5Bxw== Connection: keep-alive, Upgrade Pragma: no-cache Cache-Control: no-cache Upgrade: websocket
服務端接受websocket客戶端的請求訊息後，若要與客戶端進行握手認證，要遵循的規則如下：
- 從上述客戶端請求資訊中提取 Sec-WebSocket-Key
- 利用magic_string 和 Sec-WebSocket-Key 進行hmac1加密，再進行base64加密 （magic string為：258EAFA5-E914-47DA-95CA-C5AB0DC85B11 固定不變）
- 將加密結果響應給客戶端
返回的請求頭如下：
HTTP/1.1 101 Switching Protocols Upgrade:websocket Connection: Upgrade Sec-WebSocket-Accept: Ip8Lp7v3m6xnPYlNIQ83SgGwrwA= WebSocket-Location: ws://127.0.0.1:8080/
Sec-WebSocket-Accept為最重要的驗證欄位，其計算過程如下：
-
將 Sec-WebSocket-Key 跟 258EAFA5-E914-47DA-95CA-C5AB0DC85B11 拼接；
-
通過 SHA1 計算出摘要，並轉成 base64 字串。
程式碼實現如下：
#coding: utf-8 import socket import base64 import hashlib #處理請求頭訊息 def get_header(data): data = str(data) header_dict={} if data: header,body = data.split('\r\n\r\n',1) header_list = header.split('\r\n') #print header_list for i in range(0,len(header_list)): if i==0: lenth = len(header_list[i].split(' ')) if lenth==3: header_dict['Method'],header_dict['Url'],header_dict['Protocol']=header_list[i].split(' ') else: k,v=header_list[i].split(':',1) header_dict[k]=v.strip() # 此處注意要去除空格，否則後面的Sec-WebSocket-Key的加密驗證會失敗 return header_dict soc = socket.socket(socket.AF_INET,socket.SOCK_STREAM) soc.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) soc.bind(('127.0.0.1',8080)) soc.listen(5) client,address = soc.accept() data = client.recv(8096) header = get_header(data) response_tpl = "HTTP/1.1 101 Switching Protocols\r\n" \ "Upgrade:websocket\r\n" \ "Connection: Upgrade\r\n" \ "Sec-WebSocket-Accept: %s\r\n" \ "WebSocket-Location: ws://%s%s\r\n\r\n" magic_string = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11' msg = header['Sec-WebSocket-Key'].strip()+magic_string #注意header['Sec-WebSocket-Key']前後是否有多餘的空格 print msg encrypt_msg = base64.b64encode(hashlib.sha1(msg).digest()) #加密得到Sec-WebSocket-Accept response_str=response_tpl%(encrypt_msg,header['Host'],header['Url']) print response_str client.send(response_str)
2.websocket服務端和客戶端通訊
websocket客戶端傳送過來的資料報文格式如下，服務端需要對報文進行解析，然後再將回復內容進行封包，傳送給客戶端。
（websocket protocol: https://tools.ietf.org/html/rfc6455#section-5.1）
相關含義如下：
The MASK bit simply tells whether the message is encoded. Messages from the client must be masked, so your server should expect this to be 1. (In fact, section 5.1 of the spec says that your server must disconnect from a client if that client sends an unmasked message.) When sending a frame back to the client, do not mask it and do not set the mask bit. We'll explain masking later. Note: You have to mask messages even when using a secure socket.RSV1-3 can be ignored, they are for extensions.
The opcode field defines how to interpret the payload data: 0x0 for continuation,
0x1for text (which is always encoded in UTF-8),0x2for binary, and other so-called "control codes" that will be discussed later. In this version of WebSockets,0x3to0x7and0xBto0xFhave no meaning.The FIN bit tells whether this is the last message in a series. If it's 0, then the server will keep listening for more parts of the message; otherwise, the server should consider the message delivered. More on this later.
Decoding Payload Length
To read the payload data, you must know when to stop reading. That's why the payload length is important to know. Unfortunately, this is somewhat complicated. To read it, follow these steps:
- Read bits 9-15 (inclusive) and interpret that as an unsigned integer. If it's 125 or less, then that's the length; you're done. If it's 126, go to step 2. If it's 127, go to step 3.
- Read the next 16 bits and interpret those as an unsigned integer. You're done.
- Read the next 64 bits and interpret those as an unsigned integer (The most significant bit MUST be 0). You're done.
Reading and Unmasking the Data
If the MASK bit was set (and it should be, for client-to-server messages), read the next 4 octets (32 bits); this is the masking key. Once the payload length and masking key is decoded, you can go ahead and read that number of bytes from the socket. Let's call the data ENCODED, and the key MASK. To get DECODED, loop through the octets (bytes a.k.a. characters for text data) of ENCODED and XOR the octet with the (i modulo 4)th octet of MASK. In pseudo-code (that happens to be valid JavaScript):
var DECODED = "";
for (var i = 0; i < ENCODED.length; i++) {
DECODED[i] = ENCODED[i] ^ MASK[i % 4];
}Now you can figure out what DECODED means depending on your application.
第一步：對客戶端資料報文解析
解包流程：
1，根據payload len的值（位元組序號1的後七位）來確定payload佔幾個位元組
2， 確定payload佔的位元組數後，其後四個位元組即為Masking-key（MASK bit 設定為1時，Masking-key才存在），Masking-key後面的所有位元組為payload data
3，利用Masking-key對payload data進行異或運算進行解碼，拿到客戶端傳送的資料
程式碼實現解包流程如下：
python 2.7
def get_data(msg): length = ord(msg[1])&127 #127的二進位制為01111111，和127進行與運算，能拿到msg[1]的後七位 if length==126: #不加ord時，msg[1]為字元竄，不支援與運算 mask = msg[4:8] pay_data = msg[8:] elif length==127: mask = msg[10:14] pay_data = msg[14:] else: mask = msg[2:6] pay_data = msg[6:] decode='' for i in range(len(pay_data)): decode+=chr(ord(pay_data[i]) ^ ord(mask[i%4])) return decode #python3環境下程式碼 # def get_data(msg): # length = msg[1]&127 # if length==126: # mask = msg[4:8] # pay_data = msg[8:] # elif length==127: # mask = msg[10:14] # pay_data = msg[14:] # else: # mask = msg[2:6] # pay_data = msg[6:] # bytes_list = bytearray() # for i in range(len(pay_data)): # chunk=pay_data[i] ^ mask[i%4] # decode=str(bytes_list.append(chunk),encoding='utf-8') # return decodeView Code
第二步：將資料封包，傳送給客戶端
返回資料報文的MASK bit為0，因此沒有Masking-key，資料報文組成：token（位元組序號0）+payload lenth +payload data
實現程式碼如下：
def response_data(msg): token = struct.pack('B',129) #寫入第一個位元組 10000001 payload_len = len(msg) if payload_len <=125: token += struct.pack('B',payload_len) elif payload_len<=126: token += struct.pack('BH',126,payload_len) else: token += struct.pack('BH', 127, payload_len) data = token+msg return dataView Code
3. 基於websocket的聊天簡單測試
客戶端：以js中的websocket做為客戶端
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> </head> <body> <div id="content" style="border:solid gray 1px; width:400px; height:400px;margin:100px 0px 0px 100px"></div> <div style="margin-left:100px"> <input type="text" id="msg"/> <button onclick="sendMsg();">傳送</button> <button onclick="closeCon();">斷開連線</button> </div> <script> var web = new WebSocket("ws://127.0.0.1:8080/"); web.onopen=function () { var newTag = document.createElement('div'); newTag.innerHTML='[連線成功]'; document.getElementById('content').appendChild(newTag); } web.onerror=function (error) { console.log('Error:'+error); } web.onmessage=function (event) { var newTag = document.createElement('div'); newTag.innerHTML=event.data; document.getElementById('content').appendChild(newTag); }; web.onclose=function () { var newTag = document.createElement('div'); newTag.innerHTML='[斷開連線]'; document.getElementById('content').appendChild(newTag); }; function sendMsg() { var mstag = document.getElementById('msg'); web.send(mstag.value); mstag.value=''; }; function closeCon() { web.close(); var newTag = document.createElement('div'); newTag.innerHTML='[斷開連線]'; document.getElementById('content').appendChild(newTag); }; </script> </body> </html>client
伺服器：基於上面的握手和通訊過程，對於客戶端發過來的訊息，回覆其訊息
#coding:utf-8 import socket import base64 import hashlib import struct #處理請求頭訊息 def get_header(data): data = str(data) header_dict={} if data: header,body = data.split('\r\n\r\n',1) header_list = header.split('\r\n') #print header_list for i in range(0,len(header_list)): if i==0: lenth = len(header_list[i].split(' ')) if lenth==3: header_dict['Method'],header_dict['Url'],header_dict['Protocol']=header_list[i].split(' ') else: k,v=header_list[i].split(':',1) header_dict[k]=v.strip() # 此處注意要去除空格，否則後面的Sec-WebSocket-Key的加密驗證會失敗 return header_dict def get_data(msg): length = ord(msg[1])&127 #127的二進位制為01111111，和127進行與運算，能拿到msg[1]的後七位 if length==126: #不加ord時，msg[1]為字元竄，不支援與運算 mask = msg[4:8] pay_data = msg[8:] elif length==127: mask = msg[10:14] pay_data = msg[14:] else: mask = msg[2:6] pay_data = msg[6:] decode='' for i in range(len(pay_data)): decode+=chr(ord(pay_data[i]) ^ ord(mask[i%4])) return decode def response_data(msg): token = struct.pack('B',129) #寫入第一個位元組 10000001 payload_len = len(msg) if payload_len <=125: token += struct.pack('B',payload_len) elif payload_len<=126: token += struct.pack('BH',126,payload_len) else: token += struct.pack('BH', 127, payload_len) data = token+msg return data def run(): soc = socket.socket(socket.AF_INET,socket.SOCK_STREAM) soc.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1) soc.bind(('127.0.0.1',8080)) soc.listen(5) client,address = soc.accept() data = client.recv(8096) header = get_header(data) response_tpl = "HTTP/1.1 101 Switching Protocols\r\n" \ "Upgrade:websocket\r\n" \ "Connection: Upgrade\r\n" \ "Sec-WebSocket-Accept: %s\r\n" \ "WebSocket-Location: ws://%s%s\r\n\r\n" magic_string = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11' hand_str = header['Sec-WebSocket-Key'].strip()+magic_string #注意header['Sec-WebSocket-Key']前後是否有多餘的空格 encrypt_str = base64.b64encode(hashlib.sha1(hand_str).digest()) response_str=response_tpl%(encrypt_str,header['Host'],header['Url']) print response_str client.send(response_str) while True: try: msg = client.recv(8096) decoded_msg = get_data(msg) print decoded_msg send_msg = response_data('回覆：'+decoded_msg) print send_msg client.send(send_msg) #client.send('%c%c%s' % (0x81, 4, 'zack')) except Exception as e: print e if __name__ == '__main__': run()server
4.tonardo框架中websocket的使用
https://www.tornadoweb.org/en/stable/websocket.html?highlight=websocket
tornado.websocket.WebSocketHandler中封裝的三個方法如下：
class EchoWebSocket(tornado.websocket.WebSocketHandler): def open(self): #客戶端連線時執行 print("WebSocket opened") def on_message(self, message): #接收到客戶端訊息時執行 self.write_message(u"You said: " + message) def on_close(self): #斷開連線時執行 print("WebSocket closed")
簡單線上聊天室實現：
app.py
#coding:utf-8 import tornado.web import tornado.websocket import tornado.ioloop import uuid Users = set() class IndexHandler(tornado.web.RequestHandler): def get(self): self.render('index.html') class ChatHandler(tornado.websocket.WebSocketHandler): def open(self): self.id = str(uuid.uuid4()) Users.add(self) def on_message(self, message): for client in Users: content = client.render_string('message.html',id=self.id,msg=message) client.write_message(content) def on_close(self): delattr(self,'id') Users.remove(self) settings={ 'template_path':'templates', 'static_path':'statics', 'static_url_prefix':'/statics/', } app = tornado.web.Application([ (r'/',IndexHandler), (r'/chat',ChatHandler), ],**settings) if __name__ == '__main__': app.listen(8000) tornado.ioloop.IOLoop.instance().start()app.py
index.html
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> <style> #content{ border:solid gray 2px; height:400px; margin:20px 0px 0px 100px; overflow: auto; } </style> </head> <body> <div style="width: 750px; margin: 0 auto"> <h3>websocket聊天室</h3> <div id="content" > </div> <div style="margin-left:100px"> <input type="text" id="msg"/> <button onclick="sendMsg();">傳送</button> <button onclick="closeCon();">斷開連線</button> </div> </div> <script src="/statics/jquery-3.3.1.min.js"></script> <script> var web = new WebSocket("ws://127.0.0.1:8000/chat"); web.onopen=function () { var newTag = document.createElement('div'); newTag.innerHTML='[連線成功]'; document.getElementById('content').appendChild(newTag); }; web.onerror=function (error) { console.log('Error:'+error); }; web.onmessage=function (event) { console.log(event); $('#content').append(event.data); //document.getElementById('content').append(event.data); 新增為字元竄，不是tag標籤？ //document.getElementById('content').appendChild(event.data); 失敗？ }; web.onclose=function () { var newTag = document.createElement('div'); newTag.innerHTML='[斷開連線]'; document.getElementById('content').appendChild(newTag); }; function sendMsg() { var mstag = document.getElementById('msg'); web.send(mstag.value); mstag.value=''; }; function closeCon() { web.close(); var newTag = document.createElement('div'); newTag.innerHTML='[斷開連線]'; document.getElementById('content').appendChild(newTag); }; </script> </body> </html>index.html
message.html
<div style="margin: 20px; background-color: green">{{id}}:{{msg}}</div>message.html
參考文章：
http://www.cnblogs.com/wupeiqi/p/6558766.html
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers
https://www.cnblogs.com/aguncn/p/5059337.html
https://www.cnblogs.com/JetpropelledSnake/p/9033064.html