tornado框架學習
阿新 • • 發佈：2018-11-11
tornado是一個非阻塞的web伺服器框架，每秒可以處理上千個客戶端連線（都是在一個執行緒中，不需要為每個客戶端建立執行緒，資源消耗少），適合用來開發web長連線應用，如long polling（輪詢），WebSocket協議等（http協議為短連線）。
1，簡單使用
#coding:utf-8 import tornado.ioloop import tornado.web from controllers.login import LoginHandler class HomeHandler(tornado.web.RequestHandler): #處理'/index'的請求，若是get請求，即呼叫get方法def get(self, *args, **kwargs): self.write('home page') settings = { 'template_path':'views' #配置html檔案的目錄，即html檔案儲存在views資料夾路徑下
'static_path':'statics', # 配置靜態url路徑，用來存放css，js檔案等
} app = tornado.web.Application([ (r'/index',HomeHandler), # 路由分發器，HomeHandler為該路由的處理類 (r'/login',LoginHandler), ],**settings) #加入配置檔案 if __name__ == '__main__': app.listen(8080) #監聽埠號 tornado.ioloop.IOLoop.instance().start() #開啟伺服器
上面程式碼即建立起一個web伺服器，在瀏覽器輸入127.0.0.1:8080/index， 就會得到包含‘home page’字元的網頁。另外，上面將所有程式碼寫在了有個程式碼檔案中，也可以利用MVC的設計方式分開來寫，如下面的的架構和程式碼：將處理‘/login’請求的類LoginHandler
#coding:utf-8 import tornado.ioloop import tornado.web from controllers.login import LoginHandler class HomeHandler(tornado.web.RequestHandler): #處理'/index'的請求，若是get請求，即呼叫get方法 def get(self, *args, **kwargs): self.write('home page') settings = { 'template_path':'views' #配置html檔案的目錄，即html檔案儲存在views資料夾路徑下 } app = tornado.web.Application([ (r'/index',HomeHandler), # 路由分發器，HomeHandler為該路由的處理類 (r'/login',LoginHandler), ],**settings) #加入配置檔案 if __name__ == '__main__': app.listen(8080) #監聽埠號 tornado.ioloop.IOLoop.instance().start() #開啟伺服器app.py
#coding:utf-8 import tornado class LoginHandler(tornado.web.RequestHandler): def get(self): self.render('login.html')login.py
2.模板
tornado也支援和django類似的模板引擎語言，表達語句用{{ item[0] }}，控制語句{% if %}。。。。 {% end %}，tornado支援if，while，for，try等，但都是以{% end %}結束，不同於django。tornado也支援模板繼承，{% extends 'index.html' %} 和 {% block body%}。。。。{% end %}（也是以{% end %}結尾）。
http://www.tornadoweb.org/en/stable/template.html
https://github.com/tornadoweb/tornado/blob/master/tornado/template.py
Tornado預設提供的這些功能其實本質上就是 UIMethod 和 UIModule，我們也可以自定義從而實現類似於Django的simple_tag的功能：
定義：
#coding:utf-8 from tornado import escape def mytag(request,value): #預設會傳遞一個引數（HomeHandler object），前端需要傳值時需要再加一個引數value #print request return '<h3>我是tag%s</h3>'%value # 前端預設會對和h3進行轉義，需要不轉義時前端使用raw 關鍵字uimethods.py
#coding:utf-8 from tornado import escape from tornado.web import UIModule class CustomUIModule(UIModule): def embedded_javascript(self): # render執行時，會在html檔案中加入javascript return "console.log(123);" def javascript_files(self): ## render執行時，會在html檔案中引入javascript檔案 return 'commons.js' def embedded_css(self): return '.but{color:red}' def css_files(self): return 'commons.css' def render(self, value): v = '<h3>我是一個UIModule tag%s</h3>'%value #預設不轉義</h3>，前端顯示我是一個UIModule tag3 #v = escape.xhtml_escape(v) # 轉義</h3>，前端顯示<h3>我是一個UIModule tag3</h3> return vuimodules.py
設定：
#coding:utf-8 import tornado.ioloop import tornado.web from controllers.login import LoginHandler import uimethods import uimodules class HomeHandler(tornado.web.RequestHandler): #處理'/index'的請求，若是get請求，即呼叫get方法 def get(self, *args, **kwargs): #self.write('home page') self.render('home.html') settings = { 'template_path':'views', #配置html檔案的目錄，即html檔案儲存在views資料夾路徑下 'static_path':'statics', # 配置靜態url路徑，用來存放css，js檔案等 'ui_methods':uimethods, 'ui_modules':uimodules, } app = tornado.web.Application([ (r'/index',HomeHandler), # 路由分發器，HomeHandler為該路由的處理類 (r'/login',LoginHandler), ],**settings) #加入配置檔案 if __name__ == '__main__': app.listen(8080) #監聽埠號 tornado.ioloop.IOLoop.instance().start() #開啟伺服器app.py
使用
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>主頁</title> </head> <body> {{ mytag(1)}} {% raw mytag(2) %} {% module CustomUIModule(3) %} <p class="but">驗證css程式碼</p> <p class="but2">驗證css檔案</p> </body> </html>home.html
網頁效果：
注意的是在UIModule中可以向html檔案中加入css，js程式碼及檔案。
3，靜態檔案設定
app配置
settings = { 'static_path':'statics', # 配置靜態url路徑，用來存放css，js檔案等 'static_url_prefix':'/statics/', #href中的起始路徑 }
html
<link rel="stylesheet" href="/statics/commons.css"> #statics目錄下的commons.css
4. 跨站請求偽造（cross site request forgery）
https://www.tornadoweb.org/en/stable/guide/security.html?highlight=ajax
app設定
settings = { "xsrf_cookies": True, }
表單使用
<form action="/new_message" method="post"> {% module xsrf_form_html() %} <input type="text" name="message"/> <input type="submit" value="Post"/> </form>
ajax使用:
本質上去cookie中獲取_xsrf，再攜帶_xsrf值提交資料（document.cookie：_xsrf=2|160fb996|ce7f56d73e0cbe6c89a74cb0f92db4b2|1541324310）
function getCookie(name) { var r = document.cookie.match("\\b" + name + "=([^;]*)\\b"); return r ? r[1] : undefined; } jQuery.postJSON = function(url, args, callback) { args._xsrf = getCookie("_xsrf"); $.ajax({url: url, data: $.param(args), dataType: "text", type: "POST", success: function(response) { callback(eval("(" + response + ")")); }}); };
function getCookie(name) { var r = document.cookie.match("\\b" + name + "=([^;]*)\\b"); return r ? r[1] : undefined; } $('#send').click(function () { var _xsrf = getCookie('_xsrf') var msg = $('#msg').val(); $.ajax({ url:'/login', data:{ '_xsrf':_xsrf, 'msg':msg, }, type:"POST", success:function (callback) { console.log(callback); } }); });
5，ajax上傳檔案
不用ajax前端
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> </head> <body> <div> <input type="file" id="img"/> <button onclick="upload();">上傳</button> </div> </body> <script src="/statics/jquery-3.3.1.min.js"></script> <script> function upload() { var file = document.getElementById('img').files[0]; var form = new FormData(); //form.append('k1','v1'); form.append('fileobj',file); var request = new XMLHttpRequest(); request.open('post','/index',true); request.send(form); } </script> </html>View Code
ajax前端
<!DOCTYPE html> <html lang="zh-CN"> <head> <meta charset="UTF-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Title</title> </head> <body> <div> <input type="file" id="img"/> <button onclick="upload();">上傳</button> </div> </body> <script src="/statics/jquery-3.3.1.min.js"></script> <script> function upload() { var file = document.getElementById('img').files[0]; var form = new FormData(); //form.append('k1','v1'); form.append('fileobj',file); //var request = new XMLHttpRequest(); //request.open('post','/index',true); //request.send(form); $.ajax({ url:'/index', type:'POST', data:form, processData:false, //讓jquery不處理資料 contentType:false, // 讓jquery不設定contentType success:function (callback) { console.log(callback); } }); } </script> </html>View Code
後端
#coding:utf-8 import tornado.web class HomeHandler(tornado.web.RequestHandler): def get(self): self.render('LoadFile.html') def post(self): fileobjs = self.request.files['fileobj'] #fileobjs為一個列表 for file in fileobjs: file_name = file['filename'] #fileobjs[0]['filename'] print type(file_name) with open(file_name,'wb') as f: f.write(file['body']) settings={ 'template_path':'views', 'static_path':'statics', 'static_url_prefix':'/statics/', } application = tornado.web.Application([ (r'/index', HomeHandler) ],**settings) if __name__ == '__main__': application.listen(8888) tornado.ioloop.IOLoop.instance().start()View Code
6，cookie
獲取和設定cookie(不加密)：
get_cookie(self, name, default=None): 未取到時返回預設值
def set_cookie(self, name, value, domain=None, expires=None, path="/",expires_days=None, **kwargs):
class HomeHandler(tornado.web.RequestHandler): #處理'/index'的請求，若是get請求，即呼叫get方法 def get(self, *args, **kwargs): #self.write('home page') if self.get_cookie(name='id'): print self.get_cookie(name='id') else: self.set_cookie(name='id',value='asdfg') self.render('home.html')View Code
獲取和設定cookie(加密)：需要在配置中設定祕鑰：'cookie_secret'
get_secure_cookie(self, name, value=None, max_age_days=31, min_version=None): 對於加密後的cookie，get_secure_cookie拿到的為解密後的cookie值，get_cookie拿到的為加密的值
set_secure_cookie(self, name, value, expires_days=30, version=None, **kwargs):
class HomeHandler(tornado.web.RequestHandler): #處理'/index'的請求，若是get請求，即呼叫get方法 def get(self, *args, **kwargs): if self.get_secure_cookie(name='secret_id'): print self.get_secure_cookie(name='secret_id') ##前端顯示的為加密後，拿到的為明文 else: self.set_secure_cookie(name='secret_id',value='message') self.render('home.html') settings = { 'template_path':'views', #配置html檔案的目錄，即html檔案儲存在views資料夾路徑下 'static_path':'statics', # 配置靜態url路徑，用來存放css，js檔案等 'static_url_prefix':'/statics/', 'ui_methods':uimethods, 'ui_modules':uimodules, 'xsrf_cookies':True, 'cookie_secret':'asdfghhj', }View Code
cookie兩個版本的加密演算法：
def _create_signature_v1(secret, *parts): hash = hmac.new(utf8(secret), digestmod=hashlib.sha1) for part in parts: hash.update(utf8(part)) return utf8(hash.hexdigest()) def _create_signature_v2(secret, s): hash = hmac.new(utf8(secret), digestmod=hashlib.sha256) hash.update(utf8(s)) return utf8(hash.hexdigest())
#加密 def create_signed_value(secret, name, value, version=None, clock=None, key_version=None): if version is None: version = DEFAULT_SIGNED_VALUE_VERSION if clock is None: clock = time.time timestamp = utf8(str(int(clock()))) value = base64.b64encode(utf8(value)) if version == 1: signature = _create_signature_v1(secret, name, value, timestamp) value = b"|".join([value, timestamp, signature]) return value elif version == 2: # The v2 format consists of a version number and a series of # length-prefixed fields "%d:%s", the last of which is a # signature, all separated by pipes. All numbers are in # decimal format with no leading zeros. The signature is an # HMAC-SHA256 of the whole string up to that point, including # the final pipe. # # The fields are: # - format version (i.e. 2; no length prefix) # - key version (integer, default is 0) # - timestamp (integer seconds since epoch) # - name (not encoded; assumed to be ~alphanumeric) # - value (base64-encoded) # - signature (hex-encoded; no length prefix) def format_field(s): return utf8("%d:" % len(s)) + utf8(s) to_sign = b"|".join([ b"2", format_field(str(key_version or 0)), format_field(timestamp), format_field(name), format_field(value), b'']) if isinstance(secret, dict): assert key_version is not None, 'Key version must be set when sign key dict is used' assert version >= 2, 'Version must be at least 2 for key version support' secret = secret[key_version] signature = _create_signature_v2(secret, to_sign) return to_sign + signature else: raise ValueError("Unsupported version %d" % version) #解密： def _decode_signed_value_v1(secret, name, value, max_age_days, clock): parts = utf8(value).split(b"|") if len(parts) != 3: return None signature = _create_signature_v1(secret, name, parts[0], parts[1]) if not _time_independent_equals(parts[2], signature): gen_log.warning("Invalid cookie signature %r", value) return None timestamp = int(parts[1]) if timestamp < clock() - max_age_days * 86400: gen_log.warning("Expired cookie %r", value) return None if timestamp > clock() + 31 * 86400: # _cookie_signature does not hash a delimiter between the # parts of the cookie, so an attacker could transfer trailing # digits from the payload to the timestamp without altering the # signature. For backwards compatibility, sanity-check timestamp # here instead of modifying _cookie_signature. gen_log.warning("Cookie timestamp in future; possible tampering %r", value) return None if parts[1].startswith(b"0"): gen_log.warning("Tampered cookie %r", value) return None try: return base64.b64decode(parts[0]) except Exception: return None def _decode_fields_v2(value): def _consume_field(s): length, _, rest = s.partition(b':') n = int(length) field_value = rest[:n] # In python 3, indexing bytes returns small integers; we must # use a slice to get a byte string as in python 2. if rest[n:n + 1] != b'|': raise ValueError("malformed v2 signed value field") rest = rest[n + 1:] return field_value, rest rest = value[2:] # remove version number key_version, rest = _consume_field(rest) timestamp, rest = _consume_field(rest) name_field, rest = _consume_field(rest) value_field, passed_sig = _consume_field(rest) return int(key_version), timestamp, name_field, value_field, passed_sig def _decode_signed_value_v2(secret, name, value, max_age_days, clock): try: key_version, timestamp, name_field, value_field, passed_sig = _decode_fields_v2(value) except ValueError: return None signed_string = value[:-len(passed_sig)] if isinstance(secret, dict): try: secret = secret[key_version] except KeyError: return None expected_sig = _create_signature_v2(secret, signed_string) if not _time_independent_equals(passed_sig, expected_sig): return None if name_field != utf8(name): return None timestamp = int(timestamp) if timestamp < clock() - max_age_days * 86400: # The signature has expired. return None try: return base64.b64decode(value_field) except Exception: return None def get_signature_key_version(value): value = utf8(value) version = _get_version(value) if version < 2: return None try: key_version, _, _, _, _ = _decode_fields_v2(value) except ValueError: return None return key_version加密和解密演算法
tornado自帶的基於cookie的驗證機制：
必須重寫方法get_current_user(self):，self.current_user()會呼叫該方法，拿到當前使用者
@tornado.web.authenticated，裝飾器修飾的請求會要求驗證，self.current_user()中拿到值時，能進行訪問，無值時跳轉到登入頁面（必須進行配置：'login_url':'/login'）
#!/usr/bin/env python # -*- coding:utf-8 -*- import tornado.ioloop import tornado.web class BaseHandler(tornado.web.RequestHandler): def get_current_user(self): return self.get_secure_cookie("login_user") class MainHandler(BaseHandler): @tornado.web.authenticated #需要登入後才能訪問（self.current_user()拿到當前使用者），否則跳轉到登入頁面 def get(self): login_user = self.current_user self.write(login_user) class LoginHandler(tornado.web.RequestHandler): def get(self): self.current_user() self.render('login.html', **{'status': ''}) def post(self, *args, **kwargs): username = self.get_argument('name') password = self.get_argument('pwd') if username == 'wupeiqi' and password == '123': self.set_secure_cookie('login_user', 'zack') self.redirect('/') else: self.render('login.html', **{'status': '使用者名稱或密碼錯誤'}) settings = { 'template_path': 'template', 'static_path': 'static', 'static_url_prefix': '/static/', 'cookie_secret': 'aiuasdhflashjdfoiuashdfiuh', 'login_url': '/login' } application = tornado.web.Application([ (r"/index", MainHandler), (r"/login", LoginHandler), ], **settings) if __name__ == "__main__": application.listen(8888) tornado.ioloop.IOLoop.instance().start()View Code
7, 自定義session框架
預備知識一：字典
任何類實現了__getitem__(), __setitem__(), __delitem__()方法，就能向字典一樣存取，刪除資料
class Adict(object): def __init__(self): self.container = {} def __getitem__(self, key): print 'get' if key in self.container: return self.container[key] else: return None def __setitem__(self, key, value): print 'set' self.container[key]=value def __delitem__(self, key): print 'del' del self.container[key] D = Adict() D['user']='zack' #呼叫 __setitem__方法 D['user'] #呼叫 __getitem__方法 del D['user'] # 呼叫 __delitem__方法View Code
預備知識二：類繼承
#coding:utf-8 #C例項化時，先呼叫A的例項化方法，而其會呼叫self.initialize()時會只執行B的initialize()方法 class A(object): def __init__(self): print 'A' self.initialize() def initialize(self): print 'A初始化' class B(A): def initialize(self): print 'B初始化' class C(B): pass c = C()單繼承
#coding:utf-8 #C例項化時，先呼叫A的例項化方法，而其會呼叫self.initialize()時會只調用B的initialize()方法，而B的initialize()方法又呼叫了A的initialize方法 class A(object): def __init__(self): print 'A' self.initialize() def initialize(self): print 'A初始化' class B(object): def initialize(self): print 'B初始化' super(B,self).initialize() #此處super先尋找其父類，沒找到，再找A的initialize方法，（先深度，後廣度） class C(B,A): pass c = C()多繼承
預備知識三：在RequestHandler的原始碼中，__init__()函式呼叫了self.initialize()函式
class RequestHandler(object): """Base class for HTTP request handlers. Subclasses must define at least one of the methods defined in the "Entry points" section below. """ SUPPORTED_METHODS = ("GET", "HEAD", "POST", "DELETE", "PATCH", "PUT", "OPTIONS") _template_loaders = {} # type: typing.Dict[str, template.BaseLoader] _template_loader_lock = threading.Lock() _remove_control_chars_regex = re.compile(r"[\x00-\x08\x0e-\x1f]") def __init__(self, application, request, **kwargs): super(RequestHandler, self).__init__() self.application = application self.request = request self._headers_written = False self._finished = False self._auto_finish = True self._transforms = None # will be set in _execute self._prepared_future = None self._headers = None # type: httputil.HTTPHeaders self.path_args = None self.path_kwargs = None self.ui = ObjectDict((n, self._ui_method(m)) for n, m in application.ui_methods.items()) # UIModules are available as both `modules` and `_tt_modules` in the # template namespace. Historically only `modules` was available # but could be clobbered by user additions to the namespace. # The template {% module %} directive looks in `_tt_modules` to avoid # possible conflicts. self.ui["_tt_modules"] = _UIModuleNamespace(self, application.ui_modules) self.ui["modules"] = self.ui["_tt_modules"] self.clear() self.request.connection.set_close_callback(self.on_connection_close) self.initialize(**kwargs) def initialize(self): """Hook for subclass initialization. Called for each request. A dictionary passed as the third argument of a url spec will be supplied as keyword arguments to initialize(). Example:: class ProfileHandler(RequestHandler): def initialize(self, database): self.database = database def get(self, username): ... app = Application([ (r'/user/(.*)', ProfileHandler, dict(database=database)), ]) """ pass原始碼
自定義session框架
#coding:utf-8 import tornado.ioloop import tornado.web from hashlib import sha1 import time import os container={} create_session_id = lambda: sha1('%s%s' % (os.urandom(16), time.time())).hexdigest() class Session(object): #一個類實現了__setitem__，__getitem__就可以向字典一樣讀取和存取資料 session_id='session_id' def __init__(self,request): session_value = request.get_cookie(Session.session_id,None) if not session_value: self._id = create_session_id() else: if session_value in container: self._id=session_value else: self._id = create_session_id() request.set_cookie(Session.session_id,self._id) if self._id not in container: container[self._id]={} def __setitem__(self, key, value): container[self._id][key]=value print container def __getitem__(self, key): if key in container[self._id]: return container[self._id][key] else: return None def __delitem__(self, key): del container[self._id][key] def clear(self): del container[self._id] # class BaseHandler(object): # def initialize(self): # self.session = Session(self) # super(BaseHandler,self).initialize() #不會覆蓋tornado.web.RequestHandler的initialiaze方法 # # class HomeHandler(BaseHandler,tornado.web.RequestHandler): # class BaseHandler(tornado.web.RequestHandler): def initialize(self): # 覆蓋tornado.web.RequestHandler的initialiaze方法，初始化時父類中會呼叫該方法 self.session = Session(self) class HomeHandler(BaseHandler): def get(self): user = self.session['user'] if user: self.write(user) else: self.redirect('/login') class LoginHandler(BaseHandler): def get(self): self.render('login.html') def post(self): username = self.get_body_argument('username') password = self.get_body_argument('password') if username=='zack' and password=='1234': self.session['user']='zack' self.session['pwd']='1234' self.redirect('/index') else: self.render('login.html') settings={ 'template_path':'views' } application = tornado.web.Application([ (r'/index', HomeHandler), (r'/login', LoginHandler), ],**settings) if __name__ == '__main__': application.listen(9999) tornado.ioloop.IOLoop.instance().start()session框架
8，非同步非阻塞
http://www.tornadoweb.org/en/stable/guide/async.html
上面都是利用tornado的同步訪問請求，當一個請求被阻塞時，下一個請求訪問時不能被處理。如下面程式碼，當先訪問‘/mani’時，由於MainHandler中，get方法sleep會阻塞在此處，此時若訪問‘/page’，也會阻塞，等待MainHandler中get方法執行完成後，才會執行PageHandler中的get方法。
#coding:utf-8 import tornado.web import tornado.ioloop from tornado.concurrent import Future import time class MainHandler(tornado.web.RequestHandler): def get(self): time.sleep(10) self.write('main') class PageHandler(tornado.web.RequestHandler): def get(self): self.write('page') application = tornado.web.Application([ (r'/main',MainHandler), (r'/page',PageHandler) ]) if __name__ == '__main__': application.listen(8888) tornado.ioloop.IOLoop.instance().start()同步阻塞
tornado中，利用裝飾器@gen.coroutine +yield Future物件，來支援非同步非阻塞。如下面程式碼，當給MainHandler中get方法加上裝飾器@gen.coroutine，並返回Future物件時，就變成了非同步非阻塞，也就是說，當我們先訪問‘/mani’時，MainHandler中get方法會阻塞在這裡，但當我們此時去訪問訪問‘/page’，PageHandler中的get方法會立即執行，而不會阻塞。
#coding:utf-8 import tornado.web import tornado.ioloop from tornado import gen from tornado.concurrent import Future import time class MainHandler(tornado.web.RequestHandler): @gen.coroutine def get(self): future = Future() yield future self.write('main') class PageHandler(tornado.web.RequestHandler): def get(self): self.write('page') application = tornado.web.Application([ (r'/main',MainHandler), (r'/page',PageHandler) ]) if __name__ == '__main__': application.listen(8888) tornado.ioloop.IOLoop.instance().start()非同步非阻塞
上面寫的非同步非阻塞並沒實際用途，下面是它的一個應用場景，在程式碼中，MainHandler的get方法中，fetch()比較耗時，但其返回一Future物件，當我們先訪問‘/mani’時，MainHandler中get方法會阻塞在這裡，但當我們此時去訪問訪問‘/page’，PageHandler中的get方法會立即執行
#coding:utf-8 import tornado.web import tornado.ioloop from tornado import gen, httpclient from tornado.concurrent import Future class MainHandler(tornado.web.RequestHandler): @gen.coroutine def get(self): http = httpclient.AsyncHTTPClient() #傳送非同步請求 data = yield http.fetch('https://www.youtube.com/',raise_error=False) #其原始碼中可以看到return future，即返回future物件 print 'done',data self.write('main') self.finish('dd') # 加入回撥函式處理 # @gen.coroutine # def get(self): # http = httpclient.AsyncHTTPClient() #傳送非同步請求 # yield http.fetch('https://www.youtube.com/',callback=self.done,raise_error=False) #其原始碼中可以看到return future，即返回future