1. 程式人生 > >Flask核心機制--上下文原始碼剖析

Flask核心機制--上下文原始碼剖析

一、前言

  瞭解過flask的python開發者想必都知道flask中核心機制莫過於上下文管理,當然學習flask如果不瞭解其中的處理流程,可能在很多問題上不能得到解決,當然我在寫本篇文章之前也看到了很多博文有關於對flask上下文管理的剖析都非常到位,當然為了學習flask我也把對flask上下文理解寫下來供自己參考,也希望對其他人有所幫助。

二、知識儲備

threadlocal

  在多執行緒中,執行緒間的資料是共享的, 但是每個執行緒想要有自己的資料該怎麼實現? python中的threading.local物件已經實現,其原理是利用執行緒的唯一標識作為key,資料作為value來儲存其自己的資料,以下是demo演示了多個執行緒同時修改同一變數的值的結果:

#!/usr/bin/env python3
# -*- coding:utf-8 -*-
# Author:wd

import threading
import time
values=threading.local()

def run(arg):
    values.num=arg  #修改threading.local物件的name資料
    time.sleep(1)
    print(threading.current_thread().name,values.num)  #列印values.num


for i in range(3):
    th 
= threading.Thread(target=run, args=(i,), name='run thread%s' % i) th.start() 結果: run thread0 0 run thread1 1 run thread2 2

結果說明:

從結果中可以看到,values.num的值是不同的,按照普通執行緒理解因為有sleep存在,在每個執行緒最後列印values.num時候值應該都是2,但是正是因為threading.local物件內部會為每個執行緒開闢一個記憶體空間,從而使得每個執行緒都有自己的單獨資料,所以每個執行緒修改的是自己的資料(內部實現為字典),列印結果才不一樣。

有了以上的設計思想,我們可以自己定義類似於thread.local類,為了支援協程,將其唯一標識改為協程的唯一標識,其實這已經及其接近flask中的Local類了(後續在進行說明):

try:
    from greenlet import getcurrent as get_ident  # 攜程唯一標識
except ImportError:
    try:
        from thread import get_ident
    except ImportError:
        from _thread import get_ident  # 執行緒唯一標識


class Local(object):
    def __init__(self):
        object.__setattr__(self, 'storage', dict())  # 防止self.xxx 遞迴
        object.__setattr__(self, '__get_ident__', get_ident)

    def __setattr__(self, key, value):
        ident = self.__get_ident__()  # 獲取當前執行緒或協程的唯一標識
        data = self.storage.get(ident)
        if not data:  # 當前執行緒沒有資料
            data = {key: value}  # 建立資料
        else:  # 當前已經有資料
            data[key] = value

        self.storage[ident] = data  # 最後為當前執行緒設定其標識對應的資料

    def __getattr__(self, name):
        try:
            return self.storage[self.__get_ident__()].get(name)  # 返回name所對應的值
        except KeyError:
            raise AttributeError(name)

functools.partial

  partial函式是工具包的一個不常用函式,其作用是給函式傳遞引數,同時返回的也是這個函式,但是這個函式的已經帶了引數了,示例:

from functools import partial

def func(x,y,z):
    print(x,y,z)

new_fun=partial(func,1,2)  #生成新的函式,該函式中已經有一個引數
new_fun(3)

結果:
1 2 3

在以上示例中,new_func是由func生成的,它已經引數1,2了,只需要傳遞3即可執行。

werkzeug

  werkzeug是一個實現了wsgi協議的模組,用官方語言介紹:Werkzeug is a WSGI utility library for Python. It's widely used and BSD licensed。為什麼會提到它呢,這是因為flask內部使用的wsgi模組就是werkzeug,以下是一個示例(如果你瞭解wsgi協議的應該不用過多介紹):

from werkzeug.wrappers import Request, Response

@Request.application
def application(request):
    return Response('Hello World!')

if __name__ == '__main__':
    from werkzeug.serving import run_simple
    run_simple('localhost', 4000, application)

在示例中application是一個可呼叫的物件也可以是帶有__call__方法的物件,在run_simple內部執行application(),也就是在原始碼的execute(self.server.app)中執行,這裡你只需要run_simple會執行第三個引數加括號。

三、原始碼剖析

上下文管理

   在說請求上下文之前先看一個flask的hell world示例:

from flask import Flask

app=Flask(__name__)
@app.route("/")
def hello():
    return 'hello world'

if __name__=='__main__':
    app.run()

在以上示例中,app.run是請求的入口,而app是Flask例項化的物件,所以執行的是Flask類中的run方法,而在該改方法中又執行了run_simple方法,以下是run方法部分原始碼摘抄(其中self就是app物件):
from werkzeug.serving import run_simple

try:
    run_simple(host, port, self, **options)
finally:
    # reset the first request information if the development server
    # reset normally.  This makes it possible to restart the server
    # without reloader and that stuff from an interactive shell.
    self._got_first_request = False

在run_simple中會執行app(environ, start_response),參考werkzeug的原始碼,原始碼會執行app(environ, start_response)也就是執行app的__call__方法,以下是__call__方法原始碼摘抄:
def __call__(self, environ, start_response):
    """The WSGI server calls the Flask application object as the
    WSGI application. This calls :meth:`wsgi_app` which can be
    wrapped to applying middleware."""
    return self.wsgi_app(environ, start_response)

__call__方法中又呼叫了wsgi_app方法,該方法也就是flask的核心所在,下面是方法摘抄:

def wsgi_app(self, environ, start_response):
    """The actual WSGI application. This is not implemented in
    :meth:`__call__` so that middlewares can be applied without
    losing a reference to the app object. Instead of doing this::

        app = MyMiddleware(app)

    It's a better idea to do this instead::

        app.wsgi_app = MyMiddleware(app.wsgi_app)

    Then you still have the original application object around and
    can continue to call methods on it.

    .. versionchanged:: 0.7
        Teardown events for the request and app contexts are called
        even if an unhandled error occurs. Other events may not be
        called depending on when an error occurs during dispatch.
        See :ref:`callbacks-and-errors`.

    :param environ: A WSGI environment.
    :param start_response: A callable accepting a status code,
        a list of headers, and an optional exception context to
        start the response.
    """
    #ctx.app 當前app名稱
    #ctx.request request物件,由app.request_class(environ)生成
    #ctx.session session 相關資訊
    ctx = self.request_context(environ) 
    error = None
    try:
        try:
            ctx.push()
            #push資料到local,此時push的資料分請求上線文和應用上下文
            # 將ctx通過Localstack新增到local中
            # app_ctx是APPContext物件
            response = self.full_dispatch_request()
        except Exception as e:
            error = e
            response = self.handle_exception(e)
        except:
            error = sys.exc_info()[1]
            raise
        return response(environ, start_response)
    finally:
        if self.should_ignore_error(error):
            error = None
        ctx.auto_pop(error)

第一句:ctx = self.request_context(environ)呼叫request_context例項化RequestContext物件,以下是RequestContext類的構造方法:

def __init__(self, app, environ, request=None):
    self.app = app
    if request is None:
        request = app.request_class(environ)
    self.request = request
    self.url_adapter = app.create_url_adapter(self.request)
    self.flashes = None
    self.session = None

此時的request為None,所以self.request=app.request_class(environ),而在Flask類中request_class = Request,此時執行的是Request(environ),也就是例項化Request類,用於封裝請求資料,最後返回RequestContext物件,此時的ctx含有以下屬性ctx.app(app物件)、ctx.request(請求封裝的所有請求資訊)、ctx.app(當前app物件)等

第二句:ctx.push(), 呼叫RequestContext的push方法,以下是原始碼摘抄:
def push(self):
    """Binds the request context to the current context."""
    # If an exception occurs in debug mode or if context preservation is
    # activated under exception situations exactly one context stays
    # on the stack.  The rationale is that you want to access that
    # information under debug situations.  However if someone forgets to
    # pop that context again we want to make sure that on the next push
    # it's invalidated, otherwise we run at risk that something leaks
    # memory.  This is usually only a problem in test suite since this
    # functionality is not active in production environments.
    top = _request_ctx_stack.top
    if top is not None and top.preserved:
        top.pop(top._preserved_exc)

    # Before we push the request context we have to ensure that there
    # is an application context.
    app_ctx = _app_ctx_stack.top  #獲取應用上線文,一開始為none
    if app_ctx is None or app_ctx.app != self.app:
        # 建立APPContext(self)物件,app_ctx=APPContext(self)
        # 包含app_ctx.app  ,當前app物件
        # 包含app_ctx.g  , g可以看作是一個字典用來儲存一個請求週期需要儲存的值
        app_ctx = self.app.app_context()
        app_ctx.push()
        self._implicit_app_ctx_stack.append(app_ctx)
    else:
        self._implicit_app_ctx_stack.append(None)

    if hasattr(sys, 'exc_clear'):
        sys.exc_clear()
    #self 是RequestContext物件,其中包含了請求相關的所有資料
    _request_ctx_stack.push(self)

    # Open the session at the moment that the request context is available.
    # This allows a custom open_session method to use the request context.
    # Only open a new session if this is the first time the request was
    # pushed, otherwise stream_with_context loses the session.
    if self.session is None:
        session_interface = self.app.session_interface  # 獲取session資訊
        self.session = session_interface.open_session(
            self.app, self.request
        )

        if self.session is None:
            self.session = session_interface.make_null_session(self.app)

到了這裡可以看到,相關注解已經標註,flask內部將上下文分為了app_ctx(應用上下文)和_request_ctx(請求上下文),並分別用來兩個LocalStack()來存放各自的資料(以下會用request_ctx說明,當然app_ctx也一樣),其中app_ctx包含app、url_adapter一下是app_ctx構造方法:
def __init__(self, app):
    self.app = app
    self.url_adapter = app.create_url_adapter(None)
    self.g = app.app_ctx_globals_class()

    # Like request context, app contexts can be pushed multiple times
    # but there a basic "refcount" is enough to track them.
    self._refcnt = 0

然後分別執行app_ctx.push()方法和_request_ctx_stack.push(self)方法,將資料push到stack上,_request_ctx_stack.push(self),而_request_ctx_stack是一個LocalStack物件,是一個全域性物件,具體路徑在flask.globals,以下是其push方法: 
def push(self, obj):
    """Pushes a new item to the stack"""
    #找_local物件中是否有stack,沒有設定rv和_local.stack都為[]
    rv = getattr(self._local, 'stack', None)
    if rv is None:
        self._local.stack = rv = []
        # 執行Local物件的__setattr__方法,等價於a=[],rv=a, self._local.stack =a
        #建立字典,類似於storage={'唯一標識':{'stack':[]}}
    rv.append(obj)
        #列表中追加請求相關所有資料也就是storage={'唯一標識':{'stack':[RequestContext物件,]}}
    return rv

以上程式碼中的self._local是一個Local()物件原始碼定義如下,也就是用於儲存每次請求的資料,和我們剛開始定義的local及其相似,這也是為什麼要先提及下threadlocal。
class Local(object):
    __slots__ = ('__storage__', '__ident_func__')

    def __init__(self):
        object.__setattr__(self, '__storage__', {})
        object.__setattr__(self, '__ident_func__', get_ident)

    def __iter__(self):
        return iter(self.__storage__.items())

    def __call__(self, proxy):
        """Create a proxy for a name."""
        return LocalProxy(self, proxy)

    def __release_local__(self):
        self.__storage__.pop(self.__ident_func__(), None)

    def __getattr__(self, name):
        try:
            return self.__storage__[self.__ident_func__()][name]
        except KeyError:
            raise AttributeError(name)

    def __setattr__(self, name, value):
        ident = self.__ident_func__()
        storage = self.__storage__
        try:
            storage[ident][name] = value
        except KeyError:
            storage[ident] = {name: value}

    def __delattr__(self, name):
        try:
            del self.__storage__[self.__ident_func__()][name]
        except KeyError:
            raise AttributeError(name)
Local()

到這裡我們知道了,當執行ctx.push()時,local物件中已經有資料了,接著開始執行self.full_dispatch_request(),也就是開始執行檢視函式,以下是原始碼摘抄:
def full_dispatch_request(self):
    """Dispatches the request and on top of that performs request
    pre and postprocessing as well as HTTP exception catching and
    error handling.

    .. versionadded:: 0.7
    """
    self.try_trigger_before_first_request_functions()
    try:
        request_started.send(self)
        rv = self.preprocess_request()
        if rv is None:
            rv = self.dispatch_request()
    except Exception as e:
        rv = self.handle_user_exception(e)
    return self.finalize_request(rv)

在改方法中呼叫self.preprocess_request(),用於執行所有被before_request裝飾器裝飾的函式,從原始碼總可以看到如果該函式有返回,則不會執行self.dispatch_request()也就是檢視函式, 執行完畢之後呼叫self.dispatch_request()根據路由匹配執行檢視函式,然後響應最後呼叫ctx.auto_pop(error)將stack中的資料刪除,此時完成一次請求。  

全域性物件request、g、session

   在瞭解完flask的上下文管理時候,我們在檢視函式中使用的request實際上是一個全域性變數物件,當然還有g、session這裡以request為例子,它是一個LocalProxy物件,以下是原始碼片段:

request = LocalProxy(partial(_lookup_req_object, 'request'))

當我們使用request.path時候實際上是呼叫是其__getattr__方法即LocalProxy物件的__getattr__方法,我們先來看看LocalProxy物件例項化的引數:

def __init__(self, local, name=None):
    #local是傳入的函式,該句等價於self.__local=local,_類名__欄位強行設定私有欄位值
    #如果是requst則函式就是partial(_lookup_req_object, 'request')
    object.__setattr__(self, '_LocalProxy__local', local)
    object.__setattr__(self, '__name__', name) #開始的時候設定__name__的值為None
    if callable(local) and not hasattr(local, '__release_local__'):
        # "local" is a callable that is not an instance of Local or
        # LocalManager: mark it as a wrapped function.
        object.__setattr__(self, '__wrapped__', local)

在原始碼中例項化時候傳遞的是partial(_lookup_req_object, 'request')函式作為引數,也就是self.__local=該函式,partial引數也就是我們之前提到的partial函式,作用是傳遞引數,此時為_lookup_req_object函式傳遞request引數,這個在看看其__getattr__方法:

def __getattr__(self, name):
    #以獲取request.method 為例子,此時name=method
    if name == '__members__':
        return dir(self._get_current_object())
    #self._get_current_object()返回的是ctx.request,再從ctx.request獲取method (ctx.request.method)
    return getattr(self._get_current_object(), name)

在以上方法中會呼叫self._get_current_object()方法,而_get_current_object()方法中會呼叫self.__local()也就是帶引數request引數的 _lookup_req_object方法從而返回ctx.request(請求上下文),最後通過然後反射獲取name屬性的值,這裡我們name屬性是path,如果是request.method name屬性就是method,最後我們在看看_lookup_req_object怎麼獲取到的ctx.request,以下是原始碼摘抄:

def _lookup_req_object(name):
    #以name=request為列
    top = _request_ctx_stack.top
    # top是就是RequestContext(ctx)物件,裡面含有request、session 等
    if top is None:
        raise RuntimeError(_request_ctx_err_msg)
    return getattr(top, name) #到RequestContext(ctx)中獲取那麼為request的值

在原始碼中很簡單無非就是利用_request_ctx_stack(也就是LocalStack物件)的top屬性返回stack中的ctx,在通過反射獲取request,最後返回ctx.request。以上是整個flask的上下文核心機制,與其相似的全域性物件有如下(session、g):
# context locals
_request_ctx_stack = LocalStack()  #LocalStack()包含pop、push方法以及Local物件,上下文通過該物件push和pop
_app_ctx_stack = LocalStack()
current_app = LocalProxy(_find_app)
request = LocalProxy(partial(_lookup_req_object, 'request’))  #reuqest是LocalProxy的物件,設定和獲取request物件中的屬性通過LocalProxy定義的各種雙下劃線實現
session = LocalProxy(partial(_lookup_req_object, 'session'))
g = LocalProxy(partial(_lookup_app_object, 'g'))  

技巧應用

利用flask的上下文處理機制我們獲取上請求資訊還可以使用如下方式:

from flask import Flask,_request_ctx_stack

app=Flask(__name__)

@app.route("/")
def hello():
    print(_request_ctx_stack.top.request.method) #結果GET,等價於request.method
    return ’this is wd'

if __name__=='__main__':
    app.run()