Are locks unnecessary in multi-threaded Python code because of the GIL?

https://stackoverflow.com/questions/105095/are-locks-unnecessary-in-multi-threaded-python-code-because-of-the-gil

由於GIL的存在， 導致所有 python的多執行緒 只能是 併發 ， 並不是並行。

但是這樣的情況下， 共享變數，同一時間只能被一個執行緒訪問，那麼為什麼 多執行緒 模組還設計 lock？

If you are relying on an implementation of Python that has a Global Interpreter Lock (i.e. CPython) and writing multithreaded code, do you really need locks at all?

If the GIL doesn't allow multiple instructions to be executed in parallel, wouldn't shared data be unnecessary to protect?

sorry if this is a dumb question, but it is something I have always wondered about Python on multi-processor/core machines.

same thing would apply to any other language implementation that has a GIL.

answer -- data inconsistent

對於 程序間的 共享變數， 多個執行緒訪問， 沒有物理上的衝突。

但是對於 業務上的 邏輯， 會導致 業務上的衝突， 讀取後， 然後在更新， 兩個程序交替執行， 會導致資料不一致性。

You will still need locks if you share state between threads. The GIL only protects the interpreter internally. You can still have inconsistent updates in your own code.

Here, your code can be interrupted between reading the shared state (balance = shared_balance

) and writing the changed result back (shared_balance = balance), causing a lost update. The result is a random value for the shared state.

To make the updates consistent, run methods would need to lock the shared state around the read-modify-write sections (inside the loops) or have some way to detect when the shared state had changed since it was read.

#!/usr/bin/env python
import threading

shared_balance = 0

class Deposit(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            global shared_balance
            balance = shared_balance
            balance += 100
            shared_balance = balance

class Withdraw(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            global shared_balance
            balance = shared_balance
            balance -= 100
            shared_balance = balance

threads = [Deposit(), Withdraw()]

for thread in threads:
    thread.start()

for thread in threads:
    thread.join()

print shared_balance

solution 1 - thread lock

https://docs.python.org/3.6/library/threading.html#lock-objects

用法

https://haicoder.net/python/python-thread-lock.html

import threading
num = 0
# 建立互斥鎖
lock = threading.Lock()
def handler_incry():
    global num
    for i in range(100000):
        num += 1
    print("handler_incry done, num =", num)
def handler_decry():
    global num
    for i in range(100000):
        num -= 1
    print("handler_decry done, num =", num)
if __name__ == '__main__':
    print("嗨客網(www.haicoder.net)")
    # 建立執行緒
    t1 = threading.Thread(target=handler_incry)
    t2 = threading.Thread(target=handler_decry)
    # 啟動執行緒
    t1.start()
    t2.start()
    t1.join()
    t2.join()

改造

https://github.com/fanqingsong/code_snippet/blob/master/python/multithread/threadlock_plock.py

#!/usr/bin/env python
import threading
from multiprocessing import Process, Lock

shared_balance = 0

lock = Lock()

class Deposit(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            lock.acquire()
            global shared_balance
            balance = shared_balance
            balance += 100
            shared_balance = balance
            lock.release()

class Withdraw(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            lock.acquire()
            global shared_balance
            balance = shared_balance
            balance -= 100
            shared_balance = balance
            lock.release()

threads = [Deposit(), Withdraw()]

for thread in threads:
    thread.start()

for thread in threads:
    thread.join()

print shared_balance

solution2 -- process lock

https://docs.python.org/3.6/library/multiprocessing.html#multiprocessing.Lock

A non-recursive lock object: a close analog of threading.Lock. Once a process or thread has acquired a lock, subsequent attempts to acquire it from any process or thread will block until it is released; any process or thread may release it. The concepts and behaviors of threading.Lock as it applies to threads are replicated here in multiprocessing.Lock as it applies to either processes or threads, except as noted.

改造

https://github.com/fanqingsong/code_snippet/blob/master/python/multithread/threadlock_plock.py

使用 程序鎖 也可以 實現 關鍵程式碼互斥。

程序級別的互斥， 可以相容 執行緒級的互斥。

#!/usr/bin/env python
import threading
from multiprocessing import Process, Lock

shared_balance = 0

lock = Lock()

class Deposit(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            lock.acquire()
            global shared_balance
            balance = shared_balance
            balance += 100
            shared_balance = balance
            lock.release()

class Withdraw(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            lock.acquire()
            global shared_balance
            balance = shared_balance
            balance -= 100
            shared_balance = balance
            lock.release()

threads = [Deposit(), Withdraw()]

for thread in threads:
    thread.start()

for thread in threads:
    thread.join()

print shared_balance

Shared memory + proccess lock == cover thread and process data sharing

Shared memory

https://docs.python.org/3.6/library/multiprocessing.html#sharing-state-between-processes

Data can be stored in a shared memory map using Value or Array. For example, the following code

from multiprocessing import Process, Value, Array

def f(n, a):
    n.value = 3.1415927
    for i in range(len(a)):
        a[i] = -a[i]

if __name__ == '__main__':
    num = Value('d', 0.0)
    arr = Array('i', range(10))

    p = Process(target=f, args=(num, arr))
    p.start()
    p.join()

    print(num.value)
    print(arr[:])

Integration of Both

https://docs.python.org/3.6/library/multiprocessing.html#sharing-state-between-processes

#!/usr/bin/env python
import threading
from multiprocessing import Process, Value, Array
from multiprocessing import Process, Lock

shared_balance = Value('i', 0)
lock = Lock()

class Deposit(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            lock.acquire()
            global shared_balance
            balance = shared_balance.value
            balance += 100
            shared_balance.value = balance
            lock.release()

class Withdraw(threading.Thread):
    def run(self):
        for _ in xrange(1000000):
            lock.acquire()
            global shared_balance
            balance = shared_balance.value
            balance -= 100
            shared_balance.value = balance
            lock.release()

threads = [Deposit(), Withdraw()]



def deposit_func(shared_balance):
    for _ in xrange(100):
        lock.acquire()
        balance = shared_balance.value
        balance += 100
        shared_balance.value = balance
        lock.release()

def withdraw_func(shared_balance):
    for _ in xrange(100):
        lock.acquire()
        balance = shared_balance.value
        balance -= 100
        shared_balance.value = balance
        lock.release()


p_deposit = Process(target=deposit_func, args=(shared_balance,))

p_withdraw = Process(target=withdraw_func, args=(shared_balance,))


for thread in threads:
    thread.start()

for thread in threads:
    thread.join()


p_deposit.start()
p_withdraw.start()

p_deposit.join()
p_withdraw.join()


print shared_balance.value