1. 程式人生 > 實用技巧 >Python 多程序踩過的一些坑

Python 多程序踩過的一些坑

spawn fork 和 forkserver

fork 最快但是fork處理檔案的時候有一些問題,如果需要利用mp.Manager().Queue()交換資料的情況,可能會發生死鎖,死鎖機制未找到To Do.
利用 spawn 和 mp.Queue() 結合的時候,mp.Queue()有一點bug,比如最後一個數據讀不出來

Note When an object is put on a queue, the object is pickled and a background thread later flushes the pickled data to an underlying pipe. This has some consequences which are a little surprising, but should not cause any practical difficulties – if they really bother you then you can instead use a queue created with a manager. After putting an object on an empty queue there may be an infinitesimal delay before the queue’s empty() method returns False and get_nowait() can return without raising Queue.Empty. If multiple processes are enqueuing objects, it is possible for the objects to be received at the other end out-of-order. However, objects enqueued by the same process will always be in the expected order with respect to each other.

解決方案

在我的實踐中 使用 spawn 和 mp.Manager().Queue()結合會是一個比較好的解決方案,但是執行速度較慢