Python3之無法在程序池中使用佇列Queue的解決方案
阿新 • • 發佈:2019-02-19
import multiprocessing
import time
class Test:
def __init__(self):
self.pool = multiprocessing.Pool()
# self.queue = multiprocessing.Queue()
m = multiprocessing.Manager()
self.queue = m.Queue()
def subprocess(self):
for i in range(10):
print("Running" )
time.sleep(1)
print("Subprocess Completed")
def start(self):
self.pool.apply_async(func=self.subprocess)
print("Subprocess has been started")
self.pool.close()
self.pool.join()
def __getstate__(self):
self_dict = self.__dict__.copy()
del self_dict['pool']
return self_dict
def __setstate__(self, state):
self.__dict__.update(state)
if __name__ == '__main__':
test = Test()
test.start()
主要是這兩行程式碼:
m = multiprocessing.Manager()
self.queue = m.Queue()
A manager object returned by Manager() controls a server process which holds Python objects and allows other processes to manipulate them using proxies.
A manager returned by Manager() will support types list, dict, Namespace, Lock, RLock, Semaphore, BoundedSemaphore, Condition, Event, Barrier, Queue, Value and Array.
共享資料主要通過在記憶體中共享某個資料型別實現的,共享資料型別比較多:列表,字典,同步鎖,遞迴鎖,訊號量,時間,還有佇列等,這個程序池用的就是共享資料的佇列型別資料
話說__getstate__和__setstate__是什麼鬼?
這個是Python中有些無法被pickle序列化的物件,呼叫這兩個方法後就可以進行pickle序列化和反序列化處理處理。示例:
class Foo(object):
def __init__(self, val=2):
self.val = val
def __getstate__(self):
# print("I'm being pickled")
self.val *= 2
print(self.__dict__)
return self.__dict__
print('=--------->')
def __setstate__(self, d):
# print("I'm being unpickled with these values:", d)
self.__dict__ = d
# self.val *= 3
# pass
print('============>')
import pickle
f = Foo()
f_string = pickle.dumps(f)
f_new = pickle.loads(f_string)
print(f_new.val)
上面的示例可以很好的解釋了__getstate__和__setstate__用法,Python 將只 pickle 當它呼叫該例項的__getstate__() 方法時返回給它的值,在 unpickle 時,Python 將提供經過 pickle 的值作為引數傳遞給例項的__setstate__() 方法。在 __setstate__() 方法內,可以根據經過 pickle 的名稱和位置資訊來重建物件,並將該物件分配給這個例項的 屬性。
這個部落格寫的超級贊:
pickle序列化全接觸