Process類，Thread類，Pool類，gevent類，ProcessPoolExecutor,ThreadPoolExecutor的用法比較
一 Process類
multiprocessing模塊下的一個類
創建子進程。
有兩種方法
方法一
from multiprocessing import Process
import os
def foo():
print(‘%s from foo‘%os.getpid())
def bar():
print(‘%s from bar‘ % os.getpid())
if __name__ == ‘__main__‘:
p1=Process(target=foo)
p2=Process(target=bar)
p1.start()
p2.start()
p1.join()
p2.join()
print(‘%s over‘%os.getpid())
輸出：
13524 from foo 12848 from bar 12696 over
方法二
from multiprocessing import Process
import os
class Myprocess(Process):
def __init__(self,name):
super().__init__()
self.name=name
def run(self):
print(‘%s from %s‘ % (os.getpid(),self.name))
if __name__ == ‘__main__‘:
p1=Myprocess(‘foo‘)
p2=Myprocess(‘bar‘)
p1.start()
p2.start()
p1.join()
p2.join()
print(‘%s over‘ % os.getpid())
輸出：
15260 from foo 5980 from bar 8844 over
二 Thread類
threading模塊下的類
創建線程
有兩種方法
與Process類一樣。
三 Pool類
Pool類的方法:
p=Pool()
p.apply_async()，異步提交任務，生成ApplyResult對象
p.close()
p.join()
ApplyResult.get()，從ApplyResult對象中獲取返回值。
from multiprocessing import Pool
import os,time
def foo():
time.sleep(1)
print(‘%s from foo‘%os.getpid())
return ‘foo‘
def bar():
time.sleep(2)
print(‘%s from bar‘ % os.getpid())
return ‘bar‘
if __name__ == ‘__main__‘:
p=Pool()
t1=time.time()
res1=p.apply_async(foo)
res2=p.apply_async(bar)
p.close()
p.join()
print(res1)
print(time.time()-t1) ##多出來的0.15秒是開啟進程所花費的時間
t2=time.clock()
print(res1.get())
print(res2.get())
print(time.clock()-t2)
輸出：
12468 from foo 8832 from bar <multiprocessing.pool.ApplyResult object at 0x00000126511CEE48> 2.1609864234924316 foo bar 2.1880321932470032e-05
map()方，比較常用。
官方介紹：
Apply `func` to each element in `iterable`, collecting the results
in a list that is returned.
廢話少說，上代碼
import time
from multiprocessing import Pool
def foo(x):
time.sleep(1)
return x*x
if __name__ == ‘__main__‘:
l = [1,2,3,4,5,6,7,8,9,10]
t1 = time.time()
p = Pool()
print(time.time()-t1)
res = p.map(foo,l) # 這行代碼表示所有的進行都已經執行完了，並且每個進程的結果都拿到，放在了res中
print(time.time()-t1)
print(res,type(res))
p.close()
p.join()
print(time.time()-t1)
輸出：
0.14187192916870117 2.2597498893737793 [1, 4, 9, 16, 25, 36, 49, 64, 81, 100] <class ‘list‘> 2.401075601577759
四 gevent
gevent是一個基於協程的Python網絡庫。
單線程實現並發，協程。
需要用到猴子補丁。
方法：
g1=gevent.spawn(func)：提交任務，生成Greenlet對象--g1。
g1.join(),阻塞，直到g1任務完成。
g1.value。從Greenlet對象g1中獲取返回值。
import gevent
from gevent import monkey;monkey.patch_all()
import os,time
from threading import current_thread
def foo():
time.sleep(1)
print(‘%s %s from foo‘%(os.getpid(),current_thread().getName()))
def bar():
time.sleep(2)
print(‘%s %s from bar‘ % (os.getpid(),current_thread().getName()))
t1=time.time()
g1=gevent.spawn(foo)
g2=gevent.spawn(bar)
print(g1,g2)
g1.join()
g2.join()
print(time.time()-t1)
輸出：
<Greenlet at 0x1e3fc1396d0: foo> <Greenlet at 0x1e3fc139800: bar> 7536 DummyThread-1 from foo 7536 DummyThread-2 from bar 2.0017032623291016 #可以看到協程開啟進程花銷非常小。
五 ProcessPoolExecutor
創建多進程
concurrent.futures庫內的ProcessPoolExecutor類
executor=ProcessPoolExecutor()：生成一個ProcessPoolExecutor對象；
future=executor.submit():提交任務，返回一個Future對象。
executor.shutdown()。相當於Pool類中的close()和join()
future.result()：從Future對象中獲取其返回值。
from concurrent.futures import ProcessPoolExecutor,ThreadPoolExecutor
import os,time
from threading import current_thread
def foo():
time.sleep(1)
print(‘%s %s from foo‘%(os.getpid(),current_thread().getName()))
return ‘foo‘
def bar():
time.sleep(2)
print(‘%s %s from bar‘ % (os.getpid(),current_thread().getName()))
return ‘bar‘
if __name__ == ‘__main__‘:
t1=time.time()
executor=ProcessPoolExecutor()
print(‘executor‘,executor)
future1=executor.submit(foo)
future2=executor.submit(bar)
print(future1,future2)
executor.shutdown()
print(future1.result())
print(future2.result())
print(time.time()-t1)
輸出：
6480 MainThread from foo 6756 MainThread from bar foo bar 2.7526917457580566 #可以看到創建進程還是比較花費時間的
六 ThreadPoolExecutor()
創建多線程
concurrent.futures庫內的ProcessPoolExecutor類
executor=ThreadPoolExecutor()：生成一個ThreadPoolExecutor對象；
future=executor.submit():提交任務，返回一個Future對象。
executor.shutdown()。相當於Pool類中的close()和join()。
future.result()：從Future對象中獲取其返回值。
from concurrent.futures import ThreadPoolExecutor
import os,time
from threading import current_thread
def foo():
time.sleep(1)
print(‘%s %s from foo‘%(os.getpid(),current_thread().getName()))
return ‘foo‘
def bar():
time.sleep(2)
print(‘%s %s from bar‘ % (os.getpid(),current_thread().getName()))
return ‘bar‘
if __name__ == ‘__main__‘:
t1=time.time()
executor=ThreadPoolExecutor()
print(‘executor‘,executor)
future1=executor.submit(foo)
future2=executor.submit(bar)
print(future1,future2)
executor.shutdown()
print(future1.result())
print(future2.result())
print(time.time()-t1)
輸出：
executor <concurrent.futures.thread.ThreadPoolExecutor object at 0x0000020073EAAAC8> <Future at 0x20073eb4198 state=running> <Future at 0x20074176940 state=running> 4380 <concurrent.futures.thread.ThreadPoolExecutor object at 0x0000020073EAAAC8>_0 from foo #可以看到進程號是一樣的 4380 <concurrent.futures.thread.ThreadPoolExecutor object at 0x0000020073EAAAC8>_1 from bar foo bar 2.001234531402588 #相比較開啟進程，線程開啟的時間非常快，花銷非常小
Process類，Thread類，Pool類，gevent類，ProcessPoolExecutor,ThreadPoolExecutor的用法比較