多執行緒與多程序又來啦啦啦
阿新 • • 發佈:2018-12-10
嘿哈~所有一切的第一,感謝谷溪老師的指導。記得一開始寫多程序也是谷老師提(xi)議(nao)的。今天由業餘級別程式設計高(cai)手(niao)來給大夥講講多執行緒,多程序這類程式的應用吧。
其實是再一次遇到了一個現實問題,實時行情API調取測速。一上手我寫了個這個,
#from multiprocessing import Pool from multiprocessing.dummy import Pool import rqdatac from aoaoao import * import datetime w.start() rqdatac.init() rq_stk_list_300 = rqdatac.index_components('000300.XSHG') #around 300 rq_stk_list_500 = rqdatac.index_components('000905.XSHG') #around 500 rq_stk_list_50 = rqdatac.index_components('000016.XSHG') #around 50 rq_stk_list_future = ['IC201812','IF201812','IH201812'] rq_stk_list = rq_stk_list_300 + rq_stk_list_500 + rq_stk_list_50 + rq_stk_list_future wd_stk_list_01 = tesww("sectorconstituent","date=2018-09-19;windcode=000300.SH").Data wd_stk_list_02 = tesww("sectorconstituent","date=2018-09-19;windcode=000016.SH").Data wd_stk_list_03 = tesww("sectorconstituent","date=2018-09-19;windcode=000905.SH").Data wd_stk_list_future = ['IC201812.CFE','IF201812.CFE','IH201812.CFE'] wd_stk_list = wd_stk_list_01 + wd_stk_list_02 + wd_stk_list_03 + wd_stk_list_future def wind_time_stamp(): data_start=datetime.datetime.now() data=qsww(wd_stk_list, "rt_date,rt_time,rt_last") data_end=datetime.datetime.now() print('wind',data_start,data_end) def rq_time_stamp(): data_start=datetime.datetime.now() data=rqdatac.current_snapshot(rq_stk_list) data_end=datetime.datetime.now() print('rq',data_start,data_end) def fake_function(func): #假函式為了map分別調 func() if __name__ == "__main__": func_list=[wind_time_stamp,rq_time_stamp] with Pool(2) as p: p.map(fake_function, func_list)
其實最主要的精華是第一行和第二行,谷老師指出這個multiprocessing的一個妙哉,就是multiprocess 和 multiprocessing.dummy的切換。
你可以先多程序multiprocessing走一遍,再註釋掉,用dummy走一遍。其他程式碼啥都不用改。以此比較到底誰快。
接下來是threading的登場,這貨請注意尾部。
import threading import rqdatac from aoaoao import * import datetime w.start() rqdatac.init() rq_stk_list_300 = rqdatac.index_components('000300.XSHG') #around 300 rq_stk_list_500 = rqdatac.index_components('000905.XSHG') #around 500 rq_stk_list_50 = rqdatac.index_components('000016.XSHG') #around 50 rq_stk_list_future = ['IC201812','IF201812','IH201812'] rq_stk_list = rq_stk_list_300 + rq_stk_list_500 + rq_stk_list_50 + rq_stk_list_future wd_stk_list_01 = tesw("sectorconstituent","date=2018-09-19;windcode=000300.SH").Data wd_stk_list_02 = tesw("sectorconstituent","date=2018-09-19;windcode=000016.SH").Data wd_stk_list_03 = tesw("sectorconstituent","date=2018-09-19;windcode=000905.SH").Data wd_stk_list_future = ['IC201812.CFE','IF201812.CFE','IH201812.CFE'] wd_stk_list = wd_stk_list_01 + wd_stk_list_02 + wd_stk_list_03 + wd_stk_list_future def wind_time_stamp(): data_start=datetime.datetime.now() data=qsww(wd_stk_list, "rt_date,rt_time,rt_last") data_end=datetime.datetime.now() print('wind',data_start,data_end) def rq_time_stamp(): data_start=datetime.datetime.now() data=rqdatac.current_snapshot(rq_stk_list) data_end=datetime.datetime.now() print('rq',data_start,data_end) threads = [] t1 = threading.Thread(target=wind_time_stamp) threads.append(t1) t2 = threading.Thread(target=rq_time_stamp) threads.append(t2) if __name__ == "__main__": for t in threads: #t.setDaemon(True) t.start() #t.join() t.join()
t.start()形同到了這兩個function的開頭,它們先很快的被開啟,放到同一起跑線。最後一個t.join()形同一起開跑但是等待對方結束才退出。
據說還有fork思路fork出來一塊記憶體一起跑,恕臣妾做不到,畢竟windows...不倒騰。