1. 程式人生 > >python並行化實現:利用multiprocessing

python並行化實現:利用multiprocessing

最近有個很繁瑣的需要提前計算資料指標的問題,很慢,於是就想到了並行化,之前沒有用過python進行並行化,搜了一下,受Caspar的譯文譯文啟發,原文在這裡,實現如下:

from multiprocessing import Pool

def compute(params):
    '''
    params:[param1,param2,param3...]
    '''
    # ...
    pass

def get_params(dataset):
    '''
    return the list of params, each param is also a list
    '''
# ... params = [] for batch_num in xrange(num_batch): param = [] param.append(param1) param.append(param2) param.append(param3) params.append(param) return params if __name__ == '__main__': dataset = 'xxxxxx' params = get_params(dataset) pool = Pool() pool.map(compute, params) pool.close() pool.join()

這裡用列表來儲存多個引數。