1. 程式人生 > >python3中線程池

python3中線程池

pen stderr blog 成了 pool hcl 換上 err urn

1.在使用多線程處理任務時也不是線程越多越好,由於在切換線程的時候,需要切換上下文環境,依然會造成cpu的大量開銷。為解決這個問題,線程池的概念被提出來了。預先創建好一個較為優化的數量的線程,讓過來的任務立刻能夠使用,就形成了線程池。在python中,沒有內置的較好的線程池模塊,需要自己實現或使用第三方模塊。下面是一個簡單的線程池:

import threading,time,os,queue

class ThreadPool(object):
    def __init__(self,maxsize):
        self.maxsize = maxsize
        self._q = queue.Queue(self.maxsize)
        for i in range(self.maxsize):
            self._q.put(threading.Thread)

    def getThread(self):
        return self._q.get()

    def addThread(self):
        self._q.put(threading.Thread)

def fun(num,p):
    print(‘this is thread [%s]‘%num)
    time.sleep(1)
    p.addThread()


if __name__ == ‘__main__‘:
    pool = ThreadPool(2)
    for i in range(103):
        t = pool.getThread()
        a = t(target = fun,args = (i,pool))
        a.start()

2.利用線程池和paramiko實現對遠程服務器的訪問獲取到相關信息:(自己寫的例子,比較low)

import paramiko,threading
import queue

class ThreadPool(object):
    def __init__(self,maxsize):
        self.maxsize = maxsize
        self._q = queue.Queue(self.maxsize)
        for i in range(self.maxsize):
            self._q.put(threading.Thread)

    def getThread(self):
        return self._q.get()

    def addThread(self):
        self._q.put(threading.Thread)

def ssh_fun(ip,user,password,pool):
    try:
        ssh = paramiko.SSHClient()
        ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        ssh.connect(ip, 22, user, password)
        stdin, stdout, stderr = ssh.exec_command(‘hostname‘)
        info = stdout.read().decode().strip()
        print(‘IP:%s  hostname:%s‘%(ip,info))
        ssh.close()
    except Exception:
        print(‘sorry I can`t connect this server [%s]‘%ip)
    pool.addThread()

if __name__ == ‘__main__‘:
    t_list = []
    pool = ThreadPool(2)
    with open(‘aaa‘,‘r+‘,encoding=‘utf-8‘) as f:
        for line in f:
            split = line.split()
            ip,user,password = split[0],split[1],split[2]
            th = pool.getThread()
            t = th(target=ssh_fun,args=(ip,user,password,pool))
            t.start()
            t_list.append(t)
    for i in t_list:
        i.join()

在這裏我為了測試線程池中只有兩個線程,並且我這個是讀取aaa文件的,這個文件中包含用戶名和密碼等相關信息,樣子如下(其實可以把這些放進數據庫中,使用python從數據庫中進行讀取):

192.168.32.167  root    111111
192.168.32.110  root    111111
192.168.32.120  root    111111
192.168.32.150  root    111111

而最後執行的效果如下:

IP:192.168.32.167 hostname:ns.root
sorry I can`t connect this server [192.168.32.110]
IP:192.168.32.150 hostname:localhost.localdomain
sorry I can`t connect this server [192.168.32.120]

python3中線程池