ubuntu18.04 配置python tensorflow虛擬環境+pycharm
在ubuntu18.04環境下裝tensorflow的時候,總是會被裝到~/.local下,尋找時候很麻煩,於是決定使用虛擬環境。
pycharm可以直接建立新的虛擬環境,並且可以直接在pycharm裡新增安裝環境需要的庫(文末會提到),不過在pycharm裡操作可控性不好,而且安裝tensorflow往往會出現錯誤,所以這裡還是先不使用pycharm來建立虛擬環境。
ubuntu18.04 預設的python版本是3.6.5
先安裝python3 的自帶的虛擬環境建立工具:
sudo apt-get install python3-venv
然後建立一個名為venv4ML0的虛擬環境(最好到需要建立環境的路徑操作):
python3 -m venv venv4ML0
進入建立的該環境:
source ./venv4ML0/bin/activate
進入之後命令列頭部會有標識:
(venv4ML0) [email protected]:~/MachineLearning$
接下來直接安裝tensorflow,有一些依賴庫在安裝tensorflow的時候會自動安裝,例如numpy1.14.3,基本上自動安裝的都是最新版,所以比自己裝方便多了:
pip3 install tensorflow==1.8.0
不過裝的時候容易出以下的錯誤:
(venv4ML0) [email protected]
:~/MachineLearning$ pip3 install tensorflow==1.8.0 Collecting tensorflow==1.8.0 Using cached https://files.pythonhosted.org/packages/22/c6/d08f7c549330c2acc1b18b5c1f0f8d9d2af92f54d56861f331f372731671/tensorflow-1.8.0-cp36-cp36m-manylinux1_x86_64.whl Collecting tensorboard<1.9.0,>=1.8.0 (from tensorflow==1.8.0) Using cached https://files.pythonhosted.org/packages/59/a6/0ae6092b7542cfedba6b2a1c9b8dceaf278238c39484f3ba03b03f07803c/tensorboard-1.8.0-py3-none-any.whl Collecting astor>=0.6.0 (from tensorflow==1.8.0) Using cached https://files.pythonhosted.org/packages/b2/91/cc9805f1ff7b49f620136b3a7ca26f6a1be2ed424606804b0fbcf499f712/astor-0.6.2-py2.py3-none-any.whl Collecting wheel>=0.26 (from tensorflow==1.8.0) Using cached https://files.pythonhosted.org/packages/81/30/e935244ca6165187ae8be876b6316ae201b71485538ffac1d718843025a9/wheel-0.31.1-py2.py3-none-any.whl Collecting termcolor>=1.1.0 (from tensorflow==1.8.0) Collecting numpy>=1.13.3 (from tensorflow==1.8.0) Using cached https://files.pythonhosted.org/packages/71/90/ca61e203e0080a8cef7ac21eca199829fa8d997f7c4da3e985b49d0a107d/numpy-1.14.3-cp36-cp36m-manylinux1_x86_64.whl Collecting absl-py>=0.1.6 (from tensorflow==1.8.0) Requirement already satisfied: six>=1.10.0 in ./venv4ML0/lib/python3.6/site-packages (from tensorflow==1.8.0) Requirement already satisfied: grpcio>=1.8.6 in ./venv4ML0/lib/python3.6/site-packages (from tensorflow==1.8.0) Collecting gast>=0.2.0 (from tensorflow==1.8.0) Requirement already satisfied: protobuf>=3.4.0 in ./venv4ML0/lib/python3.6/site-packages (from tensorflow==1.8.0) Collecting werkzeug>=0.11.10 (from tensorboard<1.9.0,>=1.8.0->tensorflow==1.8.0) Exception: Traceback (most recent call last): File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 350, in _make_request self._validate_conn(conn) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 837, in _validate_conn conn.connect() File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connection.py", line 323, in connect ssl_context=context) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/ssl_.py", line 324, in ssl_wrap_socket return context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket _context=self, _session=session) File "/usr/lib/python3.6/ssl.py", line 814, in __init__ self.do_handshake() File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake self._sslobj.do_handshake() File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake self._sslobj.do_handshake() socket.timeout: _ssl.c:817: The handshake operation timed out During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 594, in urlopen chunked=chunked) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 353, in _make_request self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 313, in _raise_timeout raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value) requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='pypi.org', port=443): Read timed out. (read timeout=15) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main status = self.run(options, args) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run wb.build(autobuilding=True) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/wheel.py", line 749, in build self.requirement_set.prepare_files(self.finder) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files ignore_dependencies=self.ignore_dependencies)) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file require_hashes File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link self.link = finder.find_requirement(self, upgrade) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement all_candidates = self.find_all_candidates(req.name) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates for page in self._get_pages(url_locations, project_name): File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages page = self._get_page(location) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page return HTMLPage.get_page(link, session=self.session) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/index.py", line 792, in get_page "Cache-Control": "max-age=600", File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 501, in get return self.request('GET', url, **kwargs) File "/home/yangguang/MachineLearning/venv4ML0/lib/python3.6/site-packages/pip/download.py", line 386, in request return super(PipSession, self).request(method, url, *args, **kwargs) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 488, in request resp = self.send(prep, **send_kwargs) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 630, in send history = [resp for resp in gen] if allow_redirects else [] File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 630, in <listcomp> history = [resp for resp in gen] if allow_redirects else [] File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 190, in resolve_redirects **adapter_kwargs File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/sessions.py", line 609, in send r = adapter.send(request, **kwargs) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/CacheControl-0.11.7-py2.py3-none-any.whl/cachecontrol/adapter.py", line 47, in send resp = super(CacheControlAdapter, self).send(request, **kw) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/requests-2.12.4-py2.py3-none-any.whl/requests/adapters.py", line 423, in send timeout=timeout File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 643, in urlopen _stacktrace=sys.exc_info()[2]) File "/home/yangguang/MachineLearning/venv4ML0/share/python-wheels/urllib3-1.19.1-py2.py3-none-any.whl/urllib3/util/retry.py", line 315, in increment total -= 1 TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
個人覺得可能是網路的問題,遇到這種情況就單獨裝出錯處的lib,筆者出現過多次出錯,但是重複安裝就成功了:
pip3 install Werkzeug==0.14.1
然後再一遍:
pip3 install tensorflow==1.8.0
可能還會有其他同樣的問題,同樣處理,直到pip3 install tensorflow==1.8.0執行成功!
這樣tensorflow就裝好了,可以使用hello world測試一下,此處不做說明。
現在開啟pycharm,File->Setting->Project Interpreter->設定->Existing environment->選擇該虛擬環境路徑 bin下的python3即可,匯入虛擬環境後pycharm顯示:
可以直接點選右邊的+進行新增新的庫,也可以一開始就在這裡安裝虛擬環境裡的tensorflow,有興趣可以嘗試,不過我這樣操作的體驗並不好。
測試:
安裝一些其他依賴的模組:
pip3 install matplotlib --upgrade
pip3 install scikit-learn --upgrade
pip3 install scikit-image --upgrade
使用完環境退出虛擬環境:
deactivate
使用這種方法建立虛擬環境不會匯入系統python環境裡pip安裝的包(ubuntu18.04會將pip安裝的包存在~/.local中),預設使用最乾淨的環境。
可以使用pip freeze > requirements.txt
將當前環境中的包的名稱及版本匯出到文字中
然後在另一個環境中 pip install -r requirements.txt
安裝,虛擬環境會自動建立和系統環境裡包的連結,不需要重新下載一遍。
如果從系統環境中匯出,會有很多沒有用的lib,可以在requirement.txt裡刪除,或者自己手動編輯一個requirement.txt檔案。
如果同時需要python2的開發環境,可以再建立一個python2的虛擬環境,這個虛擬環境和python3的虛擬環境完全隔離:
sudo apt-get install python-virtualenv
virtualenv -p /usr/bin/python2 py2env4ML
之後的操作和python3的虛擬環境一樣。