python3 常用爬蟲庫安裝
系統:deepin 15.5
python版本:python3.5
爬蟲開發常用庫的安裝
pip3 install requests selenium lxml beautifulsoup4 pyquery pymysql pymongo redis flask django jupyter
安裝chromedriver以及phantomjs
sudo apt-get install xvfb
sudo apt-get install unzip
chromedriver安裝
chrome版本:62.0.3202.62
chromedriver下載地址:http://chromedriver.storage.googleapis.com/index.html 我下載的是2.34版
(chromedriver與chrome版本對應:http://chromedriver.storage.googleapis.com/2.33/notes.txt )
下載完成後終端進入下載的目錄中,
unzip chromedriver_linux64.zip
chmod +x chromedriver
sudo mv -f chromedriver /usr/local/share/chromedriver
sudo ln -s /usr/local/share/chromedriver /usr/local/bin/chromedriver
sudo ln -s /usr/local/share/chromedriver /usr/bin/chromedriver
安裝完後簡單測試下,
開啟終端輸入python,然後依次輸入:
from selenium import webdriver
driver = webdriver.Chrome()
selenium和chromedriver安裝正確的話,會自行開啟chrome,如下圖所示
再嘗試輸入
driver.get('https://www.python.org')
會自動開啟python官網
phantomjs安裝
官網下載http://phantomjs.org/download.html
tar -xvf phantomjs-2.1.1-linux-x86_64.tar.bz2
sudo mv phantomjs-2.1.1-linux-x86_64 /usr/local/share/phantomjs
sudo ln -s /usr/local/share/phantomjs/bin/phantomjs /usr/local/bin/phantomjs