scrapy框架的安裝配置
阿新 • • 發佈:2018-12-26
Scrapy框架安裝配置
1.建立一個虛擬環境,在虛擬環境中進行操作
安裝:
pip install virtualenv
建立:
到對應目錄下:
virtualenv 虛擬環境名稱
進入虛擬環境:
D:\virtu\spiderenv\Scripts>activate
2.安裝twiste依賴,
https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
到該網站下載對應python版本對應位數的twisted,下載後
pip install 路徑\twisted包名
3.安裝pywin32
https://www.lfd.uci.edu/~gohlke/pythonlibs/#pywin32
下載對應python版本的安裝包後安裝:
pip install 路徑\pywin32包名
或者直接安裝
安裝後在虛擬環境中執行python,輸入import win32com
,如果沒有報錯,則說明安裝成功
D:\virtu\spiderenv\Scripts>python Python 3.6.7 (v3.6.7:6ec5cf24b7, Oct 20 2018, 13:35:33) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import win32com
4.安裝lxml
pip install lxml
5.安裝Scrapy
pip install scrapy
6.驗證安裝是否成功,如下情況則為安裝成功
(spiderenv) D:\virtu\spiderenv\Scripts>scrapy Scrapy 1.5.1 - no active project Usage: scrapy <command> [options] [args] Available commands: bench Run quick benchmark test fetch Fetch a URL using the Scrapy downloader genspider Generate new spider using pre-defined templates runspider Run a self-contained spider (without creating a project) settings Get settings values shell Interactive scraping console startproject Create new project version Print Scrapy version view Open URL in browser, as seen by Scrapy [ more ] More commands available when run from project directory Use "scrapy <command> -h" to see more info about a command