一、利用pip安裝scrapy-splash庫：

pip install scrapy-splash

二、安裝docker, 安裝好後執行docker（參考docker筆記） https://blog.csdn.net/u011394598/article/details/83060982

三、拉取映象(pull the image)：

docker pull scrapinghub/splash

四、用docker執行scrapinghub/splash：

docker run -p 8050:8050 scrapinghub/splash

五、在爬蟲的settings中配置

1、設定下載中介軟體：

DOWNLOADER_MIDDLEWARES = {
    'scrapy_splash.SplashCookiesMiddleware': 723,
    'scrapy_splash.SplashMiddleware': 725,
    'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware': 810,
}

2.設定爬蟲中介軟體：

SPIDER_MIDDLEWARES = {
    'scrapy_splash.SplashDeduplicateArgsMiddleware': 100,
}
 

使用splash解析，要在配置檔案中設定：

SPLASH_URL = 'http://127.0.0.1:8050/'

4.配置訊息佇列所使用的過濾類：

DUPEFILTER_CLASS = 'scrapy_splash.SplashAwareDupeFilter'

配置訊息佇列需要使用的類：

HTTPCACHE_STORAGE = 'scrapy_splash.SplashAwareFSCacheStorage'