scrapy_AttributeError: type object 'ImagesPipeline' has no attribute 'startswith'錯誤及解決
阿新 • • 發佈:2019-01-08
問題描述:
在使用scrapy進行爬蟲的時候,啟用前沒有問題,啟用管道後就出現了問題。
報錯如下:
AttributeError: type object 'ImagesPipeline' has no attribute 'startswith'
無法在python中的SCRAPY中建立對'str'物件的弱引用
spider檔案程式碼如下:
# -*- coding: utf-8 -*- import scrapy from Bmw.items import BmwItem class BmySpider(scrapy.Spider): name = 'bmy' allowed_domains = ['XXXXXXXXXX'] start_urls = ['https://XXXXXXXXX/pic/series/65.html'] def parse(self, response): uibox_list = response.xpath("//div[@class='uibox']")[1:] for uibox in uibox_list: # 圖片標題 uibox_title = uibox.xpath(".//div[@class='uibox-title']/a/text()").get() # print(uibox_title) # 圖片連結 # uibox_img_list = uibox.xpath(".//div[@class='uibox-con carpic-list03']//img/@src").getall() uibox_img_list = uibox.xpath(".//div[contains(@class,'uibox-con carpic-list03')]//img/@src").getall() # 利用列表推導式對每個元素進行url地址拼接 # uibox_img_list = ["http:" + i for i in uibox_img_list] # map函式對url地址進行拼接 uibox_img_list = list(map(lambda x: response.urljoin(x) if "http" not in x else x, uibox_img_list)) item = BmwItem(uibox_title=uibox_title, image_urls=uibox_img_list) yield item
Pipelines檔案程式碼:
ITEM_PIPELINES = {
Bmw.pipelines.ImagePipeline: 300,
}
原因分析:
啟用前沒有問題,啟用後出了問題,但是在管道內加了print並沒有打印出來,也就是說,程式並沒有進入到管道中執行,先檢視settings中管道配置資訊是否有誤
果然,配置檔案中沒有加引號!!!!!
修改後程式碼:
ITEM_PIPELINES = {
'Bmw.pipelines.ImagePipeline': 300,
}
再次執行,成功執行!!