1. 程式人生 > >scrapy_AttributeError: type object 'ImagesPipeline' has no attribute 'startswith'錯誤及解決

scrapy_AttributeError: type object 'ImagesPipeline' has no attribute 'startswith'錯誤及解決

問題描述:

在使用scrapy進行爬蟲的時候,啟用前沒有問題,啟用管道後就出現了問題。

報錯如下:

AttributeError: type object 'ImagesPipeline' has no attribute 'startswith'

無法在python中的SCRAPY中建立對'str'物件的弱引用

spider檔案程式碼如下:

# -*- coding: utf-8 -*-
import scrapy
from Bmw.items import BmwItem


class BmySpider(scrapy.Spider):
    name = 'bmy'
    allowed_domains = ['XXXXXXXXXX']
    start_urls = ['https://XXXXXXXXX/pic/series/65.html']

    def parse(self, response):
        uibox_list = response.xpath("//div[@class='uibox']")[1:]
        for uibox in uibox_list:
            # 圖片標題
            uibox_title = uibox.xpath(".//div[@class='uibox-title']/a/text()").get()
            # print(uibox_title)
            # 圖片連結
            # uibox_img_list = uibox.xpath(".//div[@class='uibox-con carpic-list03']//img/@src").getall()
            uibox_img_list = uibox.xpath(".//div[contains(@class,'uibox-con carpic-list03')]//img/@src").getall()
            # 利用列表推導式對每個元素進行url地址拼接
            # uibox_img_list = ["http:" + i for i in uibox_img_list]
            # map函式對url地址進行拼接
            uibox_img_list = list(map(lambda x: response.urljoin(x) if "http" not in x else x, uibox_img_list))

            item = BmwItem(uibox_title=uibox_title, image_urls=uibox_img_list)

            yield item

Pipelines檔案程式碼:

ITEM_PIPELINES = {
    Bmw.pipelines.ImagePipeline: 300,
}

 原因分析:

啟用前沒有問題,啟用後出了問題,但是在管道內加了print並沒有打印出來,也就是說,程式並沒有進入到管道中執行,先檢視settings中管道配置資訊是否有誤

果然,配置檔案中沒有加引號!!!!!

修改後程式碼:

ITEM_PIPELINES = {
    'Bmw.pipelines.ImagePipeline': 300,
}

再次執行,成功執行!!