1. 程式人生 > >Python3.7 Scrapy crawl 執行出錯解決方法

Python3.7 Scrapy crawl 執行出錯解決方法

筆記本安裝的是Python3.7,裝上依賴包和scrapy後執行爬蟲命令出錯

from scrapy import cmdline

cmdline.execute('scrapy crawl xxx'.split(' '))

主要錯誤如下:

 File "D:\Python37\lib\site-packages\scrapy\extensions\telnet.py", line 12, in <module>
    from twisted.conch import manhole, telnet
  File "D:\Python37\lib\site-packages\twisted\conch\manhole.py"
, line 154 def write(self, data, async=False): ^ SyntaxError: invalid syntax Process finished with exit code 1

解決方法:

    def write(self, data, shark=False):
        self.handler.addOutput(data, shark)

    def addOutput(self, data, shark=False):
        if shark:
            self.terminal.eraseLine()
            self.terminal.cursorBackward(len(self.lineBuffer) + len(self.ps[self.pn]))

        self.terminal.write(data)

        if
shark: if self._needsNewline(): self.terminal.nextLine() self.terminal.write(self.ps[self.pn])

將原始碼manhole.py中的async引數更改為shark(注意更換全部)
可以直接點選錯誤跳轉
也可以通過檔案路徑查詢
D:\Python37\Lib\site-packages\twisted\conch\manhole.py

改完再執行就OK了