scrapy輸出csv檔案資料多空行問題解決
阿新 • • 發佈:2018-12-15
使用scrapy命令列將資料儲存為csv檔案時,發現csv多空行。
檢視原始碼scrapy.exporters.CsvItemExporter,在io.TextIOWrapper加入引數newline='',問題就解決啦。
- class CsvItemExporter(BaseItemExporter):
- def __init__(self, file, include_headers_line=True, join_multivalued=',', **kwargs):
- self._configure(kwargs, dont_fail=True)
- if not self.encoding:
- self.encoding = 'utf-8'
- self.include_headers_line = include_headers_line
- self.stream = io.TextIOWrapper(
- file,
- newline='',
- line_buffering=False,
- write_through=True,
- encoding=self.encoding
- ) if six.PY3 else file
- self.csv_writer = csv.writer(self.stream, **kwargs)
- self._headers_not_written = True
- ...