1. 程式人生 > >python scrapy 儲存資料到mysql的坑(1064錯誤)

python scrapy 儲存資料到mysql的坑(1064錯誤)

在將scrapy中的資料儲存到mysql中,出現部分資料插入不進去的問題, 一直報:
在這裡插入圖片描述

You have an error in your SQL syntax;

mysql相對於mongo來說,對資料要嚴格些,有時候,很多SQL語句寫對了,還是插不進去資料。針對上面的問題,我使用了一個pymysql中的方法file_count = pymysql.escape_string(item['file_content']),這樣就成功的把資料插入到MySQL中了。

下面貼一下全部的程式碼:

class MysqlPipeline(object):

    def __init__
(self): self.host = settings.DB_HOST self.port = settings.DB_PORT self.user = settings.DB_USER self.pwd = settings.DB_PWD self.name = settings.DB_NAME self.charset = settings.DB_CHARSET self.connect() def connect(self): self.conn =
pymysql.connect(host=self.host, port=self.port, user=self.user, password=self.pwd, db=self.name, charset=self.charset) self.
cursor = self.conn.cursor() def close_spider(self, spider): self.conn.close() self.cursor.close() def process_item(self, item, spider): try: print('在插入 *************************************** ') # sql = 'insert into zb_contest_data(craw_date, cd_filename, cd_content, create_time) values("%s", "%s", "%s", "%s")' % ( # item['craw_date'], item['file_name'], str(item['file_content']), item['create_time']) file_count = pymysql.escape_string(item['file_content']) sql = 'insert into zb_contest_data(craw_date, cd_filename, cd_content, create_time) values("%s", "%s", "%s", "%s")' % ( item['craw_date'], item['file_name'], file_count, item['create_time']) # sql = 'insert into zb_contest_data(craw_date, cd_filename, cd_content, create_time) values("%s", "%s", "%s", "%s")' % ( # item['craw_date'], item['file_name'], item['file_content'], item['create_time']) # 執行sql語句 self.cursor.execute(sql) self.conn.commit() pass except Exception as e: print(e) pass return item

個人微信:hll643435675(備註:部落格)

在這裡插入圖片描述

更多資源請訪問:

https://blog.csdn.net/xudailong_blog/article/details/78762262

慕課視訊教程:https://blog.csdn.net/xudailong_blog/article/details/82909611