1. 程式人生 > >Python MongoDB 插入資料,已存在則不執行,不存在則插入

Python MongoDB 插入資料,已存在則不執行,不存在則插入

想把QQ日誌爬蟲(Python)爬下來的日誌儲存到 MongoDB 裡面。 
但 insert 的時候報錯:

<code class="hljs mel has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">E11000 <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">duplicate</span> key <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">error</span> collection: QQ.Blog index: _id_ dup key: { : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"965464518_1301232446"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

後來知道錯誤的原因是:插入的資料和已有資料的 ID 重複了。

我想要的是:插入一篇日誌,如果該日誌(ID)已存在,則不執行(也不更新);如果不存在則插入。 
百度中大部分的答案都是用更新,但如果 ID 存在的話我是想直接不執行的,沒有更新已有資料的必要。

解決方案:

update裡有個引數 ‘$setOnInsert’ 可以實現”存在則不執行”的功能,可見 $setOnInsert 官方文件

示例:

起始資料:

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.insert</span>({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span>})
> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.find</span>()
{ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li></ul>


加入相同 ID 的日誌:

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.update</span>({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>}, {$setOnInsert:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span>}}, {upsert:true})
WriteResult({ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nMatched"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nUpserted"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nModified"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> })
>
> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.find</span>()
{ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li></ul>


加入不同 ID 的日誌:

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.update</span>({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123"</span>}, {$setOnInsert:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span>}}, {upsert:true})
WriteResult({ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nMatched"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nUpserted"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nModified"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123"</span> })
>
> db<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.Blog</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.find</span>()
{ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《My Test》"</span>
{ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span> }</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>


如果某些內容想更新的話(例如更新 title )可以用 ‘$set’:

<code class="hljs d has-numbering" style="display: block; padding: 0px; background: transparent; color: inherit; box-sizing: border-box; font-family: "Source Code Pro", monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal;">> db.Blog.update({<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>}, {$setOnInsert:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abc123"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"other"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"hello world!"</span>}, $set:{<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span>:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《New Title》"</span>}}, {upsert:<span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>})
WriteResult({ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nMatched"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nUpserted"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"nModified"</span> : <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> })
>
> db.Blog.find()
{ <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"_id"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"123456"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"blog_cont"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"abcdef"</span>, <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"title"</span> : <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"《New Title》
{ "</span>_id<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" : "</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">123</span><span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">", "</span>blog_cont<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" : "</span>abc123<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">", "</span>othe<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">r" : "</span>hello world!<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">" }</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; background-color: rgb(238, 238, 238); top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right: 1px solid rgb(221, 221, 221); list-style: none; text-align: right;"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>


希望對你有所幫助!