1. 程式人生 > >Python requests“Max retries exceeded with url” error

Python requests“Max retries exceeded with url” error

今天寫python網路爬蟲的時候遇到一個問題,報錯的具體內容如下:

HTTPConnectionPool(host='dds.cr.usgs.gov', port=80): Max retries exceeded with url: /ltaauth//sno18/ops/l1/2016/138/037/LC81380372016038LGN00.tar.gz?id=stfb9e0bgrpmc4j9lcg45ikrj1&iid=LC81380372016038LGN00&did=227966479&ver=production (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x105b9d210
>: Failed to establish a new connection: [Errno 65] No route to host',))

多方查閱後發現瞭解決問題的原因:http連線太多沒有關閉導致的

解決辦法:

1、增加重試連線次數

  requests.adapters.DEFAULT_RETRIES = 5

2、關閉多餘的連線

requests使用了urllib3庫,預設的http connection是keep-alive的,requests設定False關閉。

操作方法

s = requests.session()
  s.keep
_alive = False