1. 程式人生 > >使用 flask和redis維護 代理池

使用 flask和redis維護 代理池

https://github.com/germey/proxypool

該模組 使用是 直接 在 ubuntu上部署  安裝 需要 python大於3.5.2使用aiohttp

[email protected]:~# which pip3.6
[email protected]:~# ln -s /usr/local/python3.6/bin/pip3.6 /usr/bin/pip3.6
[email protected]:~# which pip3.6
/usr/bin/pip3.6

上面是建立 安裝好的pip的連結

1.下載

cd mkdir code git init git clone https://github.com/germey/proxypool cd proxypool

2.安裝 pip3.6 install -r requirements.txt

3. python3.6 run.py

[email protected]:/code/proxypool/ProxyPool# python3.6 run.py
Traceback (most recent call last):
File "run.py", line 1, in <module>
from proxypool.api import app
File "/code/proxypool/ProxyPool/proxypool/api.py", line 1, in <module>
from flask import Flask, g
File "/usr/local/python3.6/lib/python3.6/site-packages/flask/__init__.py", line 21, in <module>
from .app import Flask, Request, Response
File "/usr/local/python3.6/lib/python3.6/site-packages/flask/app.py", line 25, in <module>
from . import cli, json
File "/usr/local/python3.6/lib/python3.6/site-packages/flask/cli.py", line 18, in <module>
import ssl
File "/usr/local/python3.6/lib/python3.6/ssl.py", line 101, in <module>
import _ssl # if we can't import it, let the error propagate
ModuleNotFoundError: No module named '_ssl'

[email protected]:/code/proxypool/ProxyPool# pip3.6 install ssl
Collecting ssl
Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/83/21/f469c9923235f8c36d5fd5334ed11e2681abad7e0032c5aba964dcaf9bbb/ssl-1.16.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-build-sf7597gx/ssl/setup.py", line 33
print 'looking for', f
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print('looking for', f)?

----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-sf7597gx/ssl/

簡單點說就是 少了 ssl 但是 直接pip安裝無法實現

1.安裝ssl
sudo apt-get install openssl

sudo apt-get install libssl-dev

2. 修改Moudles/Setup (該目錄在python的解壓目錄下)

vim Modules/Setup
#修改結果如下:
# Socket module helper for socket(2)
_socket socketmodule.c timemodule.c

# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
#SSL=/usr/local/ssl
_ssl _ssl.c \
-DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
-L$(SSL)/lib -lssl -lcrypto


3.重新安裝一次 這裡和上面 一個一致 https://www.cnblogs.com/billhsu2009/p/9958515.html

./configure --prefix=/usr/local/python3.6

make

make install


4.python3

---------------------
參考 https://blog.csdn.net/zr1076311296/article/details/75136612

5 切入 python3.6 import ssl ok

6 重新執行 

需要安裝 fake_useragent 直接pip安裝就ok

File "/usr/local/python3.6/lib/python3.6/site-packages/redis/client.py", line 668, in execute_command
return self.parse_response(connection, command_name, **options)
File "/usr/local/python3.6/lib/python3.6/site-packages/redis/client.py", line 680, in parse_response
response = connection.read_response()
File "/usr/local/python3.6/lib/python3.6/site-packages/redis/connection.py", line 629, in read_response
raise response
redis.exceptions.ResponseError: NOAUTH Authentication required.

https://github.com/WiseDoge/ProxyPool

https://github.com/germey/proxypool