Scrapy 這個爬網框架真心不錯，但在OSX上安裝總是會出現各種的問題，在這裡就作一個收集彙總。我的系統環境是 OS X El Capitan (10.11.1)

首先要保證 pip , virtualenv 和 setuptools 是最新版本

pip install --upgrade pip virtualenv setuptools

如果你能直接執行以下的安裝指令，只能說明你的人品足夠的好：

pip install Scrapy

從上帝的角度來看，每個人都是有罪的，所以我估計也很難出現這樣人品出眾的人吧。至少，每次我一執行這指令就會彈出各種的錯，以下來羅列一下：

ffi 安裝失敗

出現的跟蹤資訊：

/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/include
 
/ffi -I/usr/include/libffi -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c c/_cffi_backend.c -o build/temp.macosx-10.6-intel-2.7/c/_cffi_backend.o
    c/_cffi_backend.c:13:10: fatal error: 'ffi.h' file not found
    #include <ffi.h>
             ^
    1 error generated.
    error: command '/usr/bin/clang'
 
 failed with exit status 1
    
    ----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools, tokenize;__file__='/private/tmp/pip-build-ARGDvZ/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__
 
).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-BnZEqB-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-ARGDvZ/cffi

解決辦法:

brew install libffi

openssl 安裝失敗

如果出現 openssl 失敗，而你又確認真的安裝了 Openssl 的話，你將會看到這樣的錯誤。

/usr/bin/clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c build/temp.macosx-10.6-intel-2.7/_openssl.c -o build/temp.macosx-10.6-intel-2.7/build/temp.macosx-10.6-intel-2.7/_openssl.o
    build/temp.macosx-10.6-intel-2.7/_openssl.c:408:10: fatal error: 'openssl/aes.h' file not found
    #include <openssl/aes.h>
             ^
    1 error generated.
    error: command '/usr/bin/clang' failed with exit status 1
    
    ----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools, tokenize;__file__='/private/tmp/pip-build-WKTcA7/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-FyT7Fx-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-WKTcA7/cryptography

解決辦法:

env CRYPTOGRAPHY_OSX_NO_LINK_FLAGS=1 LDFLAGS="$(brew --prefix openssl)/lib/libssl.a $(brew --prefix openssl)/lib/libcrypto.a" CFLAGS="-I$(brew --prefix openssl)/include" pip install scrapy

lxml

這是最可怕的一個包了，嘗試過各種的方法和各種的失敗，有興趣的你可以 google 一下 lxml 將會知道有多少人掉到安裝的坑裡面。

building 'lxml.etree' extension
    /usr/bin/clang -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/opt/openssl/include -I/usr/include/libxml2 -I/private/var/folders/pn/9vl_v4ls1_sfy3mr9cb6ln300000gp/T/pip-build-ZjFZ1r/lxml/src/lxml/includes -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.macosx-10.6-intel-2.7/src/lxml/lxml.etree.o -w -flat_namespace
    In file included from src/lxml/lxml.etree.c:239:
    /private/var/folders/pn/9vl_v4ls1_sfy3mr9cb6ln300000gp/T/pip-build-ZjFZ1r/lxml/src/lxml/includes/etree_defs.h:14:10: fatal error: 'libxml/xmlversion.h' file not found
    #include "libxml/xmlversion.h"
             ^
    1 error generated.
    error: command '/usr/bin/clang' failed with exit status 1
    
    ----------------------------------------
Command "/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c "import setuptools, tokenize;__file__='/private/var/folders/pn/9vl_v4ls1_sfy3mr9cb6ln300000gp/T/pip-build-ZjFZ1r/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/pn/9vl_v4ls1_sfy3mr9cb6ln300000gp/T/pip-ayVXMn-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/pn/9vl_v4ls1_sfy3mr9cb6ln300000gp/T/pip-build-ZjFZ1r/lxml

解決方法:

不要嘗試各種網上奇葩的方法，因為基本都不會有用的，真正的原因在 xcode command tools 出現問題，所以只要執行以下指令，撥出 xcode command tools 的安裝介面，下載10來分鐘，重新執行 scrapy 的安裝就好了。

xcode-select --install

當以上的所有的問題都解決後 Scrapy 就安裝完成了。