伺服器caffe安裝配置檔案
阿新 • • 發佈：2019-01-03
本篇部落格記錄我在伺服器上第一次搭建caffe環境所遇到得一些問題以及解決辦法。
伺服器環境配置：CUDA Version 9.0.103
python2.7
opencv-python 3.4.3.18
1. 在安裝caffe之前，伺服器已經將相關依賴庫都編譯安裝好了，因此，免去很多事情，直接在我個人主目錄下下載原始碼：
git clone https://github.com/BVLC/caffe.git
2. 配置Makefile.config
cp Makefile.config.example Makefile.config vi Makefile.config
配置檔案如下
Makefile.config
## Refer to http://caffe.berkeleyvision.org/installation.html # Contributions simplifying and improving our build system are welcome! # cuDNN acceleration switch (uncomment to build with cuDNN). USE_CUDNN := 1 # CPU-only switch (uncomment to build without GPU support). # CPU_ONLY := 1 # uncomment to disable IO dependencies and corresponding data layers USE_OPENCV := 1 # USE_LEVELDB := 0 USE_LMDB := 1 # This code is taken from https://github.com/sh1r0/caffe-android-lib #USE_HDF5 := 0 # uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary) # You should not set this flag if you will be reading LMDBs with any # possibility of simultaneous read and write # ALLOW_LMDB_NOLOCK := 1 # Uncomment if you're using OpenCV 3 OPENCV_VERSION := 3 # To customize your choice of compiler, uncomment and set the following. # N.B. the default for Linux is g++ and the default for OSX is clang++ # CUSTOM_CXX := g++ # CUDA directory contains bin/ and lib/ directories that we need. CUDA_DIR := /usr/local/cuda # On Ubuntu 14.04, if cuda tools are installed via # "sudo apt-get install nvidia-cuda-toolkit" then use this instead: # CUDA_DIR := /usr # CUDA architecture setting: going with all of them. # For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility. # For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility. # For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility. CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \ #此處去掉了前兩行，因為cuda9.0不支援 -gencode arch=compute_35,code=sm_35 \ -gencode arch=compute_50,code=sm_50 \ -gencode arch=compute_52,code=sm_52 \ -gencode arch=compute_60,code=sm_60 \ -gencode arch=compute_61,code=sm_61 \ -gencode arch=compute_61,code=compute_61 # BLAS choice: # atlas for ATLAS (default) # mkl for MKL # open for OpenBlas BLAS := atlas # Custom (MKL/ATLAS/OpenBLAS) include and lib directories. # Leave commented to accept the defaults for your choice of BLAS # (which should work)! # BLAS_INCLUDE := /path/to/your/blas # BLAS_LIB := /path/to/your/blas # Homebrew puts openblas in a directory that is not on the standard search path # BLAS_INCLUDE := $(shell brew --prefix openblas)/include # BLAS_LIB := $(shell brew --prefix openblas)/lib # This is required only if you will compile the matlab interface. # MATLAB directory should contain the mex binary in /bin. # MATLAB_DIR := /usr/local # MATLAB_DIR := /Applications/MATLAB_R2012b.app # NOTE: this is required only if you will compile the python interface. # We need to be able to find Python.h and numpy/arrayobject.h. PYTHON_INCLUDE := /usr/include/python2.7 \ /usr/lib/python2.7/dist-packages/numpy/core/include # Anaconda Python distribution is quite popular. Include path: # Verify anaconda location, sometimes it's in root. #ANACONDA_HOME := /AI/Software/anaconda2 #PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ $(ANACONDA_HOME)/include/python2.7 \ $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include # Uncomment to use Python 3 (default is Python 2) # PYTHON_LIBRARIES := boost_python3 python3.5m # PYTHON_INCLUDE := /usr/include/python3.5m \ # /usr/lib/python3.5/dist-packages/numpy/core/include # We need to be able to find libpythonX.X.so or .dylib. PYTHON_LIB := /usr/lib # PYTHON_LIB := $(ANACONDA_HOME)/lib # Homebrew installs numpy in a non standard path (keg only) # PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include # PYTHON_LIB += $(shell brew --prefix numpy)/lib # Uncomment to support layers written in Python (will link against Python libs) WITH_PYTHON_LAYER := 1 # Whatever else you find you need goes here. INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/ LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/ # If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies # INCLUDE_DIRS += $(shell brew --prefix)/include # LIBRARY_DIRS += $(shell brew --prefix)/lib # NCCL acceleration switch (uncomment to build with NCCL) # https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0) # USE_NCCL := 1 # Uncomment to use `pkg-config` to specify OpenCV library paths. # (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.) # USE_PKG_CONFIG := 1 # N.B. both build and distribute dirs are cleared on `make clean` BUILD_DIR := build DISTRIBUTE_DIR := distribute # Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171 # DEBUG := 1 # The ID of the GPU that 'make runtest' will use to run unit tests. TEST_GPUID := 0 # enable pretty build (comment to see full commands) Q ?= @ LINKFLAGS := -Wl,-rpath,/AI/Software/anaconda2/lib #建立連結，指定一些依賴庫為anaconda目錄下
3. 配置Makefile檔案
vi Makefile
將這兩行修改為指定版本的protoc
$(Q)protoc --proto_path=$(PROTO_SRC_DIR) --cpp_out=$(PROTO_BUILD_DIR) $<
$(Q)protoc --proto_path=$(PROTO_SRC_DIR) --python_out=$(PY_PROTO_BUILD_DIR) $<
修改為
$(Q)/usr/bin/protoc --proto_path=$(PROTO_SRC_DIR) --cpp_out=$(PROTO_BUILD_DIR) $< $(Q)/usr/bin/protoc --proto_path=$(PROTO_SRC_DIR) --python_out=$(PY_PROTO_BUILD_DIR) $<
這一行
NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)
替換為（我沒有修改，但是看很多部落格上面都有修改）
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS
4. 編譯執行
make clean
make all -j32 #-j32為執行緒並行引數，根據自己的電腦修改，-j4,-j8,-j16等等
進行測試
make runtest -j32
若出現的結果如下則表示測試通過
編譯pycaffe
make pycaffe -j32
測試caffe是否安裝成功
cd python
python
import caffe
print dir(caffe)
出現如下結果則表明安裝成功
5. 配置環境變數，以便在任何目錄下python均可以呼叫caffe
vi ~/.bashrc
# 加入這一行export PYTHONPATH=~/caffe/python:$PYTHONPATH
source ~/.bashrc
問題總結：
- build_release/lib/libcaffe.so：對‘cv::imdecode(cv::_InputArray const&, int)’未定義的引用
修改Makefile:
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial matio opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs - /XX/anaconda2/lib/libpng16.so.16：對‘[email protected]_1.2.9’未定義的引用
在Makefile.config下加入這一行LINKFLAGS := -Wl,-rpath,/AI/Software/anaconda2/lib - “fatal error: hdf5.h: 沒有那個檔案或目錄”解決方法
修改INCLUDE_DIRS和LIBRARY_DIRS為如下
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/
修改Makefile將如下第一行程式碼修改為第二行
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5
LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial
- F1129 14:29:10.120764 48271 syncedmem.hpp:22] Check failed: error == cudaSuccess (46 vs. 0) all CUDA-capable devices are busy or unavailable
Makefile:542: recipe for target ‘runtest’ failedMakefile:542: recipe for target ‘runtest’ failed
在跑測試程式時報錯，原因是沒有空閒的顯示卡可供使用，此時檢查是否有空閒下的顯示卡。 - 在編譯時顯示protobuf版本不對，caffe需要的版本為2.6，而預設路徑指定版本為anaconda3.0以上的版本
參照知乎上面的解答，可以在Makefile中指定路徑，首先自行安裝一個2.6版本的protobuf,然後將Makefile中第649和654行
$(Q)protoc --proto_path=$(PROTO_SRC_DIR) --cpp_out=$(PROTO_BUILD_DIR) $<
$(Q)protoc --proto_path=$(PROTO_SRC_DIR) --python_out=$(PY_PROTO_BUILD_DIR) $<
修改為
$(Q)/usr/bin/protoc --proto_path=$(PROTO_SRC_DIR) --cpp_out=$(PROTO_BUILD_DIR) $<
$(Q)/usr/bin/protoc --proto_path=$(PROTO_SRC_DIR) --python_out=$(PY_PROTO_BUILD_DIR) $<
參考連結：
https://github.com/BVLC/caffe/issues
https://blog.csdn.net/hhhuua/article/details/80436160
https://blog.csdn.net/DonatelloBZero/article/details/51304162
https://blog.csdn.net/m0_37407756/article/details/70789271