1. 程式人生 > >caffe ubuntu安裝全過程 GPU CPU均可參考

caffe ubuntu安裝全過程 GPU CPU均可參考

基本常時命令

sudo uname --m 檢視自己執行ubuntu是32位還是64sudo uname --s 顯示核心名字s 
sudo uname --r 顯示核心版本 
sudo uname --n 顯示網路主機名 
sudo uname --p 顯示cpu  
sudo uname -a 檢視Ubuntu版本號
1)把驅動加入黑名單中: /etc/modprobe.d/blacklist.conf  在後面加入:
blacklist nouveau
options nouveau modeset=0
不要sudo reboot,不然進不來圖形化介面只能命令行了。z
確認下Nouveau是已經被幹掉,使用命令: lsmod | grep nouveau
(3)重啟系統至init 3(文字模式),也可先進入圖形桌面再執行init 3進入文字模式,再安裝下載的驅動就無問題,安裝命令示例
GTX1080顯示卡驅動367.35安裝 gtx1070桌上型電腦測試成功
sudo apt-get update##更新源 
sudo add-apt-repository ppa:graphics-drivers/ppa 
按回車繼續 
sudo apt-get update 
sudo apt-get install nvidia-367 
sudo apt-get install mesa-common-dev 
sudo apt-get install freeglut3-dev 

以上裝完後重啟電腦,此時驅動安裝完畢,系統沒有卡頓現象
GTX1050ti 的顯示卡:
裝驅動前關閉 x server

sudo /etc/init.d/lightdm stop 或sudo service lightdm stop
sudo /etc/init.d/lightdm status 或sudo service lightdm status
sudo apt-get update##更新源 
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt-get update 
sudo apt-get install nvidia-378
sudo apt-get install mesa-common-dev sudo apt-get install freeglut3-dev

安裝後開啟X server

sudo service lightdm start/restart
//如果裝完驅動發現進不了桌面了,可以將裝的驅動刪除(
    sudo dpkg -l|grep nvidia(查詢)
    sudoapt-getremove --purge nvidia*(刪除)
)
顯示機器上的顯示卡資訊:
lspci  | grep -i vga
檢視更詳細的資訊:
lspci -v

開始安裝英偉達驅動(非命令列方式)
如果這種方式安裝顯示卡後進不了圖形化介面的話,試試下面這種方法:

首先,點選system setting,進入下圖所示的選單。 
Ubuntu system setting
點選software & updates 
ubuntu software
把Download from改成Main server,然後點選close。 
這個時候,系統會問你要不要reload,點選reload等待載入完畢即可。
再次進入system setting,點選Details 
system details

因為我已經安裝好了,這個時候Graphics看到的是GeForce GTX 860M/PCIe/SSE2
如果你沒有安裝NVIDIA驅動,這個時候看到的應該是獨顯的資訊。
當然,這個時候你的右下角按鈕顯示的也不是System Up-To-Date。點選更新按鈕,更新一下系統檔案。然後重啟。

上一步的重啟完成之後,點選system setting – >software & updates – > addtional drivers 。然後點選Using NVIDIA binary driver -version 367.57,再點選右下角Apply Changes。
additional drivers

之後就是等待,等安裝完成後重啟電腦。如果安裝正常,就可以在System Detail中看到如下介面了(Graphics 變成了)。

system detail
執行nvidia-smi可以看到更詳細的資訊。 
nvidia-smi

到此,NVIDIA驅動安裝完畢。
輸入指令檢視是否安裝成功 nvidia-smi
安裝過程如下:

如果沒有GPU支援的話就可以直接跳到第10步。
安裝CUDA 8.0
cuda下載地址:https://developer.nvidia.com/cuda-downloads
驗證過程省略,按照官方文件自己操作吧(遇到問題首先要看官方文件啊,血淚教訓)
安裝CUDA有兩種方法,
離線.run安裝:從官網下載對應版本的.run安裝包安裝,安裝過程挺複雜,嘗試過幾次沒成功,遂放棄。
在離線.deb安裝:deb安裝分離線和線上,我都嘗試過都安裝成功了,官網下載地址
安裝之前請先進行md5校驗,確保下載的安裝包完整
1.下載cuda_8.0.27_linux.run和cudnn-8.0-linux-x64-v5.1.tgz
這裡我提供了百度網盤,這兩個檔案我先在win10下下載好,並用u盤拷貝到ubuntu的下載目錄下
2.安裝cuda8.0
終端cd 下載/

sh cuda_8.0.27_linux.run --override 

啟動安裝程式,一直按空格到最後,輸入accept接受條款

輸入n不安裝nvidia影象驅動,之前已經安裝過了 
輸入y安裝cuda 8.0工具 
回車確認cuda預設安裝路徑:/usr/local/cuda-8.0 
輸入y用sudo許可權執行安裝,輸入密碼 
輸入y或者n安裝或者不安裝指向/usr/local/cuda的符號連結
輸入y安裝CUDA 8.0 Samples,以便後面測試
Installing the CUDA Toolkit in /usr/local/cuda-8.0 …
Missing recommended library: libXi.so
Missing recommended library: libXmu.so 
Installing the CUDA Samples in /home/zhou …
Copying samples to /home/zhou/NVIDIA_CUDA-8.0_Samples now…
Finished copying samples. 
===========
= Summary =
===========
Driver: Not Selected
Toolkit: Installed in /usr/local/cuda-8.0
Samples: Installed in /home/zhou, but missing recommended libraries 
Please make sure that
- PATH includes /usr/local/cuda-8.0/bin
- LD_LIBRARY_PATH includes /usr/local/cuda-8.0/lib64, or, add /usr/local/cuda-8.0/lib64 to /etc/ld.so.conf and run ldconfig as root 
To uninstall the CUDA Toolkit, run the uninstall script in /usr/local/cuda-8.0/bin 
Please see CUDA_Installation_Guide_Linux.pdf in /usr/local/cuda-8.0/doc/pdf for detailed information on setting up CUDA. 
***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 361.00 is required for CUDA 8.0 functionality to work.
To install the driver using this installer, run the following command, replacing with the name of this run file:
sudo .run -silent -driver 
Logfile is /tmp/cuda_install_2961.log

安裝完成,但是缺少一些庫。
安裝sudo apt-get install libxmu-dev libxi-dev
安裝完成。
4) 設定環境變數
在終端輸入這兩句:

export PATH=/usr/local/cuda-8.0/bin:$PATHexport LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH

然後修改檔案中環境變數設定

sudo vi /etc/profile

輸入上面export的兩句,儲存,退出。

sudo ldconfig //環境變數立即生效

5) 驗證安裝是否完成

nvidia-smi 
nvcc –V
顯示: 
nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2016
NVIDIA Corporation Built on Wed_May__4_21:01:56_CDT_2016 Cuda
compilation tools, release 8.0, V8.0.26 

6) 測試cuda的samples

sudo apt-get install gcc
sudo apt-get install g++
cd 下載/ 
tar zxvf cudnn-8.0-linux-x64-v5.1.tgz 

解壓在下載目錄下產生一個cuda目錄
複製標頭檔案 安裝cuDNN比較簡單,解壓後把相應的檔案拷貝到對應的CUDA目錄下即可:

tar -zxvf cudnn-8.0-linux-x64-v5.1.tgz

顯示以下資訊:

*cuda/include/cudnn.h
cuda/lib64/libcudnn.so
cuda/lib64/libcudnn.so.5
cuda/lib64/libcudnn.so.5.0.5
cuda/lib64/libcudnn_static.a*

繼續執行以下指令:

sudo cp cuda/include/cudnn.h /usr/local/cuda/include/sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64/sudo chmod a+r /usr/local/cuda/include/cudnn.hsudo chmod a+r /usr/local/cuda/lib64/libcudnn*

8) 驗證一下;

cd /NVIDIA_CUDA-8.0_Samples/1_Utilities/deviceQuery
make

然後deviceQuery就生成了./deviceQuery 就可以了。
cuda的samples 裡面有個deviceQuery 執行之後會顯示資訊,最後一行出行pass說明成功啦

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 8.0, NumDevs = 1, Device0 = GeForce GTX 1050 Ti
Result = PASS

安裝開發所需的依賴包

sudo apt-get install build-essential vim cmake git libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler 
sudo apt-get install --no-install-recommends 
sudo apt-get install libatlas-base-dev 
sudo apt-get install python-dev 

編譯Caffe
下載caffe:

git clone https://github.com/bvlc/caffe.git 
cd caffe/ 
mv Makefile.config.example Makefile.config

配置CPU版本時
修改Makefile.config檔案,開啟CPU_ONLY選項,儲存

## Refer to http://caffe.berkeleyvision.org/installation.html  
# Contributions simplifying and improving our build system are welcome!  

# cuDNN acceleration switch (uncomment to build with cuDNN).  
USE_CUDNN := 1  

# CPU-only switch (uncomment to build without GPU support).  
# CPU_ONLY := 1  

# To customize your choice of compiler, uncomment and set the following.  
# N.B. the default for Linux is g++ and the default for OSX is clang++  
# CUSTOM_CXX := g++  

# CUDA directory contains bin/ and lib/ directories that we need.  
CUDA_DIR := /usr/local/cuda  
# On Ubuntu 14.04, if cuda tools are installed via  
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:  
# CUDA_DIR := /usr  

# CUDA architecture setting: going with all of them.  
# For CUDA < 6.0, comment the *_50 lines for compatibility.  
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \  
        -gencode arch=compute_20,code=sm_21 \  
        -gencode arch=compute_30,code=sm_30 \  
        -gencode arch=compute_35,code=sm_35 \  
        -gencode arch=compute_50,code=sm_50 \  
        -gencode arch=compute_50,code=compute_50  

# BLAS choice:  
# atlas for ATLAS (default)  
# mkl for MKL  
# open for OpenBlas  
BLAS := atlas  
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.  
# Leave commented to accept the defaults for your choice of BLAS  
# (which should work)!  
# BLAS_INCLUDE := /path/to/your/blas  
# BLAS_LIB := /path/to/your/blas  

# Homebrew puts openblas in a directory that is not on the standard search path  
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include  
# BLAS_LIB := $(shell brew --prefix openblas)/lib  

# This is required only if you will compile the matlab interface.  
# MATLAB directory should contain the mex binary in /bin.  
# MATLAB_DIR := /usr/local  
# MATLAB_DIR := /Applications/MATLAB_R2012b.app  

# NOTE: this is required only if you will compile the python interface.  
# We need to be able to find Python.h and numpy/arrayobject.h.  
#PYTHON_INCLUDE := /usr/include/python2.7 \  
        /usr/lib/python2.7/dist-packages/numpy/core/include  
# Anaconda Python distribution is quite popular. Include path:  
# Verify anaconda location, sometimes it's in root.  
 ANACONDA_HOME := $(HOME)/anaconda  
 PYTHON_INCLUDE := $(ANACONDA_HOME)/include \  
         $(ANACONDA_HOME)/include/python2.7 \  
         $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \  

# We need to be able to find libpythonX.X.so or .dylib.  
#PYTHON_LIB := /usr/lib  
PYTHON_LIB := $(ANACONDA_HOME)/lib  

# Homebrew installs numpy in a non standard path (keg only)  
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include  
# PYTHON_LIB += $(shell brew --prefix numpy)/lib  

# Uncomment to support layers written in Python (will link against Python libs)  
# WITH_PYTHON_LAYER := 1  

# Whatever else you find you need goes here.  
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include  
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib  

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies  
# INCLUDE_DIRS += $(shell brew --prefix)/include  
# LIBRARY_DIRS += $(shell brew --prefix)/lib  

# Uncomment to use `pkg-config` to specify OpenCV library paths.  
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)  
# USE_PKG_CONFIG := 1  

BUILD_DIR := build  
DISTRIBUTE_DIR := distribute  

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171  
# DEBUG := 1  

# The ID of the GPU that 'make runtest' will use to run unit tests.  
TEST_GPUID := 0  

# enable pretty build (comment to see full commands)  
Q ?= @  
儲存退出
配置GPU版本時:
根據個人情況修改檔案:
a.若使用cudnn,則
將
#USE_CUDNN := 1
修改成: 
USE_CUDNN := 1
b.若使用的opencv版本是3的,則
將
#OPENCV_VERSION := 3 
修改為: 
OPENCV_VERSION := 3
c.若要使用Python來編寫layer,則
將
#WITH_PYTHON_LAYER := 1 
修改為 
WITH_PYTHON_LAYER := 1

安裝caffe的Python介面及python相關庫
安裝完該介面後,就能夠在python中,通過import caffe方式來使用caffe,如此可以用到python的一些強大的庫用來顯示和計算,如matplotlib,sklearn等,這是命令列式caffe所難以達到的效果。
Cd caffe/python

1)安裝python包安裝工具pip 
sudo apt-get install python-dev python-pip2)sudo apt-get install python-opencv 
 sudo pip install pyzmq tornado jinja2 jsonschema jupyter 
(3)安裝python包依賴,在requirements.txt檔案裡 
sudo apt-get install gfortran libatlas-dev libblas-dev 

此步應先執行,否則下一步requirement裡的scipy會安裝失敗
安裝requirement裡的python依賴包

for req in $(cat requirements.txt);do sudo pip install $req; done 

(4)python包都安裝無誤後,編譯pycaffe

Cd .. 
Make pycaffe –j8 

此時即可在python裡import caffe
編譯

make all -j4  
make test  
make runtest  

遇到錯誤&&解決方法:
error while loading shared libraries: libcudart.so.8.0: cannot open shared object file: No such file or directory

sudo cp /usr/local/cuda-8.0/lib64/libcudart.so.8.0.44 /usr/local/lib/libcudart.so.8.0 && sudo ldconfig
sudo cp /usr/local/cuda-8.0/lib64/libcublas.so.8.0.45 /usr/local/lib/libcublas.so.8.0 && sudo ldconfig
sudo cp /usr/local/cuda-8.0/lib64/libcurand.so.8.0.44 /usr/local/lib/libcurand.so.8.0 && sudo ldconfig
sudo cp /usr/local/cuda-8.0/lib64/libcudnn.so.5 /usr/local/lib/libcudnn.so.5 && sudo ldconfig

編譯Python wrapper錯誤總結

make  pycaffe  

遇到錯誤:
make pycaffe出現錯誤
python/caffe/_caffe.cpp:10:31: fatal error: numpy/arrayobject.h: no such file or directory
Makefile:498: recipe for target ‘python/caffe/_caffe.so’ failed make: * [python/caffe/_caffe.so] Error 1
原因
python-numpy沒有安裝到位
解決方法
輸入命令

sudo apt-get install python-numpy

錯誤提示:
building ‘dfftpack’ library
error: library dfftpack has Fortran sources but no Fortran compiler found
也就是沒有安裝gfortran編譯器

sudo apt-get install gfortran  

ImportError: No module named packaging.version

wget https://bootstrap.pypa.io/get-pip.pysudo python get-pip.py
pip install setuptools 

安裝MATLAB
MATLAB安裝步驟見我之後的筆記。
caffe中配置matlab:
修改Makefile.config檔案,將MATLAB目錄新增進來

修改Makefile.config,加上這一句:
MATLAB_DIR := /opt//MATLAB/R2014a
之後
make matcaffe

生成了 matlab/+caffe/private/caffe_.mex64,可以直接被MATLAB呼叫。
Ubuntu14.04自帶的gcc版本是4.8,MATLAB2014a支援的最高版本為4.7x。因此,需要安裝gcc4.7,並給gcc降級
否則會出現錯誤:
error: no matching function for call to ‘remove_if(std::vector

 sudo apt-getupdate
 sudo apt-get install gcc-4.7 g++-4.7 g++-4.7-multilib gcc-4.7-multilib

在終端執行以下系統gcc降級命令(其中數字越大,表示優先順序越高):

sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.7 100 
sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 50 
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.7 100 
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 50
sudo update-alternatives --install /usr/bin/cpp cpp-bin /usr/bin/cpp-4.7 100
sudo update-alternatives --install /usr/bin/cpp cpp-bin /usr/bin/cpp-4.8 50

驗證gcc-4.7是否安裝併成為系統的預設版本:

gcc -v
make -j4
makematcaffe -j4以上4表示採用4核。
驗證gcc-4.7是否安裝併成為系統的預設版本: 
gcc -v

/home/zkk/caffe/matlab/+caffe/private/caffe_.cpp:208:4: error: no matching function for call to ‘remove_if(std::vector