01 資料

02 測試操作步驟

02.01 下載測試資料

根據examples\cifar10\readme.md檔案說明。開啟data\cifar10\get_cifar10.sh檔案。

#!/usr/bin/env sh
# This scripts downloads the CIFAR10 (binary version) data and unzips it.

DIR="$( cd "$(dirname "$0")" ; pwd -P )"
cd "$DIR"

echo "Downloading..."

wget --no-check-certificate http://www.cs.toronto.edu/~kriz/cifar-10-binary.tar.gz

echo "
 
Unzipping..."

tar -xf cifar-10-binary.tar.gz && rm -f cifar-10-binary.tar.gz
mv cifar-10-batches-bin/* . && rm -rf cifar-10-batches-bin

# Creation is split out because leveldb sometimes causes segfault
# and needs to be re-created.

echo "Done."

02.02 生成lmdb資料

閱讀caffe\examples\cifar10\create_cifar10.sh內容。這個是Linux下面生成lmdb資料庫的指令碼。

#!/usr/bin/env sh
# This script converts the cifar data into leveldb format.
set -e

EXAMPLE=examples/cifar10
DATA=data/cifar10
DBTYPE=lmdb

echo "Creating $DBTYPE..."

rm -rf $EXAMPLE/cifar10_train_$DBTYPE $EXAMPLE/cifar10_test_$DBTYPE

./build/examples/cifar10/convert_cifar_data.bin $DATA $EXAMPLE $DBTYPE

echo "Computing image mean..."

./build/tools/compute_image_mean -backend=$DBTYPE \
  $EXAMPLE/cifar10_train_$DBTYPE $EXAMPLE/mean.binaryproto

echo "Done."

其中convert_cifar_data.bin對應vs2015的工程中的exaples目錄下的convert_cifar_data；compute_image_mean對應vs2015工程中的tools目錄下的compute_image_mean。

生成的可執行檔案是
build\x64\examples\cifar10\Debug\convert_cifar_data-d.exe
build\x64\examples\cifar10\Release\convert_cifar_data.exe
build\x64\tools\Debug\compute_image_mean-d.exe
build\x64\tools\Release\compute_image_mean.exe
如果運行了INSTALL，build\x64\install\bin下面也會有對應的檔案。
為了使用方便，模擬create_cifar10.sh寫一個windows版本的create_cifar10.bat。放在create_cifar10.sh相同目錄下。(注意文字最好採用utf8格式儲存)

@echo off & setlocal enabledelayedexpansion
:: examples\cifar10\create_cifar10.bat
::進入create_cifar10.bat所在磁碟
%~d0
::進入create_cifar10.bat所在目錄
cd %~dp0
:: 回到caffe根目錄
cd ..\..\

:: 網路定義檔案的位置，lmdb也生成在這裡
set EXAMPLE=examples\cifar10
:: 測試資料的位置，
set DATA=data\cifar10\cifar-10-batches-bin

:: 工程檔案生成的位置
::set cfg=Debug
::set BUILD=build\x64\examples\cifar10\%cfg%
::set convert_cifar_data=convert_cifar_data-d.exe
::set compute_image_mean=build\x64\tools\compute_image_mean-d.exe

:: 釋出檔案所在位置，即vs2015工程中執行INSTALL後,檔案的安裝位置
set BUILD=build\x64\install\bin
:: 可釋出的release版本lmdb生成工具
set convert_cifar_data=convert_cifar_data.exe
set compute_image_mean=build\x64\install\bin\compute_image_mean.exe
:: 生成資料型別
set DBTYPE=lmdb

echo.
echo "Creating !DBTYPE! start"
echo.
:: 刪除存在的資料庫目錄，如果目錄存在，會執行失敗
IF EXIST %EXAMPLE%\cifar10_train_%DBTYPE%. echo rmdir /S /Q %EXAMPLE%\cifar10_train_%DBTYPE%.
IF EXIST %EXAMPLE%\cifar10_train_%DBTYPE%. rmdir /S /Q %EXAMPLE%\cifar10_train_%DBTYPE%.
IF EXIST %EXAMPLE%\cifar10_test_%DBTYPE%. echo rmdir /S /Q %EXAMPLE%\cifar10_test_%DBTYPE%.
IF EXIST %EXAMPLE%\cifar10_test_%DBTYPE%. rmdir /S /Q %EXAMPLE%\cifar10_test_%DBTYPE%.

echo %BUILD%\%convert_cifar_data% %DATA% %EXAMPLE% %DBTYPE%
%BUILD%\%convert_cifar_data% %DATA% %EXAMPLE% %DBTYPE%
echo.
echo "Computing image mean..."
echo %compute_image_mean% -backend=%DBTYPE% ^
  %EXAMPLE%\cifar10_train_%DBTYPE% %EXAMPLE%\mean.binaryproto
%compute_image_mean% -backend=%DBTYPE% ^
  %EXAMPLE%\cifar10_train_%DBTYPE% %EXAMPLE%\mean.binaryproto
echo "Creating !DBTYPE! Done."

endlocal&goto :EOF
:EOF

儲存examples\mnist\create_cifar10.bat檔案後，在caffe根目錄執行：

examples\mnist\create_cifar10.bat

或者
在caffe根目錄下直接執行如下語句。(執行前確保examples\cifar10\目錄下沒有cifar10_train_lmdb和cifar10_test_lmdb目錄)

生成的資料庫檔案在examples\cifar10\目錄下。

02.03 執行測試資料

開啟caffe\examples\cifar10\train_quick.sh指令碼。內容如下：

#!/usr/bin/env sh
set -e

TOOLS=./build/tools

$TOOLS/caffe train \
  --solver=examples/cifar10/cifar10_quick_solver.prototxt [email protected]

# reduce learning rate by factor of 10 after 8 epochs
$TOOLS/caffe train \
  --solver=examples/cifar10/cifar10_quick_solver_lr1.prototxt \
  --snapshot=examples/cifar10/cifar10_quick_iter_4000.solverstate.h5 [email protected]

根據caffe\examples\cifar10\train_quick.sh內容，改寫windows bat指令碼caffe\examples\cifar10\train_quick.bat。(注意：最好採用utf8編碼儲存)。

@echo off & setlocal enabledelayedexpansion
:: examples\cifar10\train_quick.bat
::train_quick.bat所在磁碟
%~d0
::train_quick.bat所在目錄
cd %~dp0
:: 回到caffe根目錄
cd ..\..\
::caffe執行檔案目錄
::set caffe_path=build\x64\install\bin
set caffe_path=build\x64\tools\release
echo.
echo %caffe_path%\caffe train ^
  --solver=examples\cifar10\cifar10_quick_solver.prototxt
%caffe_path%\caffe train ^
  --solver=examples\cifar10\cifar10_quick_solver.prototxt
echo.
:: reduce learning rate by factor of 10 after 8 epochs
echo %caffe_path%\caffe train ^
  --solver=examples\cifar10\cifar10_quick_solver_lr1.prototxt ^
  --snapshot=examples\cifar10\cifar10_quick_iter_4000.solverstate.h5
%caffe_path%\caffe train ^
  --solver=examples\cifar10\cifar10_quick_solver_lr1.prototxt ^
  --snapshot=examples\cifar10\cifar10_quick_iter_4000.solverstate.h5
echo.
::mkdir build\x64\log
::examples\cifar10\train_quick.bat > build\x64\log\cifar10_train_quick.log 2>&1
endlocal&goto :EOF
:EOF

在caffe根目錄執行examples\cifar10\train_quick.bat。

examples\cifar10\train_quick.bat

執行結果：

02.04 儲存執行日誌

在caffe根目錄下(D:\Git\DeepLearning\caffe)執行如下命令。建立一個執行日誌目錄，執行測試用例。把測試結果重定向到指定執行日誌檔案。

mkdir build\x64\log
examples\cifar10\train_quick.bat > build\x64\log\cifar10_train_quick.log 2>&1

執行後，檢視日誌檔案build\x64\log\cifar10_train_quick.log。

02.05 執行其他cifar10測試用例

根據examples/cifar/*.sh，可以執行其他測試網路。
比如train_full.sh：

#!/usr/bin/env sh
set -e

TOOLS=./build/tools

$TOOLS/caffe train \
    --solver=examples/cifar10/cifar10_full_solver.prototxt [email protected]

# reduce learning rate by factor of 10
$TOOLS/caffe train \
    --solver=examples/cifar10/cifar10_full_solver_lr1.prototxt \
    --snapshot=examples/cifar10/cifar10_full_iter_60000.solverstate.h5 [email protected]

# reduce learning rate by factor of 10
$TOOLS/caffe train \
    --solver=examples/cifar10/cifar10_full_solver_lr2.prototxt \
    --snapshot=examples/cifar10/cifar10_full_iter_65000.solverstate.h5 [email protected]

根據train_full.sh指令碼描述，在cmd視窗，caffe根目錄一次執行如下命令，即完成train_full.sh測試用例。(train_full.sh測試用例比較耗時)

build\x64\tools\Release\caffe train --solver=examples\cifar10\cifar10_full_solver.prototxt

build\x64\tools\Release\caffe train --solver=examples\cifar10\cifar10_full_solver_lr1.prototxt --snapshot=examples\cifar10\cifar10_full_iter_60000.solverstate.h5

build\x64\tools\Release\caffe train --solver=examples\cifar10\cifar10_full_solver_lr2.prototxt --snapshot=examples\cifar10\cifar10_full_iter_65000.solverstate.h5