caffe編譯好後在caffe/build/tools下會生成一些工具，一一介紹使用方法。

一、caffe

這是最重要的工具

caffe: command line brew
usage: caffe <command> <args>

commands:
  train           train or finetune a model
  test            score a model
  device_query    show GPU diagnostic information
  time            benchmark model execution time

  Flags from tools/caffe.cpp:
    -gpu (Optional; run in GPU mode on given device IDs separated by ','.Use
      '-gpu all' to run on all available GPUs. The effective training batch
      size is multiplied by the number of devices.) type: string default: ""
    -iterations (The number of iterations to run.) type: int32 default: 50
    -level (Optional; network level.) type: int32 default: 0
    -model (The model definition protocol buffer text file.) type: string
      default: ""
    -phase (Optional; network phase (TRAIN or TEST). Only used for 'time'.)
      type: string default: ""
    -sighup_effect (Optional; action to take when a SIGHUP signal is received:
      snapshot, stop or none.) type: string default: "snapshot"
    -sigint_effect (Optional; action to take when a SIGINT signal is received:
      snapshot, stop or none.) type: string default: "stop"
    -snapshot (Optional; the snapshot solver state to resume training.)
      type: string default: ""
    -solver (The solver definition protocol buffer text file.) type: string
      default: ""
    -stage (Optional; network stages (not to be confused with phase), separated
      by ','.) type: string default: ""
    -weights (Optional; the pretrained weights to initialize finetuning,
      separated by ','. Cannot be set simultaneously with snapshot.)
      type: string default: ""
 

1. train 命令

訓練模型時的命令。 常接選項是

• solver：後接solve的配置檔案
• gpu：GPU的id號
• weight：權重，用於tuning，caffemodel檔案
• snapshot：儲存了訓練中的中間狀態

但是注意snapshot和weight不同時使用。

CHECK(!FLAGS_snapshot.size() || !FLAGS_weights.size())
      << "Give a snapshot to resume training or weights to finetune "
      "but not both.";
 

使用範例如下

GLOG_logtostderr=1 $CAFFETOOL/caffe train \
   --solver=solver_1st.prototxt \
   --weights=VGG/VGG_ILSVRC_16_layers.caffemodel \
   --gpu=0,1,2,3  2>&1 | tee log_1st.txt

翻譯一下官網介紹

caffe train 可以從頭學習模型，可以從快照中恢復模型，可以fine-tunes模型。
•必須的引數 -solver solver.prototxt ,配置solver檔案. 
•如果是從快照中恢復，需要 -snapshot model_iter_1000.solverstate 引數
•Fine-tuning 需要 -weights model.caffemodel 引數來做模型初始化 .

例如:

# train LeNet
caffe train -solver examples/mnist/lenet_solver.prototxt

# train on GPU 2
caffe train -solver examples/mnist/lenet_solver.prototxt -gpu 2

# 從快照中恢復
caffe train -solver examples/mnist/lenet_solver.prototxt -snapshot examples/mnist/lenet_iter_5000.solverstate

# fine-tune
caffe train -solver examples/finetuning_on_flickr_style/solver.prototxt -weights models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel
 

2. test 命令

官網介紹

caffe test scores models by running them in the test phase and reports the net output as its score. The net architecture must be properly defined to output an accuracy measure or loss as its output. The per-batch score is reported and then the grand average is reported last.

# score the learned LeNet model on the validation set as defined in the
# model architeture lenet_train_test.prototxt
caffe test -model examples/mnist/lenet_train_test.prototxt -weights examples/mnist/lenet_iter_10000.caffemodel -gpu 0 -iterations 100

模型的輸出層一定要是一個準確率或者損失層。test命令就是用訓練出來的model測試TEST資料，可以修改solver檔案中TEST資料達到測試其他資料集的效果。 執行結果大致如下

I1222 10:31:07.233053 27436 caffe.cpp:308] Batch 45, accuracy = 0.97
I1222 10:31:07.233103 27436 caffe.cpp:308] Batch 45, loss = 0.165972
I1222 10:31:07.301012 27436 caffe.cpp:308] Batch 46, accuracy = 0.98
I1222 10:31:07.301057 27436 caffe.cpp:308] Batch 46, loss = 0.126905
I1222 10:31:07.369249 27436 caffe.cpp:308] Batch 47, accuracy = 0.97
I1222 10:31:07.369297 27436 caffe.cpp:308] Batch 47, loss = 0.161663
I1222 10:31:07.437602 27436 caffe.cpp:308] Batch 48, accuracy = 0.99
I1222 10:31:07.437649 27436 caffe.cpp:308] Batch 48, loss = 0.0438317
I1222 10:31:07.505509 27436 caffe.cpp:308] Batch 49, accuracy = 0.98
I1222 10:31:07.505556 27436 caffe.cpp:308] Batch 49, loss = 0.0786786
I1222 10:31:07.505565 27436 caffe.cpp:313] Loss: 0.134281
I1222 10:31:07.505623 27436 caffe.cpp:325] accuracy = 0.9752
I1222 10:31:07.505653 27436 caffe.cpp:325] loss = 0.134281 (* 1 = 0.134281 loss)

3. device_query

官網介紹

caffe device_query reports GPU details for reference and checking device ordinals for running on a given device in multi-GPU machines.

# query the first device
caffe device_query -gpu 0

device_query 用於檢測GPU的裝置資訊, 執行結果如下

[[email protected] tools]$ ./caffe device_query -gpu 0  
I1222 11:00:09.872097 26694 caffe.cpp:138] Querying GPUs 0
I1222 11:00:10.188871 26694 common.cpp:177] Device id:                     0
I1222 11:00:10.188911 26694 common.cpp:178] Major revision number:         3
I1222 11:00:10.188916 26694 common.cpp:179] Minor revision number:         5
I1222 11:00:10.188920 26694 common.cpp:180] Name:                          Tesla K40m
I1222 11:00:10.188925 26694 common.cpp:181] Total global memory:           11995578368
I1222 11:00:10.188933 26694 common.cpp:182] Total shared memory per block: 49152
I1222 11:00:10.188938 26694 common.cpp:183] Total registers per block:     65536
I1222 11:00:10.188942 26694 common.cpp:184] Warp size:                     32
I1222 11:00:10.188946 26694 common.cpp:185] Maximum memory pitch:          2147483647
I1222 11:00:10.188951 26694 common.cpp:186] Maximum threads per block:     1024
I1222 11:00:10.188954 26694 common.cpp:187] Maximum dimension of block:    1024, 1024, 64
I1222 11:00:10.188959 26694 common.cpp:190] Maximum dimension of grid:     2147483647, 65535, 65535
I1222 11:00:10.188964 26694 common.cpp:193] Clock rate:                    745000
I1222 11:00:10.188967 26694 common.cpp:194] Total constant memory:         65536
I1222 11:00:10.188972 26694 common.cpp:195] Texture alignment:             512
I1222 11:00:10.188977 26694 common.cpp:196] Concurrent copy and execution: Yes
I1222 11:00:10.188980 26694 common.cpp:198] Number of multiprocessors:     15
I1222 11:00:10.188984 26694 common.cpp:199] Kernel execution timeout:      No

4. time

caffe time 可以測試網路時間，官網介紹如下：

caffe time benchmarks model execution layer-by-layer through timing and synchronization. This is useful to check system performance and measure relative execution times for models.

# (These example calls require you complete the LeNet / MNIST example first.)
# time LeNet training on CPU for 10 iterations
caffe time -model examples/mnist/lenet_train_test.prototxt -iterations 10

# time LeNet training on GPU for the default 50 iterations
caffe time -model examples/mnist/lenet_train_test.prototxt -gpu 0

# time a model architecture with the given weights on the first GPU for 10 iterations
caffe time -model examples/mnist/lenet_train_test.prototxt -weights examples/mnist/lenet_iter_10000.caffemodel -gpu 0 -iterations 10

例項：

#!/usr/bin/env sh
set -e
CAFFETOOL=/home/users/fangjin/caffe/build/tools

${CAFFETOOL}/caffe time --solver=lenet_solver.prototxt --model=lenet_train_test.prototxt --weights=lenet_iter_10000.caffemodel --iterations=1 --gpu=0,1,2

執行結果

I1222 11:12:06.742219 11065 caffe.cpp:361] Performing Backward
I1222 11:12:06.803246 11065 caffe.cpp:369] *** Benchmark begins ***
I1222 11:12:06.803274 11065 caffe.cpp:370] Testing for 1 iterations.
I1222 11:12:06.900956 11065 caffe.cpp:398] Iteration: 1 forward-backward time: 97 ms.
I1222 11:12:06.901000 11065 caffe.cpp:401] Average time per layer: 
I1222 11:12:06.901006 11065 caffe.cpp:404]      mnist   forward: 0.16 ms.
I1222 11:12:06.901027 11065 caffe.cpp:407]      mnist   backward: 0.001 ms.
I1222 11:12:06.901062 11065 caffe.cpp:404]      conv1   forward: 17.082 ms.
I1222 11:12:06.901072 11065 caffe.cpp:407]      conv1   backward: 17.722 ms.
I1222 11:12:06.901080 11065 caffe.cpp:404]      pool1   forward: 7.324 ms.
I1222 11:12:06.901088 11065 caffe.cpp:407]      pool1   backward: 3.516 ms.
I1222 11:12:06.901096 11065 caffe.cpp:404]      conv2   forward: 15.294 ms.
I1222 11:12:06.901104 11065 caffe.cpp:407]      conv2   backward: 28.055 ms.
I1222 11:12:06.901111 11065 caffe.cpp:404]      pool2   forward: 3.847 ms.
I1222 11:12:06.901119 11065 caffe.cpp:407]      pool2   backward: 2.194 ms.
I1222 11:12:06.901126 11065 caffe.cpp:404]        ip1   forward: 0.742 ms.
I1222 11:12:06.901134 11065 caffe.cpp:407]        ip1   backward: 1.185 ms.
I1222 11:12:06.901141 11065 caffe.cpp:404]      relu1   forward: 0.057 ms.
I1222 11:12:06.901149 11065 caffe.cpp:407]      relu1   backward: 0.117 ms.
I1222 11:12:06.901156 11065 caffe.cpp:404]        ip2   forward: 0.101 ms.
I1222 11:12:06.901165 11065 caffe.cpp:407]        ip2   backward: 0.114 ms.
I1222 11:12:06.901171 11065 caffe.cpp:404]       loss   forward: 0.105 ms.
I1222 11:12:06.901180 11065 caffe.cpp:407]       loss   backward: 0.004 ms.
I1222 11:12:06.901190 11065 caffe.cpp:412] Average Forward pass: 44.719 ms.
I1222 11:12:06.901197 11065 caffe.cpp:414] Average Backward pass: 52.923 ms.
I1222 11:12:06.901206 11065 caffe.cpp:416] Average Forward-Backward: 97 ms.
I1222 11:12:06.901213 11065 caffe.cpp:418] Total Time: 97 ms.
I1222 11:12:06.901219 11065 caffe.cpp:419] *** Benchmark ends ***

5. 其他

注意GPU這個引數項

-gpu (Optional; run in GPU mode on given device IDs separated by ','.Use
      '-gpu all' to run on all available GPUs. The effective training batch
      size is multiplied by the number of devices.) type: string default: ""

接多個GPU時，使用逗號隔開，如'-gpu 0,1,2'，全部GPU則使用’-gpu all

引數項可以寫-gpu 0或者--gpu=0，兩種寫法都可以。

二、compute_image_mean

這個命令用於計算影象均值。

在前面介紹可以看到，使用了一個

MEAN_VALUE = 128

在caffe可以先計算均值，然後對所有影象去除均值，在上文中是將均值寫成固定的128，現在展示如何使用caffe自己編譯的工具計算均值。

compute_image_mean: Compute the mean_image of a set of images given by a leveldb/lmdb
Usage:
    compute_image_mean [FLAGS] INPUT_DB [OUTPUT_FILE]


  Flags from tools/compute_image_mean.cpp:
    -backend (The backend {leveldb, lmdb} containing the images) type: string
      default: "lmdb"

寫指令碼compute_image_mean.sh

#!/usr/bin/env sh
set -e

CAFFETOOL=/home/users/fangjin/caffe/build/tools

${CAFFETOOL}/compute_image_mean number_train_lmdb image_mean.binaryproto  

就會生成均值檔案image_mean.binaryproto。 使用方式是將Data層的transform_param新增一個mean_file,訓練集和驗證集都要新增

transform_param {
    scale: 0.00390625
    mean_file: "F:/caffe/data/image_mean.binaryproto"
  }

然後同樣的方式訓練。

三、convert_imageset

convert_imageset: Convert a set of images to the leveldb/lmdb
format used as input for Caffe.
Usage:
    convert_imageset [FLAGS] ROOTFOLDER/ LISTFILE DB_NAME
The ImageNet dataset for the training demo is at
    http://www.image-net.org/download-images


  Flags from tools/convert_imageset.cpp:
    -backend (The backend {lmdb, leveldb} for storing the result) type: string
      default: "lmdb"
    -check_size (When this option is on, check that all the datum have the same
      size) type: bool default: false
    -encode_type (Optional: What type should we encode the image as
      ('png','jpg',...).) type: string default: ""
    -encoded (When this option is on, the encoded image will be save in datum)
      type: bool default: false
    -gray (When this option is on, treat images as grayscale ones) type: bool
      default: false
    -resize_height (Height images are resized to) type: int32 default: 0
    -resize_width (Width images are resized to) type: int32 default: 0
    -shuffle (Randomly shuffle the order of images and their labels) type: bool
      default: false

TOOLS=/home/users/fangjin/caffe/build/tools                                             
ESIZE_HEIGHT=32
RESIZE_WIDTH=32
TRAIN_DATA_ROOT=/home/users/fangjin/test/number_data/

echo "Creating train lmdb..."
GLOG_logtostderr=1 $TOOLS/convert_imageset \
   --resize_height=32 \
   --resize_width=32 \
   --shuffle \
   $TRAIN_DATA_ROOT \
   train.txt \
   number_train_lmdb

echo "Creating test lmdb..."
GLOG_logtostderr=1 $TOOLS/convert_imageset \
   --resize_height=32 \
   --resize_width=32 \
   --shuffle \
   $TRAIN_DATA_ROOT \
   test.txt \
   number_test_lmdb  #輸出

四、extract_features

This program takes in a trained network and an input data layer, and then extract features of the input data produced by the net.
Usage: extract_features  pretrained_net_param  feature_extraction_proto_file  extract_feature_blob_name1[,name2,...]  save_feature_dataset_name1[,name2,...]  num_mini_batches  db_type  [CPU/GPU] [DEVICE_ID=0]
Note: you can extract multiple features in one pass by specifying multiple feature blob names and dataset names separated by ','. The names cannot contain white space characters and the number of blobs and datasets must be equal.

就是一個C++的介面用於提取特徵，現實中作用可能並沒那麼大，現在很多都是使用python介面，比較方便。

官網上提供了一個例子。

先建立一個臨時檔案

mkdir examples/_temp

將圖片路徑寫入文字

find `pwd`/examples/images -type f -exec echo {} \; > examples/_temp/temp.txt

temp.txt檔案內容如下

/home/users/fangjin/caffe/examples/images/cat gray.jpg                                  
/home/users/fangjin/caffe/examples/images/cat.jpg
/home/users/fangjin/caffe/examples/images/cat_gray.jpg
/home/users/fangjin/caffe/examples/images/fish-bike.jpg

將標籤也寫入文字，這裡統一用0

sed "s/$/ 0/" examples/_temp/temp.txt > examples/_temp/file_list.txt

file_list.txt內容如下

/home/users/fangjin/caffe/examples/images/cat gray.jpg 0                                  
/home/users/fangjin/caffe/examples/images/cat.jpg 0
/home/users/fangjin/caffe/examples/images/cat_gray.jpg 0
/home/users/fangjin/caffe/examples/images/fish-bike.jpg 0

用指令碼./data/ilsvrc12/get_ilsvrc_aux.sh下載 ILSVRC 的均值。 把

我們使用data/ilsvrc212/imagenet_mean.binaryproto 和對應的prototxt檔案.

拷貝一份imagenet_val.prototxt

cp examples/feature_extraction/imagenet_val.prototxt examples/_temp

現在開始執行指令碼

./build/tools/extract_features.bin models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel examples/_temp/imagenet_val.prototxt fc7 examples/_temp/features 10 leveldb

這裡提取的是fc7這一層的特徵，也可以使用conv5,pool3等。最後的數字引數是mini-batched的個數，特徵儲存於examples/_temp/features中，leveldb格式。

[[email protected] caffe]$ ./build/tools/extract_features.bin models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel examples/_temp/imagenet_val.prototxt fc7 examples/_temp/features 10 leveldb
E1222 14:11:18.487232 14915 extract_features.cpp:62] Using CPU
E1222 14:11:19.759531 14915 extract_features.cpp:133] Extracting Features
E1222 14:11:59.131815 14915 extract_features.cpp:176] Extracted features of 500 query images for feature blob fc7
E1222 14:11:59.132774 14915 extract_features.cpp:181] Successfully extracted the features!          

五、其他

其他的一些命令，例如finetune_net等已經被拋棄了，統一在caffe命令接口裡。剩下的三個也是用於把老的結構換成新的結構，一般用不到。

• upgrade_net_proto_binary.bin
• upgrade_net_proto_text.bin
• upgrade_solver_proto_text.bin