1. 程式人生 > >用DSOD實現行人檢測

用DSOD實現行人檢測

(一)系統版本:

  1. GPU型號:
# nvidia-smi
Thu Nov  1 01:29:23 2018       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.87                 Driver Version: 390.87                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  TITAN Xp            Off  | 00000000:06:00.0  On |                  N/A |
| 36%   59C    P2   161W / 250W |   4726MiB / 12195MiB |     51%      Default |
+-------------------------------+----------------------+----------------------+
|   1  TITAN Xp            Off  | 00000000:84:00.0 Off |                  N/A |
| 23%   23C    P8    16W / 250W |     12MiB / 12196MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
+-----------------------------------------------------------------------------+

         2.cuda與cudnn版本:

# cat /usr/local/cuda/version.txt
CUDA Version 8.0.44
# cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
#define CUDNN_MAJOR      5
#define CUDNN_MINOR      1
#define CUDNN_PATCHLEVEL 10
--
#define CUDNN_VERSION    (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)

#include "driver_types.h"

         3.系統版本

#cat /proc/version
Linux version 3.10.0-693.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Tue Aug 22 21:09:27 UTC 2017

(三)復現SSD論文實驗後面填坑。

(四)安裝DSOD

  1. 需要在SSD可以執行的基礎上新建檔案然後複製,具體過程:
Create a subfolder dsod under example/, add files DSOD300_pascal.py, DSOD300_pascal++.py, DSOD300_coco.py, score_DSOD300_pascal.py and DSOD300_detection_demo.py to the folder example/dsod/.

Create a subfolder grp_dsod under example/, add files GRP_DSOD320_pascal.py and score_GRP_DSOD320_pascal.py to the folder example/grp_dsod/.

Replace the file model_libs.py in the folder python/caffe/ with ours.

      3.訓練

Train a DSOD model on VOC 07+12:

python examples/dsod/DSOD300_pascal.py
Train a DSOD model on VOC 07++12:

python examples/dsod/DSOD300_pascal++.py
Train a DSOD model on COCO trainval:

python examples/dsod/DSOD300_coco.py

     4.問題

Check failed: mdb_status == 0 (2 vs. 0) No such file or directory

因為顯示卡記憶體小,有說可能是因為batch_size設定的過大的原因

原始碼:batch_size = 32

accum_batch_size = 128

/home/caffe-ssd/examples/dsod +273

修改為:batch_size =4# 32

accum_batch_size =16# 128

Check failed: error == cudaSuccess (10 vs. 0)  invalid device ordinal

這是由於GPU數量不匹配造成的,需要將solver.prototxt檔案中的device_id項改為自己的GPU塊數,一塊就是0,兩塊就是1,以此類推。

example/dsod/DSOD300_pascal.py +278    gpus = "0,1,2,3,4,5,6,7" 改為gpus = "0” 

(五)訓練行人檢測資料

  1. 在example下新建檔案dsod_train,將example/dsod中的檔案複製到dsod_train中。
  2. 在data下新建檔案dsod_trian,將VOC0712中的內容複製到dsod_train中。
# mkdir dsod_train
# cp -rf /home/caffe-ssd/data/VOC0712/* /home/caffe-ssd/data/dsod_train/

     3. 在data下新建檔案sample,在sample下新建dsod_train與mydatasets。

     4. 準備資料,行人資料是標準的VOC資料格式,傳輸資料到mydatasets中,有以下幾個方法:

  • 下載xftp,安裝好之後在xshell中開啟xftp(ctrl+Alt+F),可以實現本地電腦與伺服器之間的檔案互傳。
  • 安裝FileZilla軟體,它是免費開源的FTP軟體,網上教程很多。

  • 使用sz,rz。Linux/Unix同Windows進行ZModem檔案傳輸的命令列工具。(與直接拖檔案進入效果一樣)

  安裝方法:

#apt-get install lrzsz

從服務端傳送檔案到客戶端:sz後回車彈出選擇的檔案。 

  • 如果沒有安裝xftp,點選後選擇取消,會彈出一個頁面,在頁面中用put的方法也可以實現檔案傳輸,個人理解這就是沒介面的xftp。不會用可以看看help,裡面寫的很清楚。
#sftp:/home/caffe-ssd/data/sample/mydatasets> put Z:\深度學習\Annotations /home/caffe-ssd/data/sample/mydatasets/

       5. 準備好了資料,解壓後開始轉成LMDB的格式。首先修改data/dsod/creat_list.sh,程式碼如下:

#!/bin/bash

#root_dir=/home/caffe-ssd/data/VOCdevkit/
root_dir=/home/caffe-ssd/data/sample/
sub_dir=ImageSets/Main
bash_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
for dataset in trainval test
do
  dst_file=$bash_dir/$dataset.txt
  if [ -f $dst_file ]
  then
    rm -f $dst_file
  fi
  #for name in VOC2007 VOC2012
  for name in mydatasets
  do
   # if [[ $dataset == "test" && $name == "VOC2012" ]]
   # then
   #   continue
   # fi
    echo "Create list for $name $dataset..."
    dataset_file=$root_dir/$name/$sub_dir/$dataset.txt

    img_file=$bash_dir/$dataset"_img.txt"
    cp $dataset_file $img_file
    sed -i "s/^/$name\/JPEGImages\//g" $img_file
    sed -i "s/$/.jpg/g" $img_file

    label_file=$bash_dir/$dataset"_label.txt"
    cp $dataset_file $label_file
    sed -i "s/^/$name\/Annotations\//g" $label_file
    sed -i "s/$/.xml/g" $label_file

    paste -d' ' $img_file $label_file >> $dst_file

    rm -f $label_file
    rm -f $img_file
  done

  # Generate image name and size infomation.
  if [ $dataset == "test" ]
  then
    $bash_dir/../../build/tools/get_image_size $root_dir $dst_file $bash_dir/$dataset"_name_size.txt"
  fi

  # Shuffle trainval file.
  if [ $dataset == "trainval" ]
  then
    rand_file=$dst_file.random
    cat $dst_file | perl -MList::Util=shuffle -e 'print shuffle(<STDIN>);' > $rand_file
    mv $rand_file $dst_file
  fi
done
~                                                                                                                                                                                    
~                                                                                                                                                                                    
~                                                

 執行./create_list.sh 

/home/caffe-ssd/data/dsod_train
Create list for mydatasets trainval...
Create list for mydatasets test...
I1101 11:48:46.939064  3113 get_image_size.cpp:61] A total of 2171 images.
I1101 11:48:51.158903  3113 get_image_size.cpp:100] Processed 1000 files.
I1101 11:48:55.381808  3113 get_image_size.cpp:100] Processed 2000 files.
I1101 11:48:56.103267  3113 get_image_size.cpp:105] Processed 2171 files.

在data/dsod_train裡面會重新生成三個檔案(以前的不用刪,執行程式會直接覆蓋以前的) 

-rw-r--r--. 1 root root 140915 Nov  1 04:01 test.txt
-rw-r--r--. 1 root root  38978 Nov  1 04:02 test_name_size.txt
-rw-r--r--. 1 root root 563089 Nov  1 04:01 trainval.txt
[email protected]:/home/caffe-ssd/data/train_sample# vim test.txt 

sample/JPEGImages/03_245525.jpg sample/Annotations/03_245525.xml
sample/JPEGImages/01_77660.jpg sample/Annotations/01_77660.xml
sample/JPEGImages/02_211004.jpg sample/Annotations/02_211004.xml
sample/JPEGImages/02_468307.jpg sample/Annotations/02_468307.xml
sample/JPEGImages/03_831804.jpg sample/Annotations/03_831804.xml
sample/JPEGImages/04_796466.jpg sample/Annotations/04_796466.xml
sample/JPEGImages/03_2584905.jpg sample/Annotations/03_2584905.xml
sample/JPEGImages/03_260282.jpg sample/Annotations/03_260282.xml
sample/JPEGImages/03_2277306.jpg sample/Annotations/03_2277306.xml
[email protected]:/home/caffe-ssd/data/train_sample# vim trainval.txt 

sample/JPEGImages/03_1164425.jpg sample/Annotations/03_1164425.xml
sample/JPEGImages/04_41943.jpg sample/Annotations/04_41943.xml
sample/JPEGImages/01_277407.jpg sample/Annotations/01_277407.xml
sample/JPEGImages/01_223079.jpg sample/Annotations/01_223079.xml
sample/JPEGImages/01_69969.jpg sample/Annotations/01_69969.xml
sample/JPEGImages/01_371695.jpg sample/Annotations/01_371695.xml
sample/JPEGImages/03_123987.jpg sample/Annotations/03_123987.xml
sample/JPEGImages/02_185663.jpg sample/Annotations/02_185663.xml
sample/JPEGImages/04_31490.jpg sample/Annotations/04_31490.xml
sample/JPEGImages/01_208821.jpg sample/Annotations/01_208821.xml
[email protected]:/home/caffe-ssd/data/train_sample# vim test_name_size.txt 

03_245525 480 640
01_77660 480 640
02_211004 480 640
02_468307 480 640
03_831804 480 640
04_796466 480 640
03_2584905 480 640
03_260282 480 640

 修改caffe-ssd/data/train_sample/labelmap_voc.prototxt 除了background之外改成自己的類別,我的是兩類。

# vim labelmap_voc.prototxt 

item {
  name: "none_of_the_above"
  label: 0
  display_name: "background"
}
item {
  name: "headshoulder"
  label: 1
  display_name: "headshoulder"
}
item {
  name: "ignore"
  label: 2
  display_name: "ignore"
}

   6. 修改data/dsod_train/creat_data.sh  

ur_dir=$(cd $( dirname ${BASH_SOURCE[0]} ) && pwd )
root_dir=$cur_dir/../..

cd $root_dir

redo=1
data_root_dir="/home/caffe-ssd/data/sample"
dataset_name="dsod_train"
mapfile="$root_dir/data/$dataset_name/labelmap_voc.prototxt"
anno_type="detection"
db="lmdb"
min_dim=0
max_dim=0
width=0
height=0

extra_cmd="--encode-type=jpg --encoded"
if [ $redo ]
then
  extra_cmd="$extra_cmd --redo"
fi
for subset in test trainval
do
  python $root_dir/scripts/create_annoset.py --anno-type=$anno_type --label-map-file=$mapfile --min-dim=$min_dim --max-dim=$max_dim --resize-width=$width --resize-height=$height --check-label $extra_cmd $data_root_dir $root_dir/data/$dataset_name/$subset.txt $data_root_dir/$dataset_name/$db/$dataset_name"_"$subset"_"$db examples/$dataset_name
done

修改完後執行./creat_data.sh 。在example/train_sample中生成兩個lmdb連結;在/data/sample/dsod_train/lmdb會生成轉化好的lmdb檔案。

   7. 修改example/dsod_train/DSOD300_pascal.py檔案

   8. 訓練  python ./example/dsod_train/DSOD300_pascal.py

明天記錄結果和測試效果。