Caffe層系列:BatchNorm Layer
阿新 • • 發佈:2019-01-11
BatchNorm Layer 是對輸入進行歸一化,消除過大噪點,有助於網路收斂
首先我們先看一下 BatchNormParameter
message BatchNormParameter { // If false, accumulate global mean/variance values via a moving average. // If true, use those accumulated values instead of computing mean/variance across the batch. // 如果為真,則使用儲存的均值和方差,否則採用滑動平均計算新的均值和方差 // 該引數預設的時候,如果是測試階段則等價為真,如果是訓練階段則等價為假 optional bool use_global_stats = 1; // How much does the moving average decay each iteration? // 滑動平均的衰減係數,預設為0.999 optional float moving_average_fraction = 2 [default = .999]; // Small value to add to the variance estimate so that we don't divide by zero. // 分母附加值,防止除以方差時出現除0操作,預設為1e-5 optional float eps = 3 [default = 1e-5]; }
BatchNorm layer 在prototxt裡面的書寫:
layer {
name: "bn_conv1"
type: "BatchNorm"
bottom: "conv1"
top: "conv1"
batch_norm_param {
use_global_stats: true
}
}
例如在MobileNet中:
layer { name: "conv6_4/bn" type: "BatchNorm" bottom: "conv6_4" top: "conv6_4/bn" param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } param { lr_mult: 0 decay_mult: 0 } batch_norm_param { use_global_stats: true eps: 1e-5 } }