之前做分類的時候，Dropout 層一般加在全連線層 防止過擬合 提升模型泛化能力。而很少見到卷積層後接Drop out （原因主要是 卷積引數少，不易過擬合），今天找了些部落格，特此記錄。

You can imagine that if neurons are randomly dropped out of the network during training, that other neurons will have to step in and handle the representation required to make predictions for the missing neurons. This is believed to result in multiple independent internal representations being learned by the network.

The effect is that the network becomes less sensitive to the specific weights of neurons. This in turn results in a network that is capable of better generalization and is less likely to overfit the training data.

在cifar資料集上使用Dropout的例項：92.45% on CIFAR-10 in Torch
這裡面卷積層和全連線層都加了Dropout。But dropout values are usually < 0.5, e.g. 0.1, 0.2, 0.3 for the convolutional layers.

在附上提出Dropout的論文中的觀點：

from the Srivastava/Hinton dropout paper:

“The additional gain in performance obtained by adding dropout in the convolutional layers (3.02% to 2.55%) is worth noting. One may have presumed that since the convolutional layers don’t have a lot of parameters, overfitting is not a problem and therefore dropout would not have much effect. However, dropout in the lower layers still helps because it provides noisy inputs for the higher fully connected layers which prevents them from overfitting.”
They use 0.7 prob for conv drop out and 0.5 for fully connected.

這次實驗我是在輸入層後加入了Dropout層，感覺像是資料擴增，還不知道效果如何。