When establish a deep learning model like CNN, we should follow these steps below.

1.Sanity check your loss.

IF You use a softmax classifier, we expect the loss for random weights (with no regularization) to be about logC where C d/enotes the number of classes.

2. Gradient check

You should use a small set of training data or even a random dataset to make sure that the backward pass you implenments is correct. BY THE WAY, you not have to set the hidden layers’ dimension or the number of hidden layers too large.

3. Overfit a small dataset

In this step, you should randomly choose just a few training samples (say 100 or 200). Your basic model should have a high training accuracy and comparatively low validation accuracy.

4. Train the Net

In this step and following, you can choose which layers you’re going to stack and how much layers you need.

5. Tuning the hyperparameter

What you have to do now is tuning the hyperparameters like learning_rate,Batch_size to get best model that owns a great performance!