# Optical Character Recognition

OCR for Baidu competition (level one)
Environment requirements:
tensorflow 1.1.0, python 2.7

This work is based on model from Peiwen Wang whose excellent work (https://github.com/ypwhs/baiduyun_deeplearning_competition) is carried out on keras. Thanks to him sincerely.  

# Contents

* Training dataset

* Model

* Loss & accuracy

* Features learned by different level layers

* Weights and biases

* Gradient descending

------------------------------------------------------------------------------------------------------------------------------
# Training datａset
Training dataset has 100,000 pictures including characters from 0123456789+-*() with **different length** as pictures below show.The label format, for example, to the first training picture, is **(7+5)+4 16**. You can download dataset here https://pan.baidu.com/s/1geT4z9x , keep in mind that the label file is inside the unpacked folder.

 

# Model

The neural network includes conv network,RNN(GRU) and CTC (Connectionist Temporal Classifier) as picture below shows.

There are three conv modules,layer1,layer2,layer3, every module has two conv layers and a max_pool layer. [3,3] kernel and [1,1] strides are used behind every conv layers. [2,2] kernel and [2,2] strides are used by max_pool layer. What's more, **learning features** by different conv modules are outputted by tensorboard.

 
# Loss & accuracy
As we can see, after training about 2.5h, there is a sharp ｄrop of CTCloss, meanwhile the training and validation accuracy turn to be ｃlose to 1.

# Features learned by different layers

Within the same level conv layers,only some of them have learned effective features.  
With the layers going deeper, features learned by them become more and more abstract.

Layer1

Layer2

Layer3

Fully connected layer1


# Weights and biases distributions

# Gradient descending

Gradient descending performance along with iterations

Gradient variation distribution,with iteration growing , gradient becomes very small, as you can see, the black line is the lastest iteration step.