3-2MNIST資料集分類-簡單版（及優化到98%以上）
阿新 • • 發佈：2018-11-27
優化方案:
1、batch_size = 50 ---> 100 (不會有很大影響只要合理)
2、將學習率劃為變數
lr = tf.Variable(0.001,dtype=float32)
sess.run(tf.assign(lr,0.001*(0.95**epoch)))
學習率要不斷降低，剛開始小球很快，後面很慢。再學習率很低的時候，步長大一些，後面穩定後，慢慢找最優解。
3、增加迭代計算次數
迭代增加到51次
4、改善網路結構
#建立一個簡單的神經網路 W1 = tf.Variable(tf.truncated_normal([784,500],stddev=0.1)) b1 = tf.Variable(tf.zeros([500])+ 0.1) L1 = tf.nn.tanh(tf.matmul(x,W1)+b1) L1_drop = tf.nn.dropout(L1,keep_prob) W2 = tf.Variable(tf.truncated_normal([500,300],stddev=0.1)) b2 = tf.Variable(tf.zeros([300])+ 0.1) L2 = tf.nn.tanh(tf.matmul(L1_drop,W2)+b2) L2_drop = tf.nn.dropout(L2,keep_prob) W3 = tf.Variable(tf.truncated_normal([300,10],stddev=0.1)) b3 = tf.Variable(tf.zeros([10])+ 0.1) prediction = tf.nn.softmax(tf.matmul(L2_drop,W3)+b3)
結果如下:
Extracting MNIST_data/train-images-idx3-ubyte.gz Extracting MNIST_data/train-labels-idx1-ubyte.gz Extracting MNIST_data/t10k-images-idx3-ubyte.gz Extracting MNIST_data/t10k-labels-idx1-ubyte.gz WARNING:tensorflow:From /home/jason.lei/liangde_project/anaconda3/envs/tensorflow_02/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py:107: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Use `tf.global_variables_initializer` instead. Iter 0,Testing Accuary0.9476,Train Accuary0.95176363,Learning Rate=0.001 Iter 1,Testing Accuary0.9636,Train Accuary0.9700364,Learning Rate=0.00095 Iter 2,Testing Accuary0.9704,Train Accuary0.9786,Learning Rate=0.0009025 Iter 3,Testing Accuary0.9711,Train Accuary0.9818909,Learning Rate=0.000857375 Iter 4,Testing Accuary0.9725,Train Accuary0.9848727,Learning Rate=0.00081450626 Iter 5,Testing Accuary0.9764,Train Accuary0.9878,Learning Rate=0.0007737809 Iter 6,Testing Accuary0.9761,Train Accuary0.9898,Learning Rate=0.0007350919 Iter 7,Testing Accuary0.9749,Train Accuary0.9902909,Learning Rate=0.0006983373 Iter 8,Testing Accuary0.9781,Train Accuary0.9924909,Learning Rate=0.0006634204 Iter 9,Testing Accuary0.9771,Train Accuary0.99305457,Learning Rate=0.0006302494 Iter 10,Testing Accuary0.979,Train Accuary0.9939273,Learning Rate=0.0005987369 Iter 11,Testing Accuary0.9786,Train Accuary0.99474543,Learning Rate=0.0005688001 Iter 12,Testing Accuary0.9788,Train Accuary0.99474543,Learning Rate=0.0005403601 Iter 13,Testing Accuary0.9787,Train Accuary0.9952,Learning Rate=0.0005133421 Iter 14,Testing Accuary0.9805,Train Accuary0.9954909,Learning Rate=0.000487675 Iter 15,Testing Accuary0.9798,Train Accuary0.9958182,Learning Rate=0.00046329122 Iter 16,Testing Accuary0.9788,Train Accuary0.99561816,Learning Rate=0.00044012666 Iter 17,Testing Accuary0.9808,Train Accuary0.9963091,Learning Rate=0.00041812033 Iter 18,Testing Accuary0.981,Train Accuary0.99636364,Learning Rate=0.00039721432 Iter 19,Testing Accuary0.9808,Train Accuary0.99654543,Learning Rate=0.0003773536 Iter 20,Testing Accuary0.9811,Train Accuary0.9964,Learning Rate=0.00035848594 Iter 21,Testing Accuary0.9806,Train Accuary0.9967273,Learning Rate=0.00034056162 Iter 22,Testing Accuary0.9809,Train Accuary0.9968909,Learning Rate=0.00032353355 Iter 23,Testing Accuary0.9819,Train Accuary0.9969818,Learning Rate=0.00030735688 Iter 24,Testing Accuary0.9818,Train Accuary0.99701816,Learning Rate=0.000291989 Iter 25,Testing Accuary0.9819,Train Accuary0.9969818,Learning Rate=0.00027738957 Iter 26,Testing Accuary0.9814,Train Accuary0.9971455,Learning Rate=0.0002635201 Iter 27,Testing Accuary0.9819,Train Accuary0.99718183,Learning Rate=0.00025034408 Iter 28,Testing Accuary0.9826,Train Accuary0.9972,Learning Rate=0.00023782688 Iter 29,Testing Accuary0.9812,Train Accuary0.99716365,Learning Rate=0.00022593554 Iter 30,Testing Accuary0.9824,Train Accuary0.99718183,Learning Rate=0.00021463877 Iter 31,Testing Accuary0.9833,Train Accuary0.99727273,Learning Rate=0.00020390682 Iter 32,Testing Accuary0.9809,Train Accuary0.99718183,Learning Rate=0.00019371149 Iter 33,Testing Accuary0.9831,Train Accuary0.9974,Learning Rate=0.0001840259 Iter 34,Testing Accuary0.983,Train Accuary0.9974727,Learning Rate=0.00017482461 Iter 35,Testing Accuary0.9828,Train Accuary0.9975273,Learning Rate=0.00016608338 Iter 36,Testing Accuary0.9826,Train Accuary0.9976182,Learning Rate=0.00015777921 Iter 37,Testing Accuary0.9823,Train Accuary0.9976182,Learning Rate=0.00014989026 Iter 38,Testing Accuary0.9823,Train Accuary0.9976364,Learning Rate=0.00014239574 Iter 39,Testing Accuary0.9823,Train Accuary0.99765456,Learning Rate=0.00013527596 Iter 40,Testing Accuary0.9827,Train Accuary0.99767274,Learning Rate=0.00012851215 Iter 41,Testing Accuary0.9821,Train Accuary0.9977091,Learning Rate=0.00012208655 Iter 42,Testing Accuary0.9824,Train Accuary0.9977091,Learning Rate=0.00011598222 Iter 43,Testing Accuary0.9821,Train Accuary0.9977273,Learning Rate=0.00011018311 Iter 44,Testing Accuary0.9825,Train Accuary0.99776363,Learning Rate=0.000104673956 Iter 45,Testing Accuary0.9827,Train Accuary0.99776363,Learning Rate=9.944026e-05 Iter 46,Testing Accuary0.9826,Train Accuary0.9977818,Learning Rate=9.446825e-05 Iter 47,Testing Accuary0.9828,Train Accuary0.9978,Learning Rate=8.974483e-05 Iter 48,Testing Accuary0.9833,Train Accuary0.9978182,Learning Rate=8.525759e-05 Iter 49,Testing Accuary0.9825,Train Accuary0.9978182,Learning Rate=8.099471e-05 Iter 50,Testing Accuary0.9828,Train Accuary0.9978182,Learning Rate=7.6944976e-05