官網例項詳解4.30(mnist_siamese.py)-keras學習筆記四
阿新 • • 發佈:2019-02-07
基於MNIST資料集上從一對數字中訓練一個 Siamese MLP。
Siamese ,連體的,相似的。
Siamese Net,孿生網路、連體網路
MLP,多層感知機,(多個隱藏層的全連線的神經網路)
程式碼註釋
'''Trains a Siamese MLP on pairs of digits from the MNIST dataset. 基於MNIST資料集上從一對數字中訓練一個 Siamese MLP。 Siamese網路是一種相似性度量方法,當類別數多,但每個類別的樣本數量少的情況下可用於類別的識別、分類等。 It follows Hadsell-et-al.'06 [1] by computing the Euclidean distance on the output of the shared network and by optimizing the contrastive loss (see paper for mode details). 通過計算共享網路的輸出上的歐幾里德距離並通過優化對比損耗(見模式細節論文)來跟蹤Ha售貨等。'06(1)。 # References 參考 - Dimensionality Reduction by Learning an Invariant Mapping 基於不變對映學習的降維 http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf Gets to 97.2% test accuracy after 20 epochs. 20週期後97.2%測試準確率 2 seconds per epoch on a Titan X Maxwell GPU 2秒/週期,基於Titan X Maxwell GPU (執行硬體) ''' from __future__ import absolute_import from __future__ import print_function import numpy as np import random from keras.datasets import mnist from keras.models import Model from keras.layers import Input, Flatten, Dense, Dropout, Lambda from keras.optimizers import RMSprop from keras import backend as K num_classes = 10 epochs = 20 def euclidean_distance(vects): x, y = vects return K.sqrt(K.maximum(K.sum(K.square(x - y), axis=1, keepdims=True), K.epsilon())) def eucl_dist_output_shape(shapes): shape1, shape2 = shapes return (shape1[0], 1) def contrastive_loss(y_true, y_pred): '''Contrastive loss from Hadsell-et-al.'06 Hadsell-et-al.'06 的對比損失 http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf 人工智慧專家: Raia Hadsell http://raiahadsell.com/index.html Sumit Chopra https://in.linkedin.com/in/schoprasumit Yann LeCun http://yann.lecun.com/ ''' margin = 1 return K.mean(y_true * K.square(y_pred) + (1 - y_true) * K.square(K.maximum(margin - y_pred, 0))) def create_pairs(x, digit_indices): '''Positive and negative pair creation. 正面和負面的創作。 Alternates between positive and negative pairs. 在正和負對之間交替。 ''' pairs = [] labels = [] n = min([len(digit_indices[d]) for d in range(num_classes)]) - 1 for d in range(num_classes): for i in range(n): z1, z2 = digit_indices[d][i], digit_indices[d][i + 1] pairs += [[x[z1], x[z2]]] inc = random.randrange(1, num_classes) dn = (d + inc) % num_classes z1, z2 = digit_indices[d][i], digit_indices[dn][i] pairs += [[x[z1], x[z2]]] labels += [1, 0] return np.array(pairs), np.array(labels) def create_base_network(input_shape): '''Base network to be shared (eq. to feature extraction). 共享的基本網路(相當於特徵提取)。 ''' input = Input(shape=input_shape) x = Flatten()(input) x = Dense(128, activation='relu')(x) x = Dropout(0.1)(x) x = Dense(128, activation='relu')(x) x = Dropout(0.1)(x) x = Dense(128, activation='relu')(x) return Model(input, x) def compute_accuracy(y_true, y_pred): '''Compute classification accuracy with a fixed threshold on distances. 用固定的閾值計算距離的分類精度。 ''' pred = y_pred.ravel() < 0.5 return np.mean(pred == y_true) def accuracy(y_true, y_pred): '''Compute classification accuracy with a fixed threshold on distances. 用固定的閾值計算距離的分類精度。 ''' return K.mean(K.equal(y_true, K.cast(y_pred < 0.5, y_true.dtype))) # the data, shuffled and split between train and test sets # 用於訓練和測試的資料集,經過了篩選(清洗、資料樣本順序打亂)和劃分(劃分為訓練和測試集) (x_train, y_train), (x_test, y_test) = mnist.load_data() x_train = x_train.astype('float32') x_test = x_test.astype('float32') x_train /= 255 x_test /= 255 input_shape = x_train.shape[1:] # create training+test positive and negative pairs # 建立訓練+測試正負兩對 digit_indices = [np.where(y_train == i)[0] for i in range(num_classes)] tr_pairs, tr_y = create_pairs(x_train, digit_indices) digit_indices = [np.where(y_test == i)[0] for i in range(num_classes)] te_pairs, te_y = create_pairs(x_test, digit_indices) # network definition # 網路定義 base_network = create_base_network(input_shape) input_a = Input(shape=input_shape) input_b = Input(shape=input_shape) # because we re-use the same instance `base_network`, # the weights of the network # will be shared across the two branches # 因為我們重新使用同一個例項“base_network”,網路的權重將在兩個分支之間共享。 processed_a = base_network(input_a) processed_b = base_network(input_b) distance = Lambda(euclidean_distance, output_shape=eucl_dist_output_shape)([processed_a, processed_b]) model = Model([input_a, input_b], distance) # train # 訓練 rms = RMSprop() model.compile(loss=contrastive_loss, optimizer=rms, metrics=[accuracy]) model.fit([tr_pairs[:, 0], tr_pairs[:, 1]], tr_y, batch_size=128, epochs=epochs, validation_data=([te_pairs[:, 0], te_pairs[:, 1]], te_y)) # compute final accuracy on training and test sets # 計算訓練和測試集的最終準確率 y_pred = model.predict([tr_pairs[:, 0], tr_pairs[:, 1]]) tr_acc = compute_accuracy(tr_y, y_pred) y_pred = model.predict([te_pairs[:, 0], te_pairs[:, 1]]) te_acc = compute_accuracy(te_y, y_pred) print('* Accuracy on training set: %0.2f%%' % (100 * tr_acc)) print('* Accuracy on test set: %0.2f%%' % (100 * te_acc))
程式碼執行
C:\ProgramData\Anaconda3\python.exe E:/keras-master/examples/mnist_siamese.py
Using TensorFlow backend.
Train on 108400 samples, validate on 17820 samples
Epoch 1/20
128/108400 [..............................] - ETA: 19:18 - loss: 1.1304 - accuracy: 0.4531
640/108400 [..............................] - ETA: 4:00 - loss: 0.4385 - accuracy: 0.5234
1024/108400 [..............................] - ETA: 2:36 - loss: 0.3660 - accuracy: 0.5322
1408/108400 [..............................] - ETA: 1:57 - loss: 0.3319 - accuracy: 0.5405
1792/108400 [..............................] - ETA: 1:35 - loss: 0.3121 - accuracy: 0.5474
2176/108400 [..............................] - ETA: 1:20 - loss: 0.2967 - accuracy: 0.5588
2688/108400 [..............................] - ETA: 1:07 - loss: 0.2839 - accuracy: 0.5666
2944/108400 [..............................] - ETA: 1:07 - loss: 0.2796 - accuracy: 0.5703
3328/108400 [..............................] - ETA: 1:01 - loss: 0.2725 - accuracy: 0.5802
3840/108400 [>.............................] - ETA: 54s - loss: 0.2658 - accuracy: 0.5859
4352/108400 [>.............................] - ETA: 49s - loss: 0.2585 - accuracy: 0.5974
4736/108400 [>.............................] - ETA: 46s - loss: 0.2540 - accuracy: 0.6041
5248/108400 [>.............................] - ETA: 42s - loss: 0.2479 - accuracy: 0.6151
5632/108400 [>.............................] - ETA: 40s - loss: 0.2438 - accuracy: 0.6239
6016/108400 [>.............................] - ETA: 38s - loss: 0.2400 - accuracy: 0.6303
6528/108400 [>.............................] - ETA: 36s - loss: 0.2360 - accuracy: 0.6374
7040/108400 [>.............................] - ETA: 34s - loss: 0.2321 - accuracy: 0.6440
7680/108400 [=>............................] - ETA: 31s - loss: 0.2278 - accuracy: 0.6530
8192/108400 [=>............................] - ETA: 30s - loss: 0.2248 - accuracy: 0.6592
8704/108400 [=>............................] - ETA: 29s - loss: 0.2217 - accuracy: 0.6629
9344/108400 [=>............................] - ETA: 27s - loss: 0.2176 - accuracy: 0.6728
9984/108400 [=>............................] - ETA: 26s - loss: 0.2139 - accuracy: 0.6801
10496/108400 [=>............................] - ETA: 25s - loss: 0.2114 - accuracy: 0.6855
11008/108400 [==>...........................] - ETA: 24s - loss: 0.2082 - accuracy: 0.6920
11392/108400 [==>...........................] - ETA: 24s - loss: 0.2064 - accuracy: 0.6951
11776/108400 [==>...........................] - ETA: 23s - loss: 0.2043 - accuracy: 0.6994
12160/108400 [==>...........................] - ETA: 23s - loss: 0.2022 - accuracy: 0.7034
12672/108400 [==>...........................] - ETA: 22s - loss: 0.2000 - accuracy: 0.7080
13184/108400 [==>...........................] - ETA: 22s - loss: 0.1977 - accuracy: 0.7125
13696/108400 [==>...........................] - ETA: 21s - loss: 0.1954 - accuracy: 0.7169
14080/108400 [==>...........................] - ETA: 21s - loss: 0.1941 - accuracy: 0.7188
14592/108400 [===>..........................] - ETA: 20s - loss: 0.1919 - accuracy: 0.7229
15104/108400 [===>..........................] - ETA: 20s - loss: 0.1894 - accuracy: 0.7274
15616/108400 [===>..........................] - ETA: 19s - loss: 0.1878 - accuracy: 0.7301
16000/108400 [===>..........................] - ETA: 19s - loss: 0.1863 - accuracy: 0.7328
16384/108400 [===>..........................] - ETA: 19s - loss: 0.1853 - accuracy: 0.7350
16768/108400 [===>..........................] - ETA: 19s - loss: 0.1837 - accuracy: 0.7380
17024/108400 [===>..........................] - ETA: 19s - loss: 0.1829 - accuracy: 0.7399
17408/108400 [===>..........................] - ETA: 19s - loss: 0.1818 - accuracy: 0.7421
17920/108400 [===>..........................] - ETA: 18s - loss: 0.1801 - accuracy: 0.7457
18432/108400 [====>.........................] - ETA: 18s - loss: 0.1783 - accuracy: 0.7491
18944/108400 [====>.........................] - ETA: 18s - loss: 0.1770 - accuracy: 0.7516
19456/108400 [====>.........................] - ETA: 17s - loss: 0.1757 - accuracy: 0.7539
19968/108400 [====>.........................] - ETA: 17s - loss: 0.1745 - accuracy: 0.7560
20480/108400 [====>.........................] - ETA: 17s - loss: 0.1733 - accuracy: 0.7580
20992/108400 [====>.........................] - ETA: 16s - loss: 0.1719 - accuracy: 0.7604
21504/108400 [====>.........................] - ETA: 16s - loss: 0.1705 - accuracy: 0.7630
22016/108400 [=====>........................] - ETA: 16s - loss: 0.1693 - accuracy: 0.7651
22528/108400 [=====>........................] - ETA: 16s - loss: 0.1683 - accuracy: 0.7667
23040/108400 [=====>........................] - ETA: 15s - loss: 0.1672 - accuracy: 0.7686
23552/108400 [=====>........................] - ETA: 15s - loss: 0.1662 - accuracy: 0.7704
24064/108400 [=====>........................] - ETA: 15s - loss: 0.1652 - accuracy: 0.7724
24576/108400 [=====>........................] - ETA: 15s - loss: 0.1641 - accuracy: 0.7741
24960/108400 [=====>........................] - ETA: 15s - loss: 0.1634 - accuracy: 0.7754
25472/108400 [======>.......................] - ETA: 15s - loss: 0.1624 - accuracy: 0.7773
25984/108400 [======>.......................] - ETA: 15s - loss: 0.1614 - accuracy: 0.7791
26496/108400 [======>.......................] - ETA: 14s - loss: 0.1607 - accuracy: 0.7805
26880/108400 [======>.......................] - ETA: 14s - loss: 0.1600 - accuracy: 0.7818
27136/108400 [======>.......................] - ETA: 14s - loss: 0.1596 - accuracy: 0.7825
27520/108400 [======>.......................] - ETA: 14s - loss: 0.1589 - accuracy: 0.7836
27904/108400 [======>.......................] - ETA: 14s - loss: 0.1582 - accuracy: 0.7851
28416/108400 [======>.......................] - ETA: 14s - loss: 0.1573 - accuracy: 0.7866
28544/108400 [======>.......................] - ETA: 14s - loss: 0.1571 - accuracy: 0.7871
28928/108400 [=======>......................] - ETA: 14s - loss: 0.1564 - accuracy: 0.7882
29440/108400 [=======>......................] - ETA: 14s - loss: 0.1554 - accuracy: 0.7898
29952/108400 [=======>......................] - ETA: 14s - loss: 0.1546 - accuracy: 0.7911
30592/108400 [=======>......................] - ETA: 13s - loss: 0.1535 - accuracy: 0.7931
31232/108400 [=======>......................] - ETA: 13s - loss: 0.1524 - accuracy: 0.7951
31744/108400 [=======>......................] - ETA: 13s - loss: 0.1515 - accuracy: 0.7966
32384/108400 [=======>......................] - ETA: 13s - loss: 0.1505 - accuracy: 0.7985
32896/108400 [========>.....................] - ETA: 13s - loss: 0.1498 - accuracy: 0.7993
33408/108400 [========>.....................] - ETA: 12s - loss: 0.1488 - accuracy: 0.8009
34048/108400 [========>.....................] - ETA: 12s - loss: 0.1479 - accuracy: 0.8027
34688/108400 [========>.....................] - ETA: 12s - loss: 0.1469 - accuracy: 0.8044
35200/108400 [========>.....................] - ETA: 12s - loss: 0.1462 - accuracy: 0.8054
35712/108400 [========>.....................] - ETA: 12s - loss: 0.1456 - accuracy: 0.8065
36224/108400 [=========>....................] - ETA: 11s - loss: 0.1450 - accuracy: 0.8076
36736/108400 [=========>....................] - ETA: 11s - loss: 0.1443 - accuracy: 0.8091
37120/108400 [=========>....................] - ETA: 11s - loss: 0.1437 - accuracy: 0.8101
37504/108400 [=========>....................] - ETA: 11s - loss: 0.1432 - accuracy: 0.8112
38016/108400 [=========>....................] - ETA: 11s - loss: 0.1425 - accuracy: 0.8123
38400/108400 [=========>....................] - ETA: 11s - loss: 0.1419 - accuracy: 0.8133
38656/108400 [=========>....................] - ETA: 11s - loss: 0.1416 - accuracy: 0.8137
39040/108400 [=========>....................] - ETA: 11s - loss: 0.1410 - accuracy: 0.8147
39424/108400 [=========>....................] - ETA: 11s - loss: 0.1406 - accuracy: 0.8156
39808/108400 [==========>...................] - ETA: 11s - loss: 0.1401 - accuracy: 0.8163
40192/108400 [==========>...................] - ETA: 11s - loss: 0.1396 - accuracy: 0.8172
40704/108400 [==========>...................] - ETA: 11s - loss: 0.1390 - accuracy: 0.8182
41216/108400 [==========>...................] - ETA: 10s - loss: 0.1383 - accuracy: 0.8193
41728/108400 [==========>...................] - ETA: 10s - loss: 0.1378 - accuracy: 0.8202
42112/108400 [==========>...................] - ETA: 10s - loss: 0.1372 - accuracy: 0.8211
42368/108400 [==========>...................] - ETA: 10s - loss: 0.1370 - accuracy: 0.8216
42624/108400 [==========>...................] - ETA: 10s - loss: 0.1367 - accuracy: 0.8221
43008/108400 [==========>...................] - ETA: 10s - loss: 0.1363 - accuracy: 0.8228
43136/108400 [==========>...................] - ETA: 10s - loss: 0.1362 - accuracy: 0.8230
43520/108400 [===========>..................] - ETA: 10s - loss: 0.1356 - accuracy: 0.8238
44032/108400 [===========>..................] - ETA: 10s - loss: 0.1350 - accuracy: 0.8250
44544/108400 [===========>..................] - ETA: 10s - loss: 0.1345 - accuracy: 0.8258
45056/108400 [===========>..................] - ETA: 10s - loss: 0.1341 - accuracy: 0.8263
45568/108400 [===========>..................] - ETA: 10s - loss: 0.1336 - accuracy: 0.8272
45952/108400 [===========>..................] - ETA: 10s - loss: 0.1332 - accuracy: 0.8279
46208/108400 [===========>..................] - ETA: 10s - loss: 0.1329 - accuracy: 0.8284
46592/108400 [===========>..................] - ETA: 9s - loss: 0.1325 - accuracy: 0.8291
46976/108400 [============>.................] - ETA: 9s - loss: 0.1321 - accuracy: 0.8297
47488/108400 [============>.................] - ETA: 9s - loss: 0.1317 - accuracy: 0.8303
47872/108400 [============>.................] - ETA: 9s - loss: 0.1313 - accuracy: 0.8309
48256/108400 [============>.................] - ETA: 9s - loss: 0.1310 - accuracy: 0.8314
48640/108400 [============>.................] - ETA: 9s - loss: 0.1306 - accuracy: 0.8321
49024/108400 [============>.................] - ETA: 9s - loss: 0.1303 - accuracy: 0.8324
49408/108400 [============>.................] - ETA: 9s - loss: 0.1299 - accuracy: 0.8331
49792/108400 [============>.................] - ETA: 9s - loss: 0.1295 - accuracy: 0.8337
50176/108400 [============>.................] - ETA: 9s - loss: 0.1291 - accuracy: 0.8345
50560/108400 [============>.................] - ETA: 9s - loss: 0.1286 - accuracy: 0.8351
50944/108400 [=============>................] - ETA: 9s - loss: 0.1284 - accuracy: 0.8356
51456/108400 [=============>................] - ETA: 9s - loss: 0.1279 - accuracy: 0.8362
51840/108400 [=============>................] - ETA: 9s - loss: 0.1276 - accuracy: 0.8368
52224/108400 [=============>................] - ETA: 8s - loss: 0.1272 - accuracy: 0.8374
52608/108400 [=============>................] - ETA: 8s - loss: 0.1269 - accuracy: 0.8378
52992/108400 [=============>................] - ETA: 8s - loss: 0.1266 - accuracy: 0.8384
53504/108400 [=============>................] - ETA: 8s - loss: 0.1262 - accuracy: 0.8392
54016/108400 [=============>................] - ETA: 8s - loss: 0.1257 - accuracy: 0.8399
54528/108400 [==============>...............] - ETA: 8s - loss: 0.1252 - accuracy: 0.8408
55040/108400 [==============>...............] - ETA: 8s - loss: 0.1248 - accuracy: 0.8415
55680/108400 [==============>...............] - ETA: 8s - loss: 0.1242 - accuracy: 0.8425
56192/108400 [==============>...............] - ETA: 8s - loss: 0.1238 - accuracy: 0.8432
56704/108400 [==============>...............] - ETA: 8s - loss: 0.1234 - accuracy: 0.8437
57216/108400 [==============>...............] - ETA: 7s - loss: 0.1230 - accuracy: 0.8444
57728/108400 [==============>...............] - ETA: 7s - loss: 0.1227 - accuracy: 0.8448
58240/108400 [===============>..............] - ETA: 7s - loss: 0.1223 - accuracy: 0.8455
58880/108400 [===============>..............] - ETA: 7s - loss: 0.1218 - accuracy: 0.8464
59520/108400 [===============>..............] - ETA: 7s - loss: 0.1213 - accuracy: 0.8473
60032/108400 [===============>..............] - ETA: 7s - loss: 0.1210 - accuracy: 0.8479
60544/108400 [===============>..............] - ETA: 7s - loss: 0.1206 - accuracy: 0.8484
61184/108400 [===============>..............] - ETA: 7s - loss: 0.1200 - accuracy: 0.8494
61696/108400 [================>.............] - ETA: 7s - loss: 0.1196 - accuracy: 0.8501
62080/108400 [================>.............] - ETA: 7s - loss: 0.1193 - accuracy: 0.8506
62592/108400 [================>.............] - ETA: 6s - loss: 0.1189 - accuracy: 0.8511
63232/108400 [================>.............] - ETA: 6s - loss: 0.1184 - accuracy: 0.8520
63872/108400 [================>.............] - ETA: 6s - loss: 0.1180 - accuracy: 0.8527
64384/108400 [================>.............] - ETA: 6s - loss: 0.1177 - accuracy: 0.8532
64896/108400 [================>.............] - ETA: 6s - loss: 0.1172 - accuracy: 0.8539
65408/108400 [=================>............] - ETA: 6s - loss: 0.1169 - accuracy: 0.8544
65920/108400 [=================>............] - ETA: 6s - loss: 0.1165 - accuracy: 0.8549
66560/108400 [=================>............] - ETA: 6s - loss: 0.1161 - accuracy: 0.8556
66944/108400 [=================>............] - ETA: 6s - loss: 0.1159 - accuracy: 0.8559
67328/108400 [=================>............] - ETA: 6s - loss: 0.1157 - accuracy: 0.8563
67840/108400 [=================>............] - ETA: 6s - loss: 0.1154 - accuracy: 0.8568
68352/108400 [=================>............] - ETA: 5s - loss: 0.1150 - accuracy: 0.8573
68864/108400 [==================>...........] - ETA: 5s - loss: 0.1147 - accuracy: 0.8577
69376/108400 [==================>...........] - ETA: 5s - loss: 0.1143 - accuracy: 0.8583
69888/108400 [==================>...........] - ETA: 5s - loss: 0.1140 - accuracy: 0.8588
70400/108400 [==================>...........] - ETA: 5s - loss: 0.1137 - accuracy: 0.8592
70784/108400 [==================>...........] - ETA: 5s - loss: 0.1134 - accuracy: 0.8597
71296/108400 [==================>...........] - ETA: 5s - loss: 0.1132 - accuracy: 0.8601
71936/108400 [==================>...........] - ETA: 5s - loss: 0.1128 - accuracy: 0.8608
72576/108400 [===================>..........] - ETA: 5s - loss: 0.1124 - accuracy: 0.8615
73216/108400 [===================>..........] - ETA: 5s - loss: 0.1120 - accuracy: 0.8621
73856/108400 [===================>..........] - ETA: 4s - loss: 0.1116 - accuracy: 0.8626
74496/108400 [===================>..........] - ETA: 4s - loss: 0.1112 - accuracy: 0.8633
75136/108400 [===================>..........] - ETA: 4s - loss: 0.1108 - accuracy: 0.8638
75648/108400 [===================>..........] - ETA: 4s - loss: 0.1105 - accuracy: 0.8644
76160/108400 [====================>.........] - ETA: 4s - loss: 0.1101 - accuracy: 0.8650
76800/108400 [====================>.........] - ETA: 4s - loss: 0.1098 - accuracy: 0.8655
77312/108400 [====================>.........] - ETA: 4s - loss: 0.1095 - accuracy: 0.8658
77952/108400 [====================>.........] - ETA: 4s - loss: 0.1091 - accuracy: 0.8663
78592/108400 [====================>.........] - ETA: 4s - loss: 0.1088 - accuracy: 0.8669
79232/108400 [====================>.........] - ETA: 4s - loss: 0.1085 - accuracy: 0.8673
79872/108400 [=====================>........] - ETA: 4s - loss: 0.1081 - accuracy: 0.8680
80384/108400 [=====================>........] - ETA: 3s - loss: 0.1078 - accuracy: 0.8685
81024/108400 [=====================>........] - ETA: 3s - loss: 0.1074 - accuracy: 0.8691
81664/108400 [=====================>........] - ETA: 3s - loss: 0.1071 - accuracy: 0.8696
82304/108400 [=====================>........] - ETA: 3s - loss: 0.1066 - accuracy: 0.8702
82944/108400 [=====================>........] - ETA: 3s - loss: 0.1062 - accuracy: 0.8708
83584/108400 [======================>.......] - ETA: 3s - loss: 0.1059 - accuracy: 0.8714
84224/108400 [======================>.......] - ETA: 3s - loss: 0.1055 - accuracy: 0.8719
84864/108400 [======================>.......] - ETA: 3s - loss: 0.1051 - accuracy: 0.8724
85504/108400 [======================>.......] - ETA: 3s - loss: 0.1048 - accuracy: 0.8729
86144/108400 [======================>.......] - ETA: 3s - loss: 0.1045 - accuracy: 0.8734
86784/108400 [=======================>......] - ETA: 2s - loss: 0.1042 - accuracy: 0.8738
87424/108400 [=======================>......] - ETA: 2s - loss: 0.1039 - accuracy: 0.8743
87808/108400 [=======================>......] - ETA: 2s - loss: 0.1038 - accuracy: 0.8746
88320/108400 [=======================>......] - ETA: 2s - loss: 0.1035 - accuracy: 0.8750
88960/108400 [=======================>......] - ETA: 2s - loss: 0.1031 - accuracy: 0.8754
89600/108400 [=======================>......] - ETA: 2s - loss: 0.1028 - accuracy: 0.8759
90112/108400 [=======================>......] - ETA: 2s - loss: 0.1026 - accuracy: 0.8763
90624/108400 [========================>.....] - ETA: 2s - loss: 0.1023 - accuracy: 0.8767
91264/108400 [========================>.....] - ETA: 2s - loss: 0.1019 - accuracy: 0.8772
91904/108400 [========================>.....] - ETA: 2s - loss: 0.1016 - accuracy: 0.8776
92416/108400 [========================>.....] - ETA: 2s - loss: 0.1014 - accuracy: 0.8780
92928/108400 [========================>.....] - ETA: 2s - loss: 0.1012 - accuracy: 0.8782
93440/108400 [========================>.....] - ETA: 2s - loss: 0.1010 - accuracy: 0.8786
93952/108400 [=========================>....] - ETA: 1s - loss: 0.1007 - accuracy: 0.8790
94464/108400 [=========================>....] - ETA: 1s - loss: 0.1005 - accuracy: 0.8794
94848/108400 [=========================>....] - ETA: 1s - loss: 0.1003 - accuracy: 0.8796
95360/108400 [=========================>....] - ETA: 1s - loss: 0.1000 - accuracy: 0.8800
95744/108400 [=========================>....] - ETA: 1s - loss: 0.0998 - accuracy: 0.8803
96128/108400 [=========================>....] - ETA: 1s - loss: 0.0997 - accuracy: 0.8805
96512/108400 [=========================>....] - ETA: 1s - loss: 0.0995 - accuracy: 0.8807
96896/108400 [=========================>....] - ETA: 1s - loss: 0.0994 - accuracy: 0.8809
97024/108400 [=========================>....] - ETA: 1s - loss: 0.0993 - accuracy: 0.8810
97152/108400 [=========================>....] - ETA: 1s - loss: 0.0992 - accuracy: 0.8811
97408/108400 [=========================>....] - ETA: 1s - loss: 0.0992 - accuracy: 0.8812
97792/108400 [==========================>...] - ETA: 1s - loss: 0.0990 - accuracy: 0.8814
98304/108400 [==========================>...] - ETA: 1s - loss: 0.0988 - accuracy: 0.8817
98944/108400 [==========================>...] - ETA: 1s - loss: 0.0985 - accuracy: 0.8820
99584/108400 [==========================>...] - ETA: 1s - loss: 0.0983 - accuracy: 0.8824
100224/108400 [==========================>...] - ETA: 1s - loss: 0.0980 - accuracy: 0.8828
100864/108400 [==========================>...] - ETA: 1s - loss: 0.0977 - accuracy: 0.8831
101504/108400 [===========================>..] - ETA: 0s - loss: 0.0975 - accuracy: 0.8834
102144/108400 [===========================>..] - ETA: 0s - loss: 0.0973 - accuracy: 0.8838
102784/108400 [===========================>..] - ETA: 0s - loss: 0.0970 - accuracy: 0.8840
103424/108400 [===========================>..] - ETA: 0s - loss: 0.0967 - accuracy: 0.8844
104064/108400 [===========================>..] - ETA: 0s - loss: 0.0964 - accuracy: 0.8849
104704/108400 [===========================>..] - ETA: 0s - loss: 0.0961 - accuracy: 0.8853
105216/108400 [============================>.] - ETA: 0s - loss: 0.0959 - accuracy: 0.8856
105728/108400 [============================>.] - ETA: 0s - loss: 0.0958 - accuracy: 0.8858
106240/108400 [============================>.] - ETA: 0s - loss: 0.0956 - accuracy: 0.8861
106624/108400 [============================>.] - ETA: 0s - loss: 0.0954 - accuracy: 0.8863
107136/108400 [============================>.] - ETA: 0s - loss: 0.0952 - accuracy: 0.8866
107776/108400 [============================>.] - ETA: 0s - loss: 0.0950 - accuracy: 0.8868
108400/108400 [==============================] - 15s 138us/step - loss: 0.0948 - accuracy: 0.8871 - val_loss: 0.0449 - val_accuracy: 0.9506
Epoch 2/20
128/108400 [..............................] - ETA: 15s - loss: 0.0733 - accuracy: 0.9219
640/108400 [..............................] - ETA: 13s - loss: 0.0597 - accuracy: 0.9344
1152/108400 [..............................] - ETA: 13s - loss: 0.0567 - accuracy: 0.9401
1664/108400 [..............................] - ETA: 12s - loss: 0.0517 - accuracy: 0.9477
2176/108400 [..............................] - ETA: 13s - loss: 0.0502 - accuracy: 0.9490
2432/108400 [..............................] - ETA: 14s - loss: 0.0511 - accuracy: 0.9457
2816/108400 [..............................] - ETA: 14s - loss: 0.0520 - accuracy: 0.9457
3328/108400 [..............................] - ETA: 13s - loss: 0.0513 - accuracy: 0.9471
3840/108400 [>.............................] - ETA: 13s - loss: 0.0523 - accuracy: 0.9448
4352/108400 [>.............................] - ETA: 13s - loss: 0.0508 - accuracy: 0.9472
4864/108400 [>.............................] - ETA: 13s - loss: 0.0506 - accuracy: 0.9480
5248/108400 [>.............................] - ETA: 13s - loss: 0.0497 - accuracy: 0.9495
5632/108400 [>.............................] - ETA: 13s - loss: 0.0496 - accuracy: 0.9496
6144/108400 [>.............................] - ETA: 12s - loss: 0.0498 - accuracy: 0.9492
6656/108400 [>.............................] - ETA: 12s - loss: 0.0503 - accuracy: 0.9483
7168/108400 [>.............................] - ETA: 12s - loss: 0.0511 - accuracy: 0.9480
7680/108400 [=>............................] - ETA: 12s - loss: 0.0513 - accuracy: 0.9478
8192/108400 [=>............................] - ETA: 12s - loss: 0.0509 - accuracy: 0.9487
8704/108400 [=>............................] - ETA: 12s - loss: 0.0508 - accuracy: 0.9492
9344/108400 [=>............................] - ETA: 11s - loss: 0.0510 - accuracy: 0.9488
9984/108400 [=>............................] - ETA: 11s - loss: 0.0510 - accuracy: 0.9488
10624/108400 [=>............................] - ETA: 11s - loss: 0.0508 - accuracy: 0.9493
11264/108400 [==>...........................] - ETA: 10s - loss: 0.0510 - accuracy: 0.9487
11904/108400 [==>...........................] - ETA: 10s - loss: 0.0511 - accuracy: 0.9486
12416/108400 [==>...........................] - ETA: 10s - loss: 0.0510 - accuracy: 0.9485
13056/108400 [==>...........................] - ETA: 10s - loss: 0.0507 - accuracy: 0.9488
13696/108400 [==>...........................] - ETA: 10s - loss: 0.0509 - accuracy: 0.9486
14336/108400 [==>...........................] - ETA: 10s - loss: 0.0509 - accuracy: 0.9482
14976/108400 [===>..........................] - ETA: 10s - loss: 0.0504 - accuracy: 0.9491
15616/108400 [===>..........................] - ETA: 9s - loss: 0.0504 - accuracy: 0.9486
16256/108400 [===>..........................] - ETA: 9s - loss: 0.0501 - accuracy: 0.9490
16896/108400 [===>..........................] - ETA: 9s - loss: 0.0501 - accuracy: 0.9488
17536/108400 [===>..........................] - ETA: 9s - loss: 0.0499 - accuracy: 0.9492
18176/108400 [====>.........................] - ETA: 9s - loss: 0.0496 - accuracy: 0.9497
18816/108400 [====>.........................] - ETA: 9s - loss: 0.0496 - accuracy: 0.9495
19456/108400 [====>.........................] - ETA: 9s - loss: 0.0496 - accuracy: 0.9496
20096/108400 [====>.........................] - ETA: 9s - loss: 0.0492 - accuracy: 0.9500
20736/108400 [====>.........................] - ETA: 8s - loss: 0.0490 - accuracy: 0.9502
21376/108400 [====>.........................] - ETA: 8s - loss: 0.0488 - accuracy: 0.9502
22016/108400 [=====>........................] - ETA: 8s - loss: 0.0486 - accuracy: 0.9503
22528/108400 [=====>........................] - ETA: 8s - loss: 0.0485 - accuracy: 0.9504
23168/108400 [=====>........................] - ETA: 8s - loss: 0.0482 - accuracy: 0.9506
23808/108400 [=====>........................] - ETA: 8s - loss: 0.0481 - accuracy: 0.9506
24448/108400 [=====>........................] - ETA: 8s - loss: 0.0481 - accuracy: 0.9506
25088/108400 [=====>........................] - ETA: 8s - loss: 0.0481 - accuracy: 0.9507
25728/108400 [======>.......................] - ETA: 8s - loss: 0.0479 - accuracy: 0.9508
26368/108400 [======>.......................] - ETA: 8s - loss: 0.0477 - accuracy: 0.9512
27008/108400 [======>.......................] - ETA: 8s - loss: 0.0477 - accuracy: 0.9512
27648/108400 [======>.......................] - ETA: 8s - loss: 0.0478 - accuracy: 0.9512
28288/108400 [======>.......................] - ETA: 7s - loss: 0.0478 - accuracy: 0.9513
28800/108400 [======>.......................] - ETA: 7s - loss: 0.0477 - accuracy: 0.9513
29440/108400 [=======>......................] - ETA: 7s - loss: 0.0476 - accuracy: 0.9515
29952/108400 [=======>......................] - ETA: 7s - loss: 0.0475 - accuracy: 0.9515
30592/108400 [=======>......................] - ETA: 7s - loss: 0.0473 - accuracy: 0.9517
31232/108400 [=======>......................] - ETA: 7s - loss: 0.0473 - accuracy: 0.9517
31744/108400 [=======>......................] - ETA: 7s - loss: 0.0473 - accuracy: 0.9519
32000/108400 [=======>......................] - ETA: 7s - loss: 0.0473 - accuracy: 0.9521
32256/108400 [=======>......................] - ETA: 7s - loss: 0.0473 - accuracy: 0.9519
32512/108400 [=======>......................] - ETA: 7s - loss: 0.0473 - accuracy: 0.9520
32896/108400 [========>.....................] - ETA: 7s - loss: 0.0473 - accuracy: 0.9522
33280/108400 [========>.....................] - ETA: 7s - loss: 0.0472 - accuracy: 0.9523
33536/108400 [========>.....................] - ETA: 7s - loss: 0.0472 - accuracy: 0.9524
33920/108400 [========>.....................] - ETA: 7s - loss: 0.0472 - accuracy: 0.9524
34176/108400 [========>.....................] - ETA: 7s - loss: 0.0472 - accuracy: 0.9524
34432/108400 [========>.....................] - ETA: 8s - loss: 0.0472 - accuracy: 0.9524
34688/108400 [========>.....................] - ETA: 8s - loss: 0.0472 - accuracy: 0.9525
34944/108400 [========>.....................] - ETA: 8s - loss: 0.0471 - accuracy: 0.9526
35328/108400 [========>.....................] - ETA: 8s - loss: 0.0470 - accuracy: 0.9527
35584/108400 [========>.....................] - ETA: 8s - loss: 0.0470 - accuracy: 0.9527
35968/108400 [========>.....................] - ETA: 8s - loss: 0.0469 - accuracy: 0.9527
36352/108400 [=========>....................] - ETA: 8s - loss: 0.0468 - accuracy: 0.9529
36736/108400 [=========>....................] - ETA: 8s - loss: 0.0468 - accuracy: 0.9529
36864/108400 [=========>....................] - ETA: 8s - loss: 0.0468 - accuracy: 0.9528
37120/108400 [=========>....................] - ETA: 8s - loss: 0.0467 - accuracy: 0.9529
37376/108400 [=========>....................] - ETA: 8s - loss: 0.0467 - accuracy: 0.9530
37760/108400 [=========>....................] - ETA: 8s - loss: 0.0467 - accuracy: 0.9531
38016/108400 [=========>....................] - ETA: 8s - loss: 0.0466 - accuracy: 0.9532
38272/108400 [=========>....................] - ETA: 8s - loss: 0.0466 - accuracy: 0.9532
38528/108400 [=========>....................] - ETA: 8s - loss: 0.0466 - accuracy: 0.9532
38912/108400 [=========>....................] - ETA: 8s - loss: 0.0465 - accuracy: 0.9533
39296/108400 [=========>....................] - ETA: 8s - loss: 0.0465 - accuracy: 0.9534
39680/108400 [=========>....................] - ETA: 8s - loss: 0.0463 - accuracy: 0.9535
40064/108400 [==========>...................] - ETA: 8s - loss: 0.0463 - accuracy: 0.9536
40448/108400 [==========>...................] - ETA: 8s - loss: 0.0463 - accuracy: 0.9537
40832/108400 [==========>...................] - ETA: 8s - loss: 0.0462 - accuracy: 0.9538
41088/108400 [==========>...................] - ETA: 8s - loss: 0.0462 - accuracy: 0.9538
41472/108400 [==========>...................] - ETA: 8s - loss: 0.0461 - accuracy: 0.9538
41856/108400 [==========>...................] - ETA: 8s - loss: 0.0461 - accuracy: 0.9538
42240/108400 [==========>...................] - ETA: 8s - loss: 0.0461 - accuracy: 0.9536
42624/108400 [==========>...................] - ETA: 8s - loss: 0.0462 - accuracy: 0.9536
43136/108400 [==========>...................] - ETA: 8s - loss: 0.0462 - accuracy: 0.9535
43648/108400 [===========>..................] - ETA: 8s - loss: 0.0462 - accuracy: 0.9533
44032/108400 [===========>..................] - ETA: 8s - loss: 0.0462 - accuracy: 0.9535
44416/108400 [===========>..................] - ETA: 8s - loss: 0.0461 - accuracy: 0.9535
44800/108400 [===========>..................] - ETA: 8s - loss: 0.0461 - accuracy: 0.9536
45184/108400 [===========>..................] - ETA: 8s - loss: 0.0460 - accuracy: 0.9537
45696/108400 [===========>..................] - ETA: 7s - loss: 0.0459 - accuracy: 0.9537
46208/108400 [===========>..................] - ETA: 7s - loss: 0.0459 - accuracy: 0.9536
46720/108400 [===========>..................] - ETA: 7s - loss: 0.0458 - accuracy: 0.9538
47232/108400 [============>.................] - ETA: 7s - loss: 0.0458 - accuracy: 0.9539
47744/108400 [============>.................] - ETA: 7s - loss: 0.0457 - accuracy: 0.9540
48256/108400 [============>.................] - ETA: 7s - loss: 0.0457 - accuracy: 0.9541
48768/108400 [============>.................] - ETA: 7s - loss: 0.0457 - accuracy: 0.9540
49280/108400 [============>.................] - ETA: 7s - loss: 0.0456 - accuracy: 0.9543
49920/108400 [============>.................] - ETA: 7s - loss: 0.0456 - accuracy: 0.9544
50432/108400 [============>.................] - ETA: 7s - loss: 0.0455 - accuracy: 0.9545
51072/108400 [=============>................] - ETA: 7s - loss: 0.0453 - accuracy: 0.9548
51584/108400 [=============>................] - ETA: 7s - loss: 0.0452 - accuracy: 0.9549
51968/108400 [=============>................] - ETA: 7s - loss: 0.0452 - accuracy: 0.9549
52480/108400 [=============>................] - ETA: 6s - loss: 0.0452 - accuracy: 0.9549
53120/108400 [=============>................] - ETA: 6s - loss: 0.0452 - accuracy: 0.9548
53760/108400 [=============>................] - ETA: 6s - loss: 0.0451 - accuracy: 0.9549
54272/108400 [==============>...............] - ETA: 6s - loss: 0.0451 - accuracy: 0.9549
54784/108400 [==============>...............] - ETA: 6s - loss: 0.0451 - accuracy: 0.9548
55424/108400 [==============>...............] - ETA: 6s - loss: 0.0450 - accuracy: 0.9551
56064/108400 [==============>...............] - ETA: 6s - loss: 0.0449 - accuracy: 0.9552
56704/108400 [==============>...............] - ETA: 6s - loss: 0.0448 - accuracy: 0.9553
57344/108400 [==============>...............] - ETA: 6s - loss: 0.0448 - accuracy: 0.9553
57984/108400 [===============>..............] - ETA: 6s - loss: 0.0448 - accuracy: 0.9553
58624/108400 [===============>..............] - ETA: 6s - loss: 0.0447 - accuracy: 0.9554
59264/108400 [===============>..............] - ETA: 5s - loss: 0.0447 - accuracy: 0.9553
59904/108400 [===============>..............] - ETA: 5s - loss: 0.0446 - accuracy: 0.9554
60544/108400 [===============>..............] - ETA: 5s - loss: 0.0447 - accuracy: 0.9553
61184/108400 [===============>..............] - ETA: 5s - loss: 0.0446 - accuracy: 0.9554
61824/108400 [================>.............] - ETA: 5s - loss: 0.0446 - accuracy: 0.9554
62336/108400 [================>.............] - ETA: 5s - loss: 0.0445 - accuracy: 0.9555
62848/108400 [================>.............] - ETA: 5s - loss: 0.0444 - accuracy: 0.9556
63360/108400 [================>.............] - ETA: 5s - loss: 0.0444 - accuracy: 0.9555
63872/108400 [================>.............] - ETA: 5s - loss: 0.0443 - accuracy: 0.9555
64384/108400 [================>.............] - ETA: 5s - loss: 0.0443 - accuracy: 0.9556
64896/108400 [================>.............] - ETA: 5s - loss: 0.0442 - accuracy: 0.9557
65280/108400 [=================>............] - ETA: 5s - loss: 0.0442 - accuracy: 0.9556
65792/108400 [=================>............] - ETA: 5s - loss: 0.0442 - accuracy: 0.9556
66432/108400 [=================>............] - ETA: 4s - loss: 0.0441 - accuracy: 0.9557
67072/108400 [=================>............] - ETA: 4s - loss: 0.0440 - accuracy: 0.9558
67712/108400 [=================>............] - ETA: 4s - loss: 0.0439 - accuracy: 0.9559
68352/108400 [=================>............] - ETA: 4s - loss: 0.0440 - accuracy: 0.9559
68992/108400 [==================>...........] - ETA: 4s - loss: 0.0439 - accuracy: 0.9559
69632/108400 [==================>...........] - ETA: 4s - loss: 0.0439 - accuracy: 0.9558
70272/108400 [==================>...........] - ETA: 4s - loss: 0.0439 - accuracy: 0.9560
70912/108400 [==================>...........] - ETA: 4s - loss: 0.0438 - accuracy: 0.9561
71552/108400 [==================>...........] - ETA: 4s - loss: 0.0437 - accuracy: 0.9562
72192/108400 [==================>...........] - ETA: 4s - loss: 0.0437 - accuracy: 0.9562
72832/108400 [===================>..........] - ETA: 4s - loss: 0.0437 - accuracy: 0.9562
73472/108400 [===================>..........] - ETA: 4s - loss: 0.0437 - accuracy: 0.9563
74112/108400 [===================>..........] - ETA: 3s - loss: 0.0436 - accuracy: 0.9563
74752/108400 [===================>..........] - ETA: 3s - loss: 0.0436 - accuracy: 0.9564
75392/108400 [===================>..........] - ETA: 3s - loss: 0.0435 - accuracy: 0.9565
76032/108400 [====================>.........] - ETA: 3s - loss: 0.0435 - accuracy: 0.9565
76672/108400 [====================>.........] - ETA: 3s - loss: 0.0434 - accuracy: 0.9566
77312/108400 [====================>.........] - ETA: 3s - loss: 0.0433 - accuracy: 0.9567
77952/108400 [====================>.........] - ETA: 3s - loss: 0.0433 - accuracy: 0.9566
78592/108400 [====================>.........] - ETA: 3s - loss: 0.0433 - accuracy: 0.9566
78848/108400 [====================>.........] - ETA: 3s - loss: 0.0433 - accuracy: 0.9566
79104/108400 [====================>.........] - ETA: 3s - loss: 0.0433 - accuracy: 0.9567
79616/108400 [=====================>........] - ETA: 3s - loss: 0.0432 - accuracy: 0.9567
79872/108400 [=====================>........] - ETA: 3s - loss: 0.0432 - accuracy: 0.9568
80128/108400 [=====================>........] - ETA: 3s - loss: 0.0432 - accuracy: 0.9568
80384/108400 [=====================>........] - ETA: 3s - loss: 0.0432 - accuracy: 0.9568
80768/108400 [=====================>........] - ETA: 3s - loss: 0.0432 - accuracy: 0.9567
81152/108400 [=====================>........] - ETA: 3s - loss: 0.0432 - accuracy: 0.9567
81664/108400 [=====================>........] - ETA: 3s - loss: 0.0431 - accuracy: 0.9567
82176/108400 [=====================>........] - ETA: 3s - loss: 0.0431 - accuracy: 0.9567
82688/108400 [=====================>........] - ETA: 3s - loss: 0.0431 - accuracy: 0.9568
83200/108400 [======================>.......] - ETA: 2s - loss: 0.0430 - accuracy: 0.9568
83712/108400 [======================>.......] - ETA: 2s - loss: 0.0430 - accuracy: 0.9568
84224/108400 [======================>.......] - ETA: 2s - loss: 0.0430 - accuracy: 0.9569
84736/108400 [======================>.......] - ETA: 2s - loss: 0.0429 - accuracy: 0.9569
85248/108400 [======================>.......] - ETA: 2s - loss: 0.0429 - accuracy: 0.9570
85760/108400 [======================>.......] - ETA: 2s - loss: 0.0428 - accuracy: 0.9570
86272/108400 [======================>.......] - ETA: 2s - loss: 0.0428 - accuracy: 0.9571
86784/108400 [=======================>......] - ETA: 2s - loss: 0.0427 - accuracy: 0.9572
87296/108400 [=======================>......] - ETA: 2s - loss: 0.0427 - accuracy: 0.9573
87808/108400 [=======================>......] - ETA: 2s - loss: 0.0426 - accuracy: 0.9574
88320/108400 [=======================>......] - ETA: 2s - loss: 0.0426 - accuracy: 0.9574
88832/108400 [=======================>......] - ETA: 2s - loss: 0.0425 - accuracy: 0.9575
89344/108400 [=======================>......] - ETA: 2s - loss: 0.0425 - accuracy: 0.9574
89856/108400 [=======================>......] - ETA: 2s - loss: 0.0425 - accuracy: 0.9575
90368/108400 [========================>.....] - ETA: 2s - loss: 0.0425 - accuracy: 0.9575
90880/108400 [========================>.....] - ETA: 2s - loss: 0.0424 - accuracy: 0.9575
91392/108400 [========================>.....] - ETA: 2s - loss: 0.0424 - accuracy: 0.9575
91904/108400 [========================>.....] - ETA: 1s - loss: 0.0424 - accuracy: 0.9575
92544/108400 [========================>.....] - ETA: 1s - loss: 0.0423 - accuracy: 0.9575
93184/108400 [========================>.....] - ETA: 1s - loss: 0.0423 - accuracy: 0.9576
93824/108400 [========================>.....] - ETA: 1s - loss: 0.0423 - accuracy: 0.9576
94208/108400 [=========================>....] - ETA: 1s - loss: 0.0422 - accuracy: 0.9577
94848/108400 [=========================>....] - ETA: 1s - loss: 0.0422 - accuracy: 0.9577
95488/108400 [=========================>....] - ETA: 1s - loss: 0.0422 - accuracy: 0.9577
96128/108400 [=========================>....] - ETA: 1s - loss: 0.0421 - accuracy: 0.9579
96384/108400 [=========================>....] - ETA: 1s - loss: 0.0421 - accuracy: 0.9579
96768/108400 [=========================>....] - ETA: 1s - loss: 0.0420 - accuracy: 0.9579
97408/108400 [=========================>....] - ETA: 1s - loss: 0.0419 - accuracy: 0.9580
98048/108400 [==========================>...] - ETA: 1s - loss: 0.0419 - accuracy: 0.9580
98560/108400 [==========================>...] - ETA: 1s - loss: 0.0419 - accuracy: 0.9581
99200/108400 [==========================>...] - ETA: 1s - loss: 0.0419 - accuracy: 0.9582
99840/108400 [==========================>...] - ETA: 0s - loss: 0.0418 - accuracy: 0.9582
100480/108400 [==========================>...] - ETA: 0s - loss: 0.0418 - accuracy: 0.9582
101120/108400 [==========================>...] - ETA: 0s - loss: 0.0418 - accuracy: 0.9582
101760/108400 [===========================>..] - ETA: 0s - loss: 0.0417 - accuracy: 0.9583
102400/108400 [===========================>..] - ETA: 0s - loss: 0.0417 - accuracy: 0.9584
103040/108400 [===========================>..] - ETA: 0s - loss: 0.0416 - accuracy: 0.9584
103680/108400 [===========================>..] - ETA: 0s - loss: 0.0416 - accuracy: 0.9585
104320/108400 [===========================>..] - ETA: 0s - loss: 0.0415 - accuracy: 0.9586
104960/108400 [============================>.] - ETA: 0s - loss: 0.0414 - accuracy: 0.9586
105600/108400 [============================>.] - ETA: 0s - loss: 0.0414 - accuracy: 0.9587
106112/108400 [============================>.] - ETA: 0s - loss: 0.0413 - accuracy: 0.9588
106752/108400 [============================>.] - ETA: 0s - loss: 0.0413 - accuracy: 0.9588
107392/108400 [============================>.] - ETA: 0s - loss: 0.0412 - accuracy: 0.9589
107904/108400 [============================>.] - ETA: 0s - loss: 0.0412 - accuracy: 0.9589
108400/108400 [==============================] - 13s 121us/step - loss: 0.0412 - accuracy: 0.9589 - val_loss: 0.0320 - val_accuracy: 0.9685
Epoch 3/20
128/108400 [..............................] - ETA: 13s - loss: 0.0147 - accuracy: 1.0000
768/108400 [..............................] - ETA: 9s - loss: 0.0338 - accuracy: 0.9688
1408/108400 [..............................] - ETA: 9s - loss: 0.0344 - accuracy: 0.9666
2048/108400 [..............................] - ETA: 9s - loss: 0.0327 - accuracy: 0.9678
2688/108400 [..............................] - ETA: 9s - loss: 0.0328 - accuracy: 0.9684
3328/108400 [..............................] - ETA: 9s - loss: 0.0329 - accuracy: 0.9691
3968/108400 [>.............................] - ETA: 9s - loss: 0.0325 - accuracy: 0.9698
4608/108400 [>.............................] - ETA: 9s - loss: 0.0330 - accuracy: 0.9685
5248/108400 [>.............................] - ETA: 9s - loss: 0.0328 - accuracy: 0.9688
5888/108400 [>.............................] - ETA: 8s - loss: 0.0325 - accuracy: 0.9689
6528/108400 [>.............................] - ETA: 8s - loss: 0.0329 - accuracy: 0.9681
7168/108400 [>.............................] - ETA: 8s - loss: 0.0331 - accuracy: 0.9682
7808/108400 [=>............................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9682
8448/108400 [=>............................] - ETA: 8s - loss: 0.0326 - accuracy: 0.9684
9088/108400 [=>............................] - ETA: 8s - loss: 0.0326 - accuracy: 0.9683
9728/108400 [=>............................] - ETA: 8s - loss: 0.0325 - accuracy: 0.9685
10368/108400 [=>............................] - ETA: 8s - loss: 0.0323 - accuracy: 0.9686
10880/108400 [==>...........................] - ETA: 8s - loss: 0.0323 - accuracy: 0.9689
11392/108400 [==>...........................] - ETA: 8s - loss: 0.0324 - accuracy: 0.9686
11904/108400 [==>...........................] - ETA: 8s - loss: 0.0325 - accuracy: 0.9683
12416/108400 [==>...........................] - ETA: 8s - loss: 0.0326 - accuracy: 0.9686
12800/108400 [==>...........................] - ETA: 8s - loss: 0.0324 - accuracy: 0.9688
13312/108400 [==>...........................] - ETA: 8s - loss: 0.0323 - accuracy: 0.9689
13824/108400 [==>...........................] - ETA: 9s - loss: 0.0323 - accuracy: 0.9690
14336/108400 [==>...........................] - ETA: 9s - loss: 0.0321 - accuracy: 0.9692
14848/108400 [===>..........................] - ETA: 9s - loss: 0.0324 - accuracy: 0.9689
15232/108400 [===>..........................] - ETA: 9s - loss: 0.0325 - accuracy: 0.9686
15616/108400 [===>..........................] - ETA: 9s - loss: 0.0325 - accuracy: 0.9684
16128/108400 [===>..........................] - ETA: 9s - loss: 0.0326 - accuracy: 0.9684
16640/108400 [===>..........................] - ETA: 9s - loss: 0.0327 - accuracy: 0.9681
17152/108400 [===>..........................] - ETA: 9s - loss: 0.0327 - accuracy: 0.9682
17664/108400 [===>..........................] - ETA: 9s - loss: 0.0328 - accuracy: 0.9680
18176/108400 [====>.........................] - ETA: 8s - loss: 0.0326 - accuracy: 0.9681
18688/108400 [====>.........................] - ETA: 8s - loss: 0.0326 - accuracy: 0.9682
19328/108400 [====>.........................] - ETA: 8s - loss: 0.0324 - accuracy: 0.9685
19968/108400 [====>.........................] - ETA: 8s - loss: 0.0325 - accuracy: 0.9682
20352/108400 [====>.........................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9680
20992/108400 [====>.........................] - ETA: 8s - loss: 0.0329 - accuracy: 0.9677
21632/108400 [====>.........................] - ETA: 8s - loss: 0.0329 - accuracy: 0.9676
22144/108400 [=====>........................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9676
22656/108400 [=====>........................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9676
23296/108400 [=====>........................] - ETA: 8s - loss: 0.0327 - accuracy: 0.9677
23936/108400 [=====>........................] - ETA: 8s - loss: 0.0325 - accuracy: 0.9681
24576/108400 [=====>........................] - ETA: 8s - loss: 0.0322 - accuracy: 0.9685
25216/108400 [=====>........................] - ETA: 8s - loss: 0.0322 - accuracy: 0.9686
25856/108400 [======>.......................] - ETA: 8s - loss: 0.0320 - accuracy: 0.9689
26496/108400 [======>.......................] - ETA: 8s - loss: 0.0320 - accuracy: 0.9689
27008/108400 [======>.......................] - ETA: 8s - loss: 0.0319 - accuracy: 0.9690
27520/108400 [======>.......................] - ETA: 7s - loss: 0.0319 - accuracy: 0.9690
28160/108400 [======>.......................] - ETA: 7s - loss: 0.0319 - accuracy: 0.9689
28800/108400 [======>.......................] - ETA: 7s - loss: 0.0318 - accuracy: 0.9690
29440/108400 [=======>......................] - ETA: 7s - loss: 0.0317 - accuracy: 0.9691
30080/108400 [=======>......................] - ETA: 7s - loss: 0.0316 - accuracy: 0.9692
30720/108400 [=======>......................] - ETA: 7s - loss: 0.0316 - accuracy: 0.9693
31360/108400 [=======>......................] - ETA: 7s - loss: 0.0316 - accuracy: 0.9695
32000/108400 [=======>......................] - ETA: 7s - loss: 0.0316 - accuracy: 0.9693
32640/108400 [========>.....................] - ETA: 7s - loss: 0.0315 - accuracy: 0.9695
33280/108400 [========>.....................] - ETA: 7s - loss: 0.0316 - accuracy: 0.9693
33920/108400 [========>.....................] - ETA: 7s - loss: 0.0315 - accuracy: 0.9694
34560/108400 [========>.....................] - ETA: 7s - loss: 0.0315 - accuracy: 0.9694
35200/108400 [========>.....................] - ETA: 7s - loss: 0.0314 - accuracy: 0.9696
35712/108400 [========>.....................] - ETA: 7s - loss: 0.0315 - accuracy: 0.9695
36224/108400 [=========>....................] - ETA: 7s - loss: 0.0315 - accuracy: 0.9693
36736/108400 [=========>....................] - ETA: 6s - loss: 0.0315 - accuracy: 0.9695
37248/108400 [=========>....................] - ETA: 6s - loss: 0.0316 - accuracy: 0.9693
37760/108400 [=========>....................] - ETA: 6s - loss: 0.0317 - accuracy: 0.9692
38272/108400 [=========>....................] - ETA: 6s - loss: 0.0316 - accuracy: 0.9694
38912/108400 [=========>....................] - ETA: 6s - loss: 0.0315 - accuracy: 0.9695
39552/108400 [=========>....................] - ETA: 6s - loss: 0.0315 - accuracy: 0.9695
40192/108400 [==========>...................] - ETA: 6s - loss: 0.0314 - accuracy: 0.9696
40832/108400 [==========>...................] - ETA: 6s - loss: 0.0314 - accuracy: 0.9696
41472/108400 [==========>...................] - ETA: 6s - loss: 0.0314 - accuracy: 0.9696
41984/108400 [==========>...................] - ETA: 6s - loss: 0.0315 - accuracy: 0.9694
42624/108400 [==========>...................] - ETA: 6s - loss: 0.0314 - accuracy: 0.9695
43264/108400 [==========>...................] - ETA: 6s - loss: 0.0313 - accuracy: 0.9696
43904/108400 [===========>..................] - ETA: 6s - loss: 0.0312 - accuracy: 0.9697
44544/108400 [===========>..................] - ETA: 6s - loss: 0.0312 - accuracy: 0.9696
45184/108400 [===========>..................] - ETA: 6s - loss: 0.0313 - accuracy: 0.9695
45824/108400 [===========>..................] - ETA: 6s - loss: 0.0313 - accuracy: 0.9694
46464/108400 [===========>..................] - ETA: 5s - loss: 0.0313 - accuracy: 0.9694
47104/108400 [============>.................] - ETA: 5s - loss: 0.0313 - accuracy: 0.9695
47744/108400 [============>.................] - ETA: 5s - loss: 0.0314 - accuracy: 0.9693
48384/108400 [============>.................] - ETA: 5s - loss: 0.0312 - accuracy: 0.9695
49024/108400 [============>.................] - ETA: 5s - loss: 0.0311 - accuracy: 0.9696
49664/108400 [============>.................] - ETA: 5s - loss: 0.0311 - accuracy: 0.9697
50304/108400 [============>.................] - ETA: 5s - loss: 0.0310 - accuracy: 0.9697
50944/108400 [=============>................] - ETA: 5s - loss: 0.0310 - accuracy: 0.9697
51456/108400 [=============>................] - ETA: 5s - loss: 0.0309 - accuracy: 0.9698
52096/108400 [=============>................] - ETA: 5s - loss: 0.0309 - accuracy: 0.9699
52608/108400 [=============>................] - ETA: 5s - loss: 0.0309 - accuracy: 0.9698
53120/108400 [=============>................] - ETA: 5s - loss: 0.0309 - accuracy: 0.9698
53760/108400 [=============>................] - ETA: 5s - loss: 0.0308 - accuracy: 0.9700
54400/108400 [==============>...............] - ETA: 5s - loss: 0.0307 - accuracy: 0.9700
55040/108400 [==============>...............] - ETA: 5s - loss: 0.0307 - accuracy: 0.9700
55680/108400 [==============>...............] - ETA: 5s - loss: 0.0306 - accuracy: 0.9701
56320/108400 [==============>...............] - ETA: 4s - loss: 0.0306 - accuracy: 0.9702
56832/108400 [==============>...............] - ETA: 4s - loss: 0.0306 - accuracy: 0.9702
57472/108400 [==============>...............] - ETA: 4s - loss: 0.0306 - accuracy: 0.9702
58112/108400 [===============>..............] - ETA: 4s - loss: 0.0307 - accuracy: 0.9701
58752/108400 [===============>..............] - ETA: 4s - loss: 0.0307 - accuracy: 0.9701
59392/108400 [===============>..............] - ETA: 4s - loss: 0.0307 - accuracy: 0.9700
60032/108400 [===============>..............] - ETA: 4s - loss: 0.0308 - accuracy: 0.9700
60672/108400 [===============>..............] - ETA: 4s - loss: 0.0307 - accuracy: 0.9700
61312/108400 [===============>..............] - ETA: 4s - loss: 0.0307 - accuracy: 0.9701
61952/108400 [================>.............] - ETA: 4s - loss: 0.0306 - accuracy: 0.9702
62592/108400 [================>.............] - ETA: 4s - loss: 0.0306 - accuracy: 0.9703
63232/108400 [================>.............] - ETA: 4s - loss: 0.0305 - accuracy: 0.9704
63872/108400 [================>.............] - ETA: 4s - loss: 0.0304 - accuracy: 0.9705
64512/108400 [================>.............] - ETA: 4s - loss: 0.0304 - accuracy: 0.9704
65152/108400 [=================>............] - ETA: 4s - loss: 0.0304 - accuracy: 0.9704
65792/108400 [=================>............] - ETA: 4s - loss: 0.0303 - accuracy: 0.9704
66432/108400 [=================>............] - ETA: 3s - loss: 0.0303 - accuracy: 0.9705
67072/108400 [=================>............] - ETA: 3s - loss: 0.0302 - accuracy: 0.9706
67712/108400 [=================>............] - ETA: 3s - loss: 0.0302 - accuracy: 0.9707
68352/108400 [=================>............] - ETA: 3s - loss: 0.0302 - accuracy: 0.9707
68864/108400 [==================>...........] - ETA: 3s - loss: 0.0301 - accuracy: 0.9707
69376/108400 [==================>...........] - ETA: 3s - loss: 0.0301 - accuracy: 0.9708
69760/108400 [==================>...........] - ETA: 3s - loss: 0.0301 - accuracy: 0.9708
70144/108400 [==================>...........] - ETA: 3s - loss: 0.0301 - accuracy: 0.9708
70528/108400 [==================>...........] - ETA: 3s - loss: 0.0301 - accuracy: 0.9708
70912/108400 [==================>...........] - ETA: 3s - loss: 0.0301 - accuracy: 0.9707
71552/108400 [==================>...........] - ETA: 3s - loss: 0.0300 - accuracy: 0.9708
72192/108400 [==================>...........] - ETA: 3s - loss: 0.0300 - accuracy: 0.9708
72832/108400 [===================>..........] - ETA: 3s - loss: 0.0300 - accuracy: 0.9708
73472/108400 [===================>..........] - ETA: 3s - loss: 0.0300 - accuracy: 0.9708
74112/108400 [===================>..........] - ETA: 3s - loss: 0.0300 - accuracy: 0.9708
74752/108400 [===================>..........] - ETA: 3s - loss: 0.0300 - accuracy: 0.9709
75392/108400 [===================>..........] - ETA: 3s - loss: 0.0299 - accuracy: 0.9709
76032/108400 [====================>.........] - ETA: 3s - loss: 0.0299 - accuracy: 0.9709
76672/108400 [====================>.........] - ETA: 2s - loss: 0.0299 - accuracy: 0.9710
77312/108400 [====================>.........] - ETA: 2s - loss: 0.0299 - accuracy: 0.9710
77952/108400 [====================>.........] - ETA: 2s - loss: 0.0299 - accuracy: 0.9709
78592/108400 [====================>.........] - ETA: 2s - loss: 0.0299 - accuracy: 0.9710
79232/108400 [====================>.........] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
79872/108400 [=====================>........] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
80512/108400 [=====================>........] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
81152/108400 [=====================>........] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
81792/108400 [=====================>........] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
82432/108400 [=====================>........] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
83072/108400 [=====================>........] - ETA: 2s - loss: 0.0297 - accuracy: 0.9710
83712/108400 [======================>.......] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
84352/108400 [======================>.......] - ETA: 2s - loss: 0.0298 - accuracy: 0.9710
84992/108400 [======================>.......] - ETA: 2s - loss: 0.0297 - accuracy: 0.9710
85632/108400 [======================>.......] - ETA: 2s - loss: 0.0297 - accuracy: 0.9710
86272/108400 [======================>.......] - ETA: 2s - loss: 0.0297 - accuracy: 0.9710
86912/108400 [=======================>......] - ETA: 2s - loss: 0.0297 - accuracy: 0.9711
87552/108400 [=======================>......] - ETA: 1s - loss: 0.0297 - accuracy: 0.9710
88192/108400 [=======================>......] - ETA: 1s - loss: 0.0296 - accuracy: 0.9710
88832/108400 [=======================>......] - ETA: 1s - loss: 0.0296 - accuracy: 0.9710
89472/108400 [=======================>......] - ETA: 1s - loss: 0.0295 - accuracy: 0.9711
90112/108400 [=======================>......] - ETA: 1s - loss: 0.0295 - accuracy: 0.9711
90752/108400 [========================>.....] - ETA: 1s - loss: 0.0295 - accuracy: 0.9711
91392/108400 [========================>.....] - ETA: 1s - loss: 0.0295 - accuracy: 0.9711
92032/108400 [========================>.....] - ETA: 1s - loss: 0.0294 - accuracy: 0.9711
92672/108400 [========================>.....] - ETA: 1s - loss: 0.0294 - accuracy: 0.9711
93312/108400 [========================>.....] - ETA: 1s - loss: 0.0294 - accuracy: 0.9712
93952/108400 [=========================>....] - ETA: 1s - loss: 0.0294 - accuracy: 0.9712
94592/108400 [=========================>....] - ETA: 1s - loss: 0.0293 - accuracy: 0.9712
95232/108400 [=========================>....] - ETA: 1s - loss: 0.0294 - accuracy: 0.9712
95872/108400 [=========================>....] - ETA: 1s - loss: 0.0293 - accuracy: 0.9712
96512/108400 [=========================>....] - ETA: 1s - loss: 0.0293 - accuracy: 0.9712
97152/108400 [=========================>....] - ETA: 1s - loss: 0.0293 - accuracy: 0.9713
97792/108400 [==========================>...] - ETA: 0s - loss: 0.0293 - accuracy: 0.9713
98432/108400 [==========================>...] - ETA: 0s - loss: 0.0293 - accuracy: 0.9713
99072/108400 [==========================>...] - ETA: 0s - loss: 0.0292 - accuracy: 0.9713
99712/108400 [==========================>...] - ETA: 0s - loss: 0.0292 - accuracy: 0.9713
100352/108400 [==========================>...] - ETA: 0s - loss: 0.0293 - accuracy: 0.9712
100992/108400 [==========================>...] - ETA: 0s - loss: 0.0292 - accuracy: 0.9713
101504/108400 [===========================>..] - ETA: 0s - loss: 0.0292 - accuracy: 0.9713
102144/108400 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9714
102784/108400 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9714
103424/108400 [===========================>..] - ETA: 0s - loss: 0.0291 - accuracy: 0.9715
104064/108400 [===========================>..] - ETA: 0s - loss: 0.0290 - accuracy: 0.9715
104704/108400 [===========================>..] - ETA: 0s - loss: 0.0290 - accuracy: 0.9715
105344/108400 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9715
105984/108400 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9716
106624/108400 [============================>.] - ETA: 0s - loss: 0.0289 - accuracy: 0.9716
107264/108400 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9716
107904/108400 [============================>.] - ETA: 0s - loss: 0.0290 - accuracy: 0.9715
108400/108400 [==============================] - 11s 98us/step - loss: 0.0290 - accuracy: 0.9716 - val_loss: 0.0288 - val_accuracy: 0.9693
Epoch 4/20
128/108400 [..............................] - ETA: 13s - loss: 0.0247 - accuracy: 0.9688
768/108400 [..............................] - ETA: 10s - loss: 0.0217 - accuracy: 0.9805
1408/108400 [..............................] - ETA: 9s - loss: 0.0215 - accuracy: 0.9815
2048/108400 [..............................] - ETA: 9s - loss: 0.0227 - accuracy: 0.9790
2560/108400 [..............................] - ETA: 10s - loss: 0.0239 - accuracy: 0.9766
2944/108400 [..............................] - ETA: 10s - loss: 0.0229 - accuracy: 0.9779
3456/108400 [..............................] - ETA: 11s - loss: 0.0232 - accuracy: 0.9780
3840/108400 [>.............................] - ETA: 11s - loss: 0.0235 - accuracy: 0.9779
4224/108400 [>.............................] - ETA: 11s - loss: 0.0236 - accuracy: 0.9780
4736/108400 [>.............................] - ETA: 11s - loss: 0.0238 - accuracy: 0.9772
5248/108400 [>.............................] - ETA: 11s - loss: 0.0231 - accuracy: 0.9783
5888/108400 [>...................