tensorflow程式碼全解析 -1- TensorBoard 入門案例
本文概要:通過mnist識別案例講解TensorFlow中TensorBoard的使用方法
原始碼
TensorBoard概述
TensorBoard 可以將模型訓練過程中的各種彙總資料展示出來。包括
- 標量 Scalars - tf.summary.scalar
- 圖片 Images - tf.summary.image
- 音訊 Audio
- 計算圖 Graphs
- 資料分佈 Distributions
- 直方圖 Histogram
- 嵌入向量 Embeddings
——這些向量將會經常用到
suammary 節點需要專門去執行才能起作用,使用tf.summary.merge_all可以將所有summary節點合併成一個節點,只要執行這個節點,就能產生之前設定的所有summary
使用tf.summary.FileWriter將執行後輸出的資料都儲存到本地磁碟中
啟動程式後,在使用命令列進入相對應目錄輸入tensorboard指定,才可以檢視視覺化檔案
使用TensorBoard 展示資料需要在執行Tensoflow計算圖的過程中,將各類資料彙總並記錄到日誌檔案中,然後在使用tensorBoard讀取這些日誌檔案,解析並生產資料視覺化的web頁面。
程式碼框架
- 讀取mnist資料集 read_data_sets(),定義初始化引數方法,定義輸入資料feed_dict()
- 定義輸入資料 x,xs
- 定義輸人標籤 y,ys
- 定義資料彙總方法 variable_summaries()
- 建立神經網路框架 nn_layer()
- 第一層 hidden1 = nn_layer(x,784,500,’layer1’)
- dropout層 dropped = tf.nn.droupout(hidden1 )
- 第二層 y = nn_layer(dropped, 500, 10, ‘layer2’, act = tf.identity)
- 輸出 y
- 建立損失函式 cross_entropy
- 建立訓練優化器 AdamOptimizer
- 定義準確度tf.summary.scalar.correct_ prediction
合併所有summary節點merged = tf.summary.merge_all() - 模型訓練 sess.run
執行環境
作業系統
- win10
- python 3.5
- tensorflow-gpu 1.0.0
注意事項及BUG
1 . BUG 執行中間發生Python執行非法指令錯誤,同時執行視窗報告:
Couldn’t open CUDA library cupti64_80.dll無法繼續執行
原因:CUDA的cupti64_80.dll的路徑沒有加入PATH
解決辦法:
將目錄C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64下的cupti64_80.dll 複製到C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin即可
參考:
I have encountered this problem before. When you use CUDA 8.0,the file cupti64_80.dll lies in C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64. I just fixed the problem by copying the dll into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin, and the file cupti.lib in the same location into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64. And it works!
2 . 初始化目錄問題及啟動tensorboard
parser.add_argument('--data_dir', type=str, default='input_data',
help='Directory for storing input data')
parser.add_argument('--log_dir', type=str, default='logs',
help='Summaries logs directory')
目錄設定好後,要在自己程式碼所在的資料夾裡面新建這兩個資料夾,如果沒有新建自己不會主動建立,反正我的是沒有,有的人是可以
呼叫tensorboard要進入目標命令列裡面
tensorboard --logdir=logs --debug
要開啟 debug 模式 就可以看到是不是讀取了日誌檔案
這個地方頭疼了好久,按照如上設定就可以正確的在網頁上顯示
3 . BUG 報如下錯誤
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input/x-input' with dtype float
[[Node: input/x-input = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/gpu:0"]()]]
[[Node: layer2_1/weights/summaries/stddev/Sqrt/_21 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_710_layer2_1/weights/summaries/stddev/Sqrt", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
這個問題花費了非常非常的時間去解決,多方查詢資料
因為是自己一個程式碼一個程式碼敲進去的,所以反覆核對了好幾遍,但都沒有發現問題
原因:因為spyder執行一次session後會組織其再次執行
解決辦法:在命令列列裡面執行就可以了
參考:
I find out that once you run it once in spyder it prevents you from runing it again on the same session
這個問題告訴我們,有時候真的不是程式碼寫錯了,是IDE出錯了
只要是人做的東西都可能出錯,犯錯的總不一定是自己
程式碼詳解
這個程式碼是原作者的程式碼
我自己寫的比較碎片化
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import argparse
import sys
import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
FLAGS = None
def train():
# 讀取資料
mnist = input_data.read_data_sets(FLAGS.data_dir,
one_hot=True,
fake_data=FLAGS.fake_data)
# 啟動預設回話
sess = tf.InteractiveSession()
# 建立模型
# 建立輸入資料的佔位符,分別建立特徵資料x,標籤資料y_
with tf.name_scope('input'):
x = tf.placeholder(tf.float32, [None, 784], name='x-input')
y_ = tf.placeholder(tf.float32, [None, 10], name='y-input')
# 準備輸入資料和訓練資料
# 如果train=true,從mnist.train中取一個batch樣本,設定dropout值;
# 如果train==false,獲取minist.test的測試資料,設定keep_prob為1,即保留所有神經元開啟。
def feed_dict(train):
"""Make a TensorFlow feed_dict: maps data onto Tensor placeholders."""
if train or FLAGS.fake_data:
xs, ys = mnist.train.next_batch(100, fake_data=FLAGS.fake_data)
k = FLAGS.dropout
else:
xs, ys = mnist.test.images, mnist.test.labels
k = 1.0
return {x: xs, y_: ys, keep_prob: k}
# 使用summary.image記錄圖片,要注意需要轉換成對應的格式
with tf.name_scope('input_reshape'):
image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])
tf.summary.image('input', image_shaped_input, 10)
# We can't initialize these variables to 0 - the network will get stuck.
# 我們初始預設引數,不能設定為零,初始化為零會難以收斂
# w 採用 truncated_normal 函式進行初始化一個標準差的正態分佈
# b 0.1初始化就可以
def weight_variable(shape):
"""Create a weight variable with appropriate initialization."""
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
"""Create a bias variable with appropriate initialization."""
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
# 記錄每一次迭代的引數資訊
def variable_summaries(var):
"""Attach a lot of summaries to a Tensor (for TensorBoard visualization)."""
with tf.name_scope('summaries'):
mean = tf.reduce_mean(var)
tf.summary.scalar('mean', mean)
# 記錄引數的標準差
with tf.name_scope('stddev'):
stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean)))
tf.summary.scalar('stddev', stddev)
tf.summary.scalar('max', tf.reduce_max(var))
tf.summary.scalar('min', tf.reduce_min(var))
tf.summary.histogram('histogram', var)
# 構建神經網路
# 應該明確輸入引數
# input_tensor:特徵資料
# input_dim:輸入資料的維度大小
# output_dim:輸出資料的維度大小( = 隱層神經元個數)
# layer_name:名稱空間
# act = tf.nn.relu:啟用函式(預設是relu)
# 該神經網路是一個MLP多層神經網路,每一層會對模型引數進行資料彙總tf.summary.histogram
def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu):
"""Reusable code for making a simple neural net layer.
It does a matrix multiply, bias add, and then uses ReLU to nonlinearize.
It also sets up name scoping so that the resultant graph is easy to read,
and adds a number of summary ops.
"""
# Adding a name scope ensures logical grouping of the layers in the graph.
with tf.name_scope(layer_name):
# This Variable will hold the state of the weights for the layer
with tf.name_scope('weights'):
weights = weight_variable([input_dim, output_dim])
variable_summaries(weights)
with tf.name_scope('biases'):
biases = bias_variable([output_dim])
variable_summaries(biases)
# y = wx +b
with tf.name_scope('Wx_plus_b'):
preactivate = tf.matmul(input_tensor, weights) + biases
tf.summary.histogram('pre_activations', preactivate)
# 呼叫激勵函式對資料進行響應
# result = relu(y)
activations = act(preactivate, name='activation')
tf.summary.histogram('activations', activations)
return activations
# 隱藏層 輸入資料維度784 輸出維度500
hidden1 = nn_layer(x, 784, 500, 'layer1')
# dropout 隨機刪除一些神經元,引數 keep_prob
with tf.name_scope('dropout'):
keep_prob = tf.placeholder(tf.float32)
tf.summary.scalar('dropout_keep_probability', keep_prob)
dropped = tf.nn.dropout(hidden1, keep_prob)
# 輸出層 輸入資料500維 輸出類別 10
y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity)
# 建立損失函式 y 模型輸出 y_ 資料標籤
with tf.name_scope('cross_entropy'):
diff = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y)
with tf.name_scope('total'):
cross_entropy = tf.reduce_mean(diff)
tf.summary.scalar('cross_entropy', cross_entropy)
# 使用AdamOptimizer優化器訓練模型,最小化交叉熵損失
with tf.name_scope('train'):
train_step = tf.train.AdamOptimizer(FLAGS.learning_rate).minimize(
cross_entropy)
# 計算準確率,並用tf.summary 進行合併
with tf.name_scope('accuracy'):
with tf.name_scope('correct_prediction'):
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
with tf.name_scope('accuracy'):
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
tf.summary.scalar('accuracy', accuracy)
# 將所有summary合併,這個直接在後面的session.run()裡面執行
merged = tf.summary.merge_all()
# 日誌資料存放位置
# 定義兩個不同的檔案記錄器,分別存放訓練和測試資料的日誌資料,這樣就可以不互相干擾
train_writer = tf.summary.FileWriter(FLAGS.log_dir + '/train', sess.graph)
test_writer = tf.summary.FileWriter(FLAGS.log_dir + '/test')
tf.global_variables_initializer().run()
# Train the model, and also write summaries.
# Every 10th step, measure test-set accuracy, and write test summaries
# All other steps, run train_step on training data, & add training summaries
# 開始訓練,每十次,進行一次資料彙總,並列印測試資料的準確率,並將測試資料中的引數寫入日誌
# 每100次,記錄元資訊
for i in range(FLAGS.max_steps):
if i % 10 == 0: # Record summaries and test-set accuracy
summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False))
test_writer.add_summary(summary, i)
print('Accuracy at step %s: %s' % (i, acc))
else: # Record train set summaries, and train
if i % 100 == 99: # Record execution stats
run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata()
summary, _ = sess.run([merged, train_step],
feed_dict=feed_dict(True),
options=run_options,
run_metadata=run_metadata)
# 記錄訓練運算時間和記憶體佔用
train_writer.add_run_metadata(run_metadata, 'step%03d' % i)
train_writer.add_summary(summary, i)
print('Adding run metadata for', i)
else: # Record a summary
summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True))
train_writer.add_summary(summary, i)
# 關閉檔案記錄器
train_writer.close()
test_writer.close()
def main(_):
if tf.gfile.Exists(FLAGS.log_dir):
tf.gfile.DeleteRecursively(FLAGS.log_dir)
tf.gfile.MakeDirs(FLAGS.log_dir)
train()
# 初始化引數
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--fake_data', nargs='?', const=True, type=bool,
default=False,
help='If true, uses fake data for unit testing.')
parser.add_argument('--max_steps', type=int, default=100000,
help='Number of steps to run trainer.')
parser.add_argument('--learning_rate', type=float, default=0.001,
help='Initial learning rate')
parser.add_argument('--dropout', type=float, default=0.9,
help='Keep probability for training dropout.')
# 這個要注意 主要要在自己的資料夾裡面新建好
parser.add_argument('--data_dir', type=str, default='input_data',
help='Directory for storing input data')
parser.add_argument('--log_dir', type=str, default='logs',
help='Summaries logs directory')
FLAGS, unparsed = parser.parse_known_args()
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)