TensorFlow模型和權重的儲存

因為大肥狼在使用儲存的模型和權重進行預測時遇到了一些問題，所以本文將介紹如何在TensorFlow中儲存模型和權重，並如何使用儲存的模型和權重來進行預測。

1.程式碼

我們的程式碼主要是吳恩達作業第二單元第三週-----tensorflow入門這一部分。連結如下
https://blog.csdn.net/u013733326/article/details/79971488

2.權重與模型的儲存

這部分比較簡單，參考連結 https://blog.csdn.net/liangyihuai/article/details/78515913
主要是使用tf.train.Saver()來儲存神經網路的網路結構圖和相關變數。這裡我新建了一個checkpoint資料夾用來儲存模型和權重檔案。
使用的程式碼如下

saver = tf.train.Saver() 

Tensorflow變數的作用範圍是在一個session裡面。在儲存模型的時候，應該在session裡面通過save方法儲存。其中sess是會話名稱，./checkpoint/model.ckpt是你儲存路徑下模型名字。

saved_path = saver.save(sess,'./checkpoint/model.ckpt' ) 

完整程式碼

def model(X_train, Y_train, X_test, Y_test, learning_rate = 0.0001,
          num_epochs = 1500,
 
 minibatch_size = 32, print_cost = True):
    ops.reset_default_graph()                         # to be able to rerun the model without overwriting tf variables
    tf.set_random_seed(1)                             # to keep consistent results
    seed = 3                                          # to keep consistent results
 

    (n_x, m) = X_train.shape                          # (n_x: input size, m : number of examples in the train set)
    n_y = Y_train.shape[0]                            # n_y : output size
    costs = []                                        # To keep track of the cost
    
    # Create Placeholders of shape (n_x, n_y)
    X, Y = create_placeholders(n_x, n_y)
    
    # Initialize parameters
    parameters = initialize_parameters()
    
    # Forward propagation: Build the forward propagation in the tensorflow graph
    Z3 = forward_propagation(X, parameters)
    tf.add_to_collection('pred_network', Z3)
    # Cost function: Add cost function to tensorflow graph
    cost = compute_cost(Z3, Y)
    
    # Backpropagation: Define the tensorflow optimizer. Use an AdamOptimizer.
    optimizer = tf.train.AdamOptimizer(learning_rate = learning_rate).minimize(cost)
    
    # Initialize all the variables
    init = tf.global_variables_initializer()

    saver = tf.train.Saver()  
    
    # Start the session to compute the tensorflow graph
    with tf.Session() as sess:
        
        # Run the initialization
        sess.run(init)
        
        # Do the training loop
        for epoch in range(num_epochs):

            epoch_cost = 0.                       # Defines a cost related to an epoch
            num_minibatches = int(m / minibatch_size) # number of minibatches of size minibatch_size in the train set
            seed = seed + 1
            minibatches = random_mini_batches(X_train, Y_train, minibatch_size, seed)

            for minibatch in minibatches:

                # Select a minibatch
                (minibatch_X, minibatch_Y) = minibatch
                
                # IMPORTANT: The line that runs the graph on a minibatch.
                # Run the session to execute the "optimizer" and the "cost", the feedict should contain a minibatch for (X,Y).
                _ , minibatch_cost = sess.run([optimizer, cost], feed_dict={X: minibatch_X, Y: minibatch_Y})
                
                epoch_cost += minibatch_cost / num_minibatches

            # Print the cost every epoch
            if print_cost == True and epoch % 100 == 0:
                print ("Cost after epoch %i: %f" % (epoch, epoch_cost))
                
            if print_cost == True and epoch % 5 == 0:
                costs.append(epoch_cost)
        #儲存模型
		saved_path = saver.save(sess,'./checkpoint/model.ckpt' ) 
        
        # plot the cost
        plt.plot(np.squeeze(costs))
        plt.ylabel('cost')
        plt.xlabel('iterations (per tens)')
        plt.title("Learning rate =" + str(learning_rate))
        plt.show()

        # lets save the parameters in a variable
        parameters = sess.run(parameters)
        print ("Parameters have been trained!")

        # Calculate the correct predictions
        correct_prediction = tf.equal(tf.argmax(Z3), tf.argmax(Y))

        # Calculate accuracy on the test set
        accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

        print ("Train Accuracy:", accuracy.eval({X: X_train, Y: Y_train}))
        print ("Test Accuracy:", accuracy.eval({X: X_test, Y: Y_test}))
        
        return parameters

這裡的例子比較複雜，包括你也可以設定每100代儲存一次權重，建議參考連結 https://blog.csdn.net/liangyihuai/article/details/78515913

儲存完後，你的檔案目錄下的checkpoint 資料夾下會多出四個檔案。如下圖所示
model.meta為模型檔案，model.data-00000-of-00001為資料檔案。後面我們恢復模型需要用到。

使用模型和權重進行預測

接下來，我們講預測時如何使用模型和權重進行預測。可以參考這個連結 https://blog.csdn.net/luoyexuge/article/details/78209670 ，寫的很簡潔。
1.匯入儲存好的模型結構，即model.meta檔案

    meta_path = './checkpoint/model.ckpt.meta'  
    saver = tf.train.import_meta_graph(meta_path) # 匯入

2.載入的（變數）引數：
使用restore()方法恢復模型的變數引數。

    data_path = './checkpoint/model.ckpt'  
    saver.restore(sess,data_path) # 匯入變數值  

也可以使用

    data_path = './checkpoint/'  
    saver.restore(sess,tf.train.latest_checkpoint(data_path)) # 匯入變數值  

注意這裡 data_path 裡面的路徑： ‘./checkpoint/model.ckpt’

3.使用graph.get_tensor_by_name()根據張量名稱獲取你想要的張量。因為我在訓練的時候設定佔位符時，將我的輸入張量名字定為x，如下圖所示。

def create_placeholders(n_x, n_y):
    X = tf.placeholder(tf.float32, shape=[n_x, None],name="x")
    Y = tf.placeholder(tf.float32, shape=[n_y, None],name="y")
    return X, Y

所以恢復輸入張量

x=graph.get_operation_by_name('x').outputs[0]

3.因為前向傳播過程中，沒有為我們計算的輸出y_hat設定張量。前向傳播中，我們只計算到z3，所以我們用tf.add_to_collection() 來載入一下你預測要用的引數。這裡我需要Z3，我就在訓練程式碼中收集Z3,並給他命名為’pred_network’。

    Z3 = forward_propagation(X, parameters)
    tf.add_to_collection('pred_network', Z3)

然後預測時恢復

   z3=tf.get_collection("pred_network")[0]

然後在會話中計算z3

    z3=sess.run(z3, feed_dict={x:my_image})

完整程式碼如下

import numpy as np
import tensorflow as tf
import scipy
from PIL import Image
from scipy import ndimage
import matplotlib.pyplot as plt
fname = "images/thumbs_up.jpg"
image = np.array(ndimage.imread(fname, flatten=False))
my_image = scipy.misc.imresize(image, size=(64,64)).reshape((1, 64*64*3)).T
with tf.Session() as sess:  
    meta_path = './checkpoint/model.ckpt.meta'  
    model_path = './checkpoint/model.ckpt'  
    saver = tf.train.import_meta_graph(meta_path) # 匯入圖  
    saver.restore(sess,model_path) # 匯入變數值  
    graph = tf.get_default_graph()
    x=graph.get_operation_by_name('x').outputs[0]
    print(x)
    z3=tf.get_collection("pred_network")[0]
    z3=sess.run(z3, feed_dict={x:my_image})
    y=np.argmax(z3)
    print(str(np.squeeze(y)))

這樣就完成預測啦。耶耶耶。寫的不夠詳細，因為太費時間了。。。
來自七七