機器人命名MC胖虎，目前只是最簡單粗暴的方法，使用tensorflow完成，有些像人工智障，符合胖虎的人物設定，看一些效果：

本文以講解寫詩機器人實現為主，不會講太多理論和tensorflow使用方法，好下面開始。

訓練資料預處理

採用3w首唐詩作為訓練資料，在github上dataset資料夾下可以看到，唐詩格式為”題目：詩句“，如下所示：

我們首先通過”：“將題目和內容分離，然後做資料清洗過濾一些不好的訓練樣本，包含特殊符號、字數太少或太多的都要去除，最後在詩的前後分別加上開始和結束符號，用來告訴LSTM這是開頭和結尾，這裡用方括號表示。

poems = []
file = open(filename, "r")
for line in file:  #every line is a poem
    #print(line)
    title, poem = line.strip().split(":")  #get title and poem
    poem = poem.replace(' ','')
    if '_' in poem or '《' in poem or '[' in poem or '(' in poem or '（' in poem:
        continue
    if len(poem) < 10 or len(poem) > 128:  #filter poem
        continue
    poem = '[' + poem + ']' #add start and end signs
    poems.append(poem)
 

然後統計每個字出現的次數，並刪除出現次數較少的生僻字

#counting words
allWords = {}
for poem in poems:
    for word in poem:
        if word not in allWords:
            allWords[word] = 1
        else:
            allWords[word] += 1
# erase words which are not common
erase = []
for key in allWords:
    if allWords[key] < 2:
        erase.append(key)
for key in erase:
    del allWords[key]
 

根據字出現的次數排序，建立字到ID的對映。為什麼需要排序呢？排序後的ID從一定程度上表示了字的出現頻率，兩者之間有一定關係，比不排序直接對映更容易使模型學出規律。

新增空格字元，因為詩的長度不一致，需要用空格填補，所以留出空格的ID。最後將詩轉成字向量的形式。

wordPairs = sorted(allWords.items(), key = lambda x: -x[1])
words, a= zip(*wordPairs)
words += (" ", )
wordToID = dict(zip(words, range(len(words)))) #word to ID
wordTOIDFun = lambda A: wordToID.get(A, len(words))
poemsVector = [([wordTOIDFun(word) for word in poem]) for poem in poems] # poem to vector
 

接下來構建訓練batch，每一個batch中所有的詩都要補空格直到長度達到最長詩的長度。因為補的都是空格，所以模型可以學出這樣一個規律：空格後面都是接著空格。X和Y分別表示輸入和輸出，輸出為輸入的錯位，即模型看到字得到的輸出應該為下一個字。

這裡注意一定要用np.copy，坑死我了！

#padding length to batchMaxLength
batchNum = (len(poemsVector) - 1) // batchSize
X = []
Y = []
#create batch
for i in range(batchNum):
    batch = poemsVector[i * batchSize: (i + 1) * batchSize]
    maxLength = max([len(vector) for vector in batch])
    temp = np.full((batchSize, maxLength), wordTOIDFun(" "), np.int32)
    for j in range(batchSize):
        temp[j, :len(batch[j])] = batch[j]
    X.append(temp)
    temp2 = np.copy(temp) #copy!!!!!!
    temp2[:, :-1] = temp[:, 1:]
    Y.append(temp2)

搭建模型

搭建一個LSTM模型，後接softmax，輸出為每一個字出現的概率。這裡對著LSTM模板抄一份，改改引數就好了。

with tf.variable_scope("embedding"): #embedding
    embedding = tf.get_variable("embedding", [wordNum, hidden_units], dtype = tf.float32)
    inputbatch = tf.nn.embedding_lookup(embedding, gtX)

basicCell = tf.contrib.rnn.BasicLSTMCell(hidden_units, state_is_tuple = True)
stackCell = tf.contrib.rnn.MultiRNNCell([basicCell] * layers)
initState = stackCell.zero_state(np.shape(gtX)[0], tf.float32)
outputs, finalState = tf.nn.dynamic_rnn(stackCell, inputbatch, initial_state = initState)
outputs = tf.reshape(outputs, [-1, hidden_units])

with tf.variable_scope("softmax"):
    w = tf.get_variable("w", [hidden_units, wordNum])
    b = tf.get_variable("b", [wordNum])
    logits = tf.matmul(outputs, w) + b

probs = tf.nn.softmax(logits)

模型訓練

先定義輸入輸出，構建模型，然後設定損失函式、學習率等引數。

gtX = tf.placeholder(tf.int32, shape=[batchSize, None])  # input
gtY = tf.placeholder(tf.int32, shape=[batchSize, None])  # output
logits, probs, a, b, c = buildModel(wordNum, gtX)
targets = tf.reshape(gtY, [-1])
#loss
loss = tf.contrib.legacy_seq2seq.sequence_loss_by_example([logits], [targets],
                                                          [tf.ones_like(targets, dtype=tf.float32)], wordNum)
cost = tf.reduce_mean(loss)
tvars = tf.trainable_variables()
grads, a = tf.clip_by_global_norm(tf.gradients(cost, tvars), 5)
learningRate = learningRateBase
optimizer = tf.train.AdamOptimizer(learningRate)
trainOP = optimizer.apply_gradients(zip(grads, tvars))
globalStep = 0

然後開始訓練，訓練時先尋找能否找到檢查點，找到則還原，否則重新訓練。然後按照batch一步步讀入資料訓練，學習率逐漸遞減，每隔幾個step就儲存一下模型。

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    saver = tf.train.Saver()
    if reload:
        checkPoint = tf.train.get_checkpoint_state(checkpointsPath)
        # if have checkPoint, restore checkPoint
        if checkPoint and checkPoint.model_checkpoint_path:
            saver.restore(sess, checkPoint.model_checkpoint_path)
            print("restored %s" % checkPoint.model_checkpoint_path)
        else:
            print("no checkpoint found!")

    for epoch in range(epochNum):
        if globalStep % learningRateDecreaseStep == 0: #learning rate decrease by epoch
            learningRate = learningRateBase * (0.95 ** epoch)
        epochSteps = len(X) # equal to batch
        for step, (x, y) in enumerate(zip(X, Y)):
            #print(x)
            #print(y)
            globalStep = epoch * epochSteps + step
            a, loss = sess.run([trainOP, cost], feed_dict = {gtX:x, gtY:y})
            print("epoch: %d steps:%d/%d loss:%3f" % (epoch,step,epochSteps,loss))
            if globalStep%1000==0:
                print("save model")
                saver.save(sess,checkpointsPath + "/poem",global_step=epoch)

自動寫詩

在自動寫詩之前，我們需要定義一個輸出概率對應到單詞的功能函式，為了避免每次生成的詩都一樣，需要引入一定的隨機性。不選擇輸出概率最高的字，而是將概率對映到一個區間上，在區間上隨機取樣，輸出概率大的字對應的區間大，被取樣的概率也大，但胖虎也有小概率會選擇其他字。因為每一個字都有這樣的隨機性，所以每次作出的詩都完全不一樣。

def probsToWord(weights, words):
    """probs to word"""
    t = np.cumsum(weights) #prefix sum
    s = np.sum(weights)
    coff = np.random.rand(1)
    index = int(np.searchsorted(t, coff * s)) # large margin has high possibility to be sampled
    return words[index]

然後開始寫詩，首先仍然是構建模型，定義相關引數，載入checkpoint。

gtX = tf.placeholder(tf.int32, shape=[1, None])  # input
logits, probs, stackCell, initState, finalState = buildModel(wordNum, gtX)
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    saver = tf.train.Saver()
    checkPoint = tf.train.get_checkpoint_state(checkpointsPath)
    # if have checkPoint, restore checkPoint
    if checkPoint and checkPoint.model_checkpoint_path:
        saver.restore(sess, checkPoint.model_checkpoint_path)
        print("restored %s" % checkPoint.model_checkpoint_path)
    else:
        print("no checkpoint found!")
        exit(0)

生成generateNum這麼多首詩，每首詩以左中括號開始，以右中括號或空格結束，每次生成的prob用probsToWord方法轉成字。

poems = []
for i in range(generateNum):
    state = sess.run(stackCell.zero_state(1, tf.float32))
    x = np.array([[wordToID['[']]]) # init start sign
    probs1, state = sess.run([probs, finalState], feed_dict={gtX: x, initState: state})
    word = probsToWord(probs1, words)
    poem = ''
    while word != ']' and word != ' ':
        poem += word
        if word == '。':
            poem += '\n'
        x = np.array([[wordToID[word]]])
        #print(word)
        probs2, state = sess.run([probs, finalState], feed_dict={gtX: x, initState: state})
        word = probsToWord(probs2, words)
    print(poem)
    poems.append(poem)

還可以寫藏頭詩，前面的搭建模型，載入checkpoint等內容一樣，作詩部分，每遇到標點符號，人為控制下一個輸入的字為指定的字就可以了。需要注意，在標點符號後，因為沒有選擇模型輸出的字，所以需要將state往前滾動一下，直接跳過這個字的生成。

flag = 1
endSign = {-1: "，", 1: "。"}
poem = ''
state = sess.run(stackCell.zero_state(1, tf.float32))
x = np.array([[wordToID['[']]])
probs1, state = sess.run([probs, finalState], feed_dict={gtX: x, initState: state})
for c in characters:
    word = c
    flag = -flag
    while word != ']' and word != '，' and word != '。' and word != ' ':
        poem += word
        x = np.array([[wordToID[word]]])
        probs2, state = sess.run([probs, finalState], feed_dict={gtX: x, initState: state})
        word = probsToWord(probs2, words)

    poem += endSign[flag]
    # keep the context, state must be updated
    if endSign[flag] == '。':
        probs2, state = sess.run([probs, finalState],
                                 feed_dict={gtX: np.array([[wordToID["。"]]]), initState: state})
        poem += '\n'
    else:
        probs2, state = sess.run([probs, finalState],
                                 feed_dict={gtX: np.array([[wordToID["，"]]]), initState: state})

print(characters)
print(poem)

大約在GPU上訓練20epoch效果就不錯了！

估計後續還會出看圖寫詩機器人-MC胖虎2.0

說了這麼多胖虎該生氣了！