Outputs: output, (h_n, c_n)

 

• output (seq_len, batch, hidden_size * num_directions): tensor containing the output features (h_t) from the last layer of the RNN, for each t. If a torch.nn.utils.rnn.PackedSequence has been given as the input, the output will also be a packed sequence.
• h_n (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t=seq_len
• c_n (num_layers * num_directions, batch, hidden_size): tensor containing the cell state for t=seq_len

renamed num_layers （有幾層LSTM/GRU疊加）to w.

output comprises all the hidden states in the last layer ("last" depth-wise, not time-wise). 

(h_n, c_n) comprises the hidden states after the last timestep, t = n, so you could potentially feed them into another LSTM.

加入num_layers=1,那麼output和hidden相等。

LOSS和logP(y|x)的區別：

decoder每一步輸出的是單詞詞典中每一個單詞的概率。就是，下一個是什麼單詞，候選集裡每個單詞的都有可能。

目標是模型的輸出接近真實值，所以

輸出序列是依賴引數的，計算出目標輸出的所對應的引數表示函式集合。（回憶考研最大似然的定義）