RNN -- 循環神經網路

1. RNN 的訓練要用 BPTT (Back Propagation Through Time)
2. 學習的時候要做 Clipping ，限制梯度範圍，這樣才不會一下飛出去！(導致 NaN : Not a Number 發生)

影片

• 周莫烦:什么是循环神经网络 RNN (深度学习)? What is Recurrent Neural Networks (deep learning)?
  • 什么是 LSTM RNN 循环神经网络 (深度学习)? What is LSTM in RNN (deep learning)?
• 李宏毅 - ML Lecture 21-1: Recurrent Neural Network (Part I)
  • ML Lecture 21-2: Recurrent Neural Network (Part II)

文章：

1. The Unreasonable Effectiveness of Recurrent Neural Networks
2. Understanding LSTM Networks

Attention -- hierarchical neural attention encoder

• The fall of RNN / LSTM