RNN - AshokBhat/ml GitHub Wiki

Recurrent neural network (RNN)

• Deep Learning Network suited for sequential data processing.
• Example - LSTMs, GRUs and Bidirectional RNNs,

Applications

• Unsegmented, connected handwriting recognition
• Speech recognition
• Sentiment analysis

BERT vs RNN

Feature	BERT	RNN
Type of model	Transformer	Recurrent neural network
Learning mechanism	Self-attention	Backpropagation through time
Ability to learn long-term dependencies	Good	Excellent
Flexibility	Flexible	Less flexible
Computation time	More computationally expensive	Less computationally expensive
Applications	Natural language processing, machine translation, question answering	Natural language processing, machine translation, time series forecasting, speech recognition
Pros	Good at understanding context, flexible, can be used for a variety of tasks	Good at learning long-term dependencies, efficient, can be used for a variety of tasks
Cons	Computationally expensive, can be difficult to interpret	Less flexible, can be difficult to train

See also

• CNN
• LSTM networks