3. Text extraction, question answering and sentiment extraction - DianaMoyano1/NLP-Sentiment_Extraction_Challenge GitHub Wiki

Understanding Text Extraction

Transformers use a unique architecture. The video below will explain in detail how this works. The simple way to understand Transfomers is that they read the entire sentence in parallel and there is a special neural network that learns important 'things' about the sentence.

Transformers let you understand why you have a particular output.

So if our output of our model (i.e. the tranformer) is 'positive' (as in the example below). Then we might ask why? In the past with normal neural networks it would be almost impossible to work backwords into the neural network and figure out what really contributed to this output - it is just too difficult for a human. However, with new "Transformers" there is the attention mechanism, which is just a really advanced neural network that 'learns' what are the important contribution to an output. The result of the 'attention mechanism' can then be pulled out of the neural network (transformer) to understand why we got a positive response. This is outlined below:

Check out this video

Yannic Kilcher's YouTube video does an excellent job explaining transformers - BUT - you will need to have a solid understanding of recurrent neural networks to grasp the contents of this video...