2. Transformers VS. SimpleTransformers - DianaMoyano1/NLP-Sentiment_Extraction_Challenge GitHub Wiki

All you need is a really advanced neural networks

AI & Machine learning has developed quickly in the last several years. Deep learning techniques have provided some very powerful algorithms that can 'automatically' learn to identify patterns in unstructured data.

Below is high-level progression of neural networks:

Type of Neural Network	Complexity	Application
Multi-layer perception (MLP)	Simple	Simple pattern recognition such as 'load default predictions'
Convolutional	Medium	Image classification, like classifying PNGs of dogs and cats
Recurrent neural network (RNN)	Medium	Predicting results based on a sequence of data, like text language translation
Transformers (the new kid on the block)	Crazy	Analyzing sequences of data and identifying relationships, like text extraction

"Dammit, Jim. I am a doctor not a computer science Ph.D!"

To implement standard transformer libraries is fairly involved and requires many lines of code. Instead Simple Transformers library can be used to get up and running quickly. Often in business we are under pressure to deliver something quickly, this library can be used to create a powerful transformer solution with only 3 lines of code:

https://pypi.org/project/simpletransformers/

Below is a code sample from our project that gives you idea of how simple it is to use:

#ROOT = '/gdrive/My Drive/Colab Notebooks/Models For Ensemble' 
NAME_OF_MODEL3 = 'roberta-large_B' 
MODEL_ARCHITECTURE3 = 'roberta' 

FULL_PATH3 = join(ROOT, NAME_OF_MODEL3)

#Change the workspace to the model folder
%cd '{FULL_PATH3}' 

#Load the model's arguments list (required to setup the existing model) 
with open('args_train.json') as json_file: 
    train_args3 = json.load(json_file) 
loaded_model3 = QuestionAnsweringModel(MODEL_ARCHITECTURE3, 'outputs/', args=train_args3, use_cuda=use_cuda)

Transfer Learning

The model loaded above will be a pre-trained model.

The SimpleTransformers library has numerous pre-trained libraries. Each model is trained for different types of text. For example, there is uncased vs cased, so if you think upper or lower case letters will help in your model then you would choose a corresponding model. Also, some models are larger than others, i.e. they take longer to train but are probably more accurate.

You will need to train the model on your specific data, i.e. transfer learning. Essentially, the model is already pretty good, but you will need to refine it to match your specific use case. In our example, we need to train it to understand Tweets and predict the sentiment.