Module 2_ICP 3: Word Embedding - acikgozmehmet/PythonDeepLearning GitHub Wiki
Module 2_ICP 3: Word Embedding
Objectives:
The following topics are covered.
- Overfitting
- Dropout
- Data augmentation
- Callbacks
- Cost/Loss Functions
- Word Embeddings (representation of text data)
Overview
Why do we use word embeddings in NLP ?
Natural language processing (NLP) is a sub-field of machine learning (ML). Dealing with text data is problematic, since our computers, scripts and machine learning models can’t read and understand text in any human sense
Word embedding is the collective name for a set of language modeling and feature learning techniques in natural language processing (NLP) where words or phrases from the vocabulary are mapped to vectors of real numbers. Conceptually it involves a mathematical embedding from a space with many dimensions per word to a continuous vector space with a much lower dimension.
Methods to generate this mapping include neural networks, dimensionality reduction on the word co-occurrence matrix, probabilistic models,explainable knowledge base method, and explicit representation in terms of the context in which words appear.
Word and phrase embeddings, when used as the underlying input representation, have been shown to boost the performance in NLP tasks such as syntactic parsing and sentiment analysis.
In Class Programming
1. In the code provided there are three mistake which stop the code to get run successfully; find those mistakes and explain why they need to be corrected to be able to get the code run
Click here to get the source code
Please find below the 3 corrections done in the code. input_dim: is the max review length, in this case it is 2470
We have to define 3 neurons ( there are 3 target types- neg, pos, unsup) in the output layer with "softmax" activation.
The "softmax" is used for classifications with more than 3 classes, namely it is used for multi-class classification.
2. Add embedding layer to the model, did you experience any improvement?
Click here to get the source code
By adding embedding layer to the model, the accuracy of the model increases. However, it requires a lot of computing power to do the iterations. With my limited computed power, I could only run 3 epocs.
Bonus Questions
3. Apply the code on 20_newsgroup data set we worked in the previous classes
Click here to get the source code
