LAB4 - Hiresh12/UMKC GitHub Wiki
BIG DATA ANALYTICS AND APPLICATIONS – LAB 4
Hiresh Jakkala Bhaskar – 8
Anvesh Mandadi – 17
AIM :
- To Develop a show and model using the dataset
- I used images that has objects like table, door, kitchen, jacket
- This dataset is used to train the model so that we can identify the indoor activities described in the image
- This is help us in track the activities of children and pets in home
- Model is trained using features from CNN and SIFT
- To make the model to tell caption for the given test images
- To compute the performance metrics of the model using BLEU.
Step Followed:
- Pre-processing the dataset to generate lemma for the captions, splitting the dataset into train and test data.
- Extract the features from the train images and captions
- Train the model using CNN, SIFT and LSTM (features (CNN+SIFT)->relu->image embedding->dropout->LSTM)
- Save the model
- Generate captions for the test data and calculate the performance metrics of the model
Code Implementation:
-
Extracting features - CNN
-
SIFT Feature Extraction and store it in sift_feature.pkl file
- Loading Captions and creating Vocabulary
- Develop Model
- Generate Captions for the test data
- Computing performance of the model