Lab Assignment 11 - nikky4222/BigDataSpring2017 GitHub Wiki

Big Data Assignment 11

---

Student id:16231555
Class id:18
Name:Lakshmi Nikitha Kona

---

Task 1
Write a TensorFlow program for the following Task.
a.Generate captions for your own dataset that is not covered in class tutorial using the model provided
.b.Generate 4 captions for each image.

---

Show And Tell Model

The Show and Tell model is an example of an encoder-decoder neural network. It works by first "encoding" an image into a fixed-length vector representation, and then "decoding" the representation into a natural language description.

The image encoder is a deep convolutional neural network. This type of network is widely used for image tasks and is currently state-of-the-art for object recognition and detection. Our particular choice of network is the Inception v3 image recognition model pretrained on the ILSVRC-2012-CLS image classification dataset.

The decoder is a long short-term memory (LSTM) network. This type of network is commonly used for sequence modeling tasks such as language modeling and machine translation. In the Show and Tell model, the LSTM network is trained as a language model conditioned on the image encoding.

Words in the captions are represented with an embedding model. Each word in the vocabulary is associated with a fixed-length vector representation that is learned during training.
Below are the pictures and the corresponding captions achieved from the show & tell model.
Image1


Captions

Image2


Captions

Image3


Captions


Task 2
Web based Q and A
a.Develop a web based question and answering system similar to that covered in class
b.Develop Q & A that is related to your project.

Installed on Heroku


Question & Answer


Web Simulator


Tried Implementing Q & A based on the code given in the class but couldnt retrieve results