Lab Assignment 5 - nikky4222/BigDataSpring2017 GitHub Wiki
Big Data Assignment 5
Student id:16231555
Class id:18
Name:Lakshmi Nikitha Kona
## Task 1
Create your own dataset for Image Classification Problem. Use the workflow as discussed in the Tutorial 4 Session using any classification algorithm (e.g., Random Forest, Naïve Bayes) excluding Decision Tree. Report the accuracy and confusion matrix obtained.
## Why Image Classification?
The intent of the classification process is to categorize all pixels in a digital image into one of several land cover classes, or "themes". This categorized data may then be used to produce thematic maps of the land cover present in an image. Normally, multispectral data are used to perform the classification and, indeed, the spectral pattern present within the data for each pixel is used as the numerical basis for categorization (Lillesand and Kiefer, 1994). The objective of image classification is to identify and portray, as a unique gray level (or color), the features occurring in an image in terms of the object or type of land cover these features actually represent on the ground.
## Steps Followed
First the data is divided to Training and Testing data.
We have divided the training data to required number of classes. Key Features are extracted using SIFT and Feature descriptor is constructed.
Kmeans clustering is performed to group similar features.
These centroids and cluster centers are passed to the bag of words to create histograms.
The histograms are given to Random tree classifier for classification.
Then the test data is used for accuracy.
## Training Data
## Image Classes
Four classes are taken as image categories.They are "dolphin","elephant","sun",flower".
## Models Generated
After performing extraction,clustering,Histogram creation and classification the models are saved.
## Test Data
Then the test data is passed to find the accuracy.
## Confusion Matrix
Finally confusion matrix is created for evaluating the performance and if the classification was done correctly.
## Task 2
Write a client application using the Spark API to connecting between Spark and your client. Your client can be either Web application or Android application. Refer to Tutorial 5 Spark API tutorial.
To establish a connection the server is executed first and it will wait for the client to get connected.
After the client gets connected the data is fetched from the server and displayed on the HTML page.
## Server Program
The below screenshots shows how the server is waiting for the client.
## Connecting to the Server
From the command prompt we are connecting to the server.
## Waiting For Client
After successful Connection the server will wait for the client.
## Client Application
After the client has connected the application predicts the image and the proccessing is done in the server program and displayed in the html page.
## Image Classification
Lets predict the sunflower.After clicking the button processing takes place.
## Result
The predicted image sun is displayed in spark console which is fetched to the client.
Similarly prediction is done for few other images.
## Image Classification For Dolphin & Elephant
Similarly its done for another image also.
## Task3Build a simple application to have a conversion using Google Conversation Actions API about the summary you had generated about your video. Refer to Tutorial 5 Conversion Actions API tutorial.
## Agent Creation
To generate the summary first we need to create an agent in API.AI.This agent helps us to create entities and intents.
This helps us to have a conversion using Google Conversation Actions API.
An agent name video has been created.Then entities and intents need to be created.
## Entity Creation
Entities represent concepts and serve as a powerful tool for extracting parameter values from natural language inputs.
The entities that are used in a particular agent will depend on the parameter values that are expected to be returned as a result of agent functioning. In other words, a developer need not create entities for every concept mentioned in the agent – only for those required for actionable data.
We have created below entities about the video in lab4.
## Intent Creation
An intent represents a mapping between what a user says and what action should be taken by your software.
Intent interfaces have the following sections:
User says
Action
Response
Contexts
We have created intents as below.
Based on the questions asked they are tagged to the coressponding entities.
## Web Simulator
To interact with the Google Conversation we need to integrate with the web simulator.It answers us based on the questions asked,by checking with the corresponding entity and intents.
## Conversation
