Module 2_ICP 2: Basics in Keras (2) - acikgozmehmet/PythonDeepLearning GitHub Wiki

Basics in Keras (2)

Objectives:

The following topics are covered.

  1. Neural Network
  2. Backpropagation
  3. Activation Function
  4. Linear Regression
  5. Cost/Loss Functions
  6. Gradient Descent (Optimization Algorithm) and Learning Rate

Overview

What is Artificial Neural Networks (ANN) ?

Artificial neural networks (ANN) or connectionist systems are computing systems vaguely inspired by the biological neural networks that constitute animal brains.

The data structures and functionality of neural nets are designed to simulate associative memory. Neural nets learn by processing examples, each of which contains a known "input" and "result," forming probability-weighted associations between the two, which are stored within the data structure of the net itself. (The "input" here is more accurately called an input set, since it generally consists of multiple independent variables, rather than a single value.) Thus, the "learning" of a neural net from a given example is the difference in the state of the net before and after processing the example. After being given a sufficient number of examples, the net becomes capable of predicting results from inputs, using the associations built from the example set. If a feedback loop is provided to the neural net about the accuracy of its predictions, it continues to refine its associations, resulting in an ever-increasing level of accuracy. In short, there is a direct relationship between the number and diversity of examples processed by a neural net and the accuracy of its predictions. This is why a neural net gets "better" with use. What is interesting about neural nets is that because they are indiscriminate in the way they form associations, they can form unexpected associations, and reveal relationships and dependencies that were not previously known.

What is the content of this module?

In this lesson, we are going to have an introduction to Deep Learning programming on Keras. Before to that, we will introduce some of the applications of the Deep Learning in the area of vision and NLP.

In Class Programming

Click here to get the source code

1. Using the history object in the source code, plot the loss and accuracy for both training data and validation data.

2. Plot one of the images in the test data, and then do inferencing to check what is the prediction of the model on that single image in the test data

3. We had used 2 hidden layers and Relu activation. Try to change the number of hidden layer and the activation to tanh or sigmoid and see what happens

When we add some more hidden layers, we see that the accuracy seems to increase.

4. Run the same code without scaling the images, how the accuracy changes?

When we run the same code without scaling the data (without normalizing the data), it takes relatively longer time for learning and convergence. So we can say that normalizing the data generally speeds up learning and leads to faster convergence. There is a decrease in the accuracy as well.

References