Neural Network Basics - utkaln/machine-learning GitHub Wiki

Shortcut to Neural-Network-Advanced

Neural Network Basics

  • Neural network can be simplified to understand as a multilayer prediction with each layer having a linear or log regression
  • Input X to a layer for computation of f(X) is the output of the previous layer

Tensorflow

  • Tensorflow provides libraries for Neural Network calculations
  • It is important to understand the difference in syntax between Num Py and Tensorflow to use the concepts _ Important : NumPy handles 1D vectors using simple array representation, but Tensorflow handles everything as 2D matrix (row, column)
  • Simple notation of calculating a layer in Tensorflow 
     # Define a neural net layer with 3 neurons, and function type as log regression
 layer_1 = Dense(units=3, activation='sigmoid')

 # Calculate prediction (output of layer 1) a1 by using input from previous layer x
 a1 = layer_1(x)
    Output is displayed as: 
    tf.Tensor([0.2 0.7 0.3](/utkaln/machine-learning/wiki/0.2-0.7-0.3), shape=(1, 3), dtype=float32)
Mathematical Representation NumPy Format Tensorflow Format
2x3 matrix: [1 2 3][4 5 6]] ](/utkaln/machine-learning/wiki/2D-vector:-np.array([[1,2,3],[4,5,6)) Tensor([1.0 2.0 3.0] [4.0 5.0 6.0](/utkaln/machine-learning/wiki/1.0-2.0-3.0]-[4.0-5.0-6.0), shape=(2, 3), dtype=float32)
row vector aka 1x2 matrix: [1 2] 2D vector: np.array([1,2]]) ](/utkaln/machine-learning/wiki/Tensor([[1.0-2.0), shape=(1, 2), dtype=float32)
simple array 1 2 1D vector: np.array([1,2]) NA

Computation using Tensorflow

Step 1: Data Setup

X = np.array([[185.32  12.69], [259.92  11.87] ...])
Y = np.array([1.],[0.](/utkaln/machine-learning/wiki/1.],[0.))

Step 2: Normalize Data

  • Fitting weights to the data will work faster if data is normalized (similar to feature scaling)
  • Normalization is done using Keras -
  • Next step after normalization is to let the model Adapt to the normalized data
  • REMEMBER to apply the same normalization to any new future data for prediction to work accurately
norm_1 = tf.keras.layers.Normalization(axis=-1)
norm_1.adapt(X)
Xn = norm_1(X)

Step 3: Replicate Data

  • Replicate training data to bigger set, so that the optimization can be found early with less number of iterations
Xt = np.tile(Xn, (1000,1))
Yt = np.tile(Y, (1000,1))

Step 4: Build Tensorflow Model

model = Sequential(
  [
    tf.keras.Input(shape=(2,)),
    Dense(3, activation='sigmoid', name='layer1')
    Dense(1, activation='sigmoid', name='layer2')
  ]
)

model.summary()

Step 5: Apply Model to Compute Loss

model.compile(
    loss = tf.keras.losses.BinaryCrossentropy(),
    optimizer = tf.keras.optimizers.Adam(learning_rate=0.01),
)

Step 6: Fit to Model (Gradient Descent)

  • Take the loss from the previous function and fit data into the model by finding Gradient Descent
model.fit(Xt, Yt, epochs=10,)

Full Code Reference Here