Home - utkaln/machine-learning GitHub Wiki

Welcome to the machine-learning wiki!

Links to Topics

• Linear Regression Basics
• Linear Regression Feature-Scaling
• Logistic Regression (Classification) Basics
• Concepts-of-Fitness-and-Regularization
• Neural-Network-Basics
• Neural-Network-Advanced
• Basics of Matrix-Multiplication
• Model-Evaluation-and-Selection-Steps
• Transfer-Learning---Neural-Network
• Decision Tree | Random Forest | XGBoost
• Hyperparameters
• K-Means-Clustering
• Anomaly-Detection
• Recommendation-System

Supervised Vs. Unsupervised Learning

Supervised	Unsupervised
Training Set has a known output, based on which new data predicts	There is no output, rather models predict similar dataset to a group or pattern
Popular Algorithms: Linear Regression, Classification, Neural Network, Decision Tree	Clustering, Regenerative
Problems Solved: Predict Continuous Number, Discrete Value, Detection, Computer Vision, Text Analysis	Recommendation, Text Generation, Art Generation

Supervised Learning

Neural Network - Types of Activation

Name	Representation	Pros	Cons
Linear Regression	f(x) = w*x + b	Expected output is continuous number value	Slow to find Gradient Descent
Logistic Regression (sigmoid)	f(x) = 1/ (1 + np.exp(-(w*x + b)	Expected output has binary classification	Slow to find Gradient Descent
ReLU	f(x) = max (0, (w*x + b))	Expected output has only zero or positive values (no negative number output)	Attention required to ensure negative output is not a real possibility

• Other less known activation functions are :
  • tan h
  • LeakyReLU
  • Swish

Choice of Model (Activation Function)

• This is more applicable to Neural Network
• The choice is dependent on the output of the layer

Output type	Layer Type	Choice of Activation
Regression	Outer (Final)	Linear Regression
Classification	Outer (Final)	Logistic Regression
Regression with no Negative value	Outer (Final)	ReLU
Any Type	Hidden (middle)	ReLU

Steps of Training Data for Prediction

• Contrast between Logistic Regression (Using Numpy) vs. Neural Network (using Tensorflow)
• This example uses Logistic Regression for illustration purpose. Other than defining model every other step stays the same regardless of what activation function is chosen

Step	Logistic Regression	Neural Network
1. Define Model	z = np.dot(w,x) + b f_x = 1/ (1+ np.exp(-z)	model = Sequential( [ Dense() Dense() Dense() ... ] )
2. Compute Loss and Cost (avg. Loss)	loss = -y * np.log(f_x) - (1-y) * np.log(1 - f_x)	model.compile(loss = BinaryCrossentropy())
3. Minimize Loss (Gradient Descent)	w = w - alpha * dj_dw b = b - alpha * dj_db	model.fit(X,y, epochs=100)

---

Unsupervised Learning

K-means Cluster Algorithm

• Step 1 : Choose how many means or centroids to choose (k). The centroids have the same dimensions as of the sample dataset, which is denoted as (n)
• Step 2 : Start the loop to first Assign points (m) to the nearest cluster centroid c[i] up to cluster count (k)
• Step 3 : In the loop that started in the previous step, move the Cluster Centroid c[i] to the mean of the points in the previous cluster
• Exception : In case a centroid has no data associated, then eliminate that from the centroid calculation in the next round. Or reinitialize the centroids by randomizing