Logistic Regression Basics - utkaln/machine-learning GitHub Wiki

Logistic Regression

This is the most famous algorithm used for basic classification
Similar to Linear Regression it uses a graph but is showed as a S graph called Sigmoid Function, which is a logistic function
X axis stays between negative to positive number, Y axis runs between 0 to 1
Equation: g(z) = 1 / (1 + e ** -z) where z = w * x + b
The output of that is returned as Probability of being 1
Another alternate notation of the Logistic Regression is : f(x) = P( y = 1 | x ; w , b)

This is the point beyond which the prediction turns 1 and below is 0
This point on the function g is called Decision Boundary
If no clue about data, then use 0.5 as Decision Boundary. However based on business case increase or decrease number

For prediction to be 1 this must be true : g(z) >= 0.5
g(z) >= 0.5 only when z >= 0
We also know that z = w.x + b
Hence w.x + b >= 0 predicts the decision to be 1 otherwise 0
For more complex functions higher order polynomial functions can be used instead of the simple linear example above. A polynomial function example - f(x) = g(x**2 + x -1)

The cost function used in linear regression can not be used here as the prediction function f(x) is sigmoid function, that would distort the shape of gradient descent graph
Hence a logarithmic function fits better as a cost function