Now that we are done with the theory for Logistic Regression. We'll be implementing a model using only numpy and no other external Machine Learning libraries in Python.

We'll develop a Logistic Regression model with a Neural Network Mindset. Now there is no need to dive into the intricacies of a Neural Network. But to give you some intuition, a plain neural network consists a node (aka neuron which performs the computation), inputs to the node along with weights, the node then generates the output.

1. Read the data
2. Multiply the data with weights and add a bias to get the net input function (let's call it z)
3. Pass z through an activation function
4. Calculate the loss and adjust the weights accordingly.
5. Repeat

Notice that finding the net input activation function is similar to getting a linear regression equation. The only difference is that we feed this linear equation to an activation function.

For our model we'll be using the sigmoid activation function also referred to as the sigmoid function. Interesting fact about the sigmoid activation function is that it's range always lies between (0,1).

As the function is continous it is differentiable.

The formula is given as:

f(z) = 1/(1+e^-z)

The loss function (cost function) is given by:

L(a,y) = -ylog(a) - (1-y)(log(1-a))

Where:

y is the actual value.

a is the predicted value.

We need to adjust the weights so for that we need to compute dL/dZ .

For that we can apply chain rule. The image below shows the derivation.

Thus we get dz as:

    dz = A - Y

Now using the value of Dz we can obtain Dw (gradient of loss wrt to weights) and Db (gradient of loss wrt bias)

dw = (1/m)*(np.dot(X,Dz.T))
db = (1/m)*(np.sum(A-Y))