Logistic Regression with a Neural Network mindset
General Architecture of the learning algorithm
It’s time to design a simple algorithm to distinguish cat images from noncat images.
I will build a Logistic Regression, using a Neural Network mindset. The following Figure explains why Logistic Regression is actually a very simple Neural Network!
Mathematical expression of the algorithm:
For one example Missing superscript or subscript argument x^{(i)} :
The cost is then computed by summing over all training examples:
Key steps:
In this exercise, I will carry out the following steps:
 Initialize the parameters of the model
 Learn the parameters for the model by minimizing the cost
 Use the learned parameters to make predictions (on the test set)
 Analyse the results and conclude
Building the parts of algorithm
The main steps for building a Neural Network are:
Define the model structure (such as number of input features)
Initialize the model’s parameters
Loop:
Calculate current loss (forward propagation)
Calculate current gradient (backward propagation)
 Update parameters (gradient descent)
You often build 13 separately and integrate them into one function we call model()
.
Helper functions
sigmoid
Using code from “Python Basics”, implement sigmoid()
. As we seen in the figure above, I will compute $sigmoid( w^T x + b) = \frac{1}{1 + e^{(w^T x + b)}}$ to make predictions. I will use np.exp().
1 

initialize_with_zeros
1 

Forward and Backward propagation
Now that our parameters are initialized, we can do the “forward” and “backward” propagation steps for learning the parameters.
Exercise: Implement a function propagate()
that computes the cost function and its gradient.
Hints:
Forward Propagation:
 I get X
 I compute $A = \sigma(w^T X + b) = (a^{(1)}, a^{(2)}, …, a^{(m1)}, a^{(m)})$
 I calculate the cost function: $J = \frac{1}{m}\sum_{i=1}^{m}y^{(i)}\log(a^{(i)})+(1y^{(i)})\log(1a^{(i)})$
Here are the two formulas I will be using:
1 

Optimization
 I have initialized our parameters.
 We are also able to compute a cost function and its gradient.
 Now, I want to update the parameters using gradient descent.
Exercise: Write down the optimization function. The goal is to learn $w$ and $b$ by minimizing the cost function $J$. For a parameter $\theta$, the update rule is $ \theta = \theta  \alpha \text{ } d\theta$, where $\alpha$ is the learning rate.
1 

predict
Exercise: The previous function will output the learned w and b. We are able to use w and b to predict the labels for a dataset X. Implement the predict()
function. There are two steps to computing predictions:
Calculate $\hat{Y} = A = \sigma(w^T X + b)$
Convert the entries of a into 0 (if activation <= 0.5) or 1 (if activation > 0.5), stores the predictions in a vector
Y_prediction
. If you wish, you can use anif
/else
statement in afor
loop (though there is also a way to vectorize this).
1 

What to remember
We’ve implemented several functions that:
 Initialize (w,b)
 Optimize the loss iteratively to learn parameters (w,b):
 computing the cost and its gradient
 updating the parameters using gradient descent
 Use the learned (w,b) to predict the labels for a given set of examples
Merge all functions into a model
You will now see how the overall model is structured by putting together all the building blocks (functions implemented in the previous parts) together, in the right order.
Exercise: Implement the model function. Use the following notation:
 Y_prediction_test for your predictions on the test set
 Y_prediction_train for your predictions on the train set
 w, costs, grads for the outputs of optimize()
1 

train
1  d = model(train_set_x, train_set_y, test_set_x, test_set_y, num_iterations=2000, learning_rate=0.005, print_cost=True) 
test our image
1  ## START CODE HERE ## (PUT YOUR IMAGE NAME) 