Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.Learn Challenge: Training the Perceptron | Neural Network from Scratch
Introduction to Neural Networks
course content

Course Content

Introduction to Neural Networks

Introduction to Neural Networks

1. Concept of Neural Network
What is a Neural Network?Neural Networks or Traditional ModelsApplications of Deep Learning in the Real WorldNeural Network StructureWhat is a Neuron?Activation FunctionsForward and Backward Propagation
2. Neural Network from Scratch
Single Neuron ImplementationChallenge: Creating a NeuronPerceptron LayersChallenge: Creating a PerceptronForward PropagationLoss FunctionBackward PropagationBackpropagation ImplementationModel TrainingChallenge: Training the PerceptronModel EvaluationChallenge: Evaluating the PerceptronNeural Network with scikit-learnBackward Propagation
3. Model Training and Evaluation
Data Preprocessing
4. Conclusion
Other types of Neural NetworksChallenge: Manual Hyperparameter TuningChallenge: Automatic Hyperparameter TuningQuiz: Neural Networks EssentialsSummary

book
Challenge: Training the Perceptron

Before proceeding with training the perceptron, keep in mind that it uses the binary cross-entropy loss function discussed earlier. The final key concept before implementing backpropagation is the formula for the derivative of this loss function with respect to the output activations, an. Below are the formulas for the loss function and its derivative:

To verify that the perceptron is training correctly, the fit() method also prints the average loss at each epoch. This is calculated by averaging the loss over all training examples in that epoch:

Finally, the formulas for computing gradients are as follows:

The sample training data along with the corresponding labels are stored as NumPy arrays in the utils.py file. Additionally, instances of the activation functions are also defined there:

Task

Swipe to start coding

  1. Compute the following gradients: dz, d_weights, d_biases, and da_prev in the backward() method of the Layer class.
  2. Compute the output of the model in the fit() method of the Perceptron class.
  3. Compute da (dan) before the loop, which is the gradient of the loss with respect to output activations.
  4. Compute da and perform backpropagation in the loop by calling the appropriate method for each of the layers.

If you implemented training correctly, given the learning rate of 0.01, the loss should steadily decrease with each epoch.

Solution

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Everything was clear?

How can we improve it?

Thanks for your feedback!

Section 2. Chapter 10
toggle bottom row

book
Challenge: Training the Perceptron

Before proceeding with training the perceptron, keep in mind that it uses the binary cross-entropy loss function discussed earlier. The final key concept before implementing backpropagation is the formula for the derivative of this loss function with respect to the output activations, an. Below are the formulas for the loss function and its derivative:

To verify that the perceptron is training correctly, the fit() method also prints the average loss at each epoch. This is calculated by averaging the loss over all training examples in that epoch:

Finally, the formulas for computing gradients are as follows:

The sample training data along with the corresponding labels are stored as NumPy arrays in the utils.py file. Additionally, instances of the activation functions are also defined there:

Task

Swipe to start coding

  1. Compute the following gradients: dz, d_weights, d_biases, and da_prev in the backward() method of the Layer class.
  2. Compute the output of the model in the fit() method of the Perceptron class.
  3. Compute da (dan) before the loop, which is the gradient of the loss with respect to output activations.
  4. Compute da and perform backpropagation in the loop by calling the appropriate method for each of the layers.

If you implemented training correctly, given the learning rate of 0.01, the loss should steadily decrease with each epoch.

Solution

Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
Everything was clear?

How can we improve it?

Thanks for your feedback!

Section 2. Chapter 10
Switch to desktopSwitch to desktop for real-world practiceContinue from where you are using one of the options below
We're sorry to hear that something went wrong. What happened?
some-alt