Forward and Backward Propagation

Forward Propagation

Backward Propagation

After the neural network has made its forward propagation prediction, we can compare that prediction with the real data and calculate the network error. Backpropagation is the process in which this error information is used to traverse the network back and adjust the weights of the neurons. Basically, we tell the network, "Here's where you went wrong, let's fix it." Based on this information, the network error is reduced and it becomes more accurate in its predictions.

Note

The neural network error can be calculated in different ways depending on the task, but it is always a floating point number.

The learning process of a neural network is the repetition of these two stages (forward and back propagation) many times. With each iteration, the network gets smarter and smarter as it learns more about the data and how to process it to make accurate predictions.

It is important to understand that this process does not end when the network reaches "perfect accuracy" or an ideal state, because such a state does not exist. Instead, training usually stops when the network reaches an acceptable level of accuracy, or when it stops improving even after many training iterations.