Training a neural network involves an **iterative process** in which the model gradually improves by adjusting its **weights and biases** to minimize the loss function. This process is known as **gradient-based optimization**, and it follows a structured algorithm.  

The dataset is first passed through the network multiple times in a loop, where each complete pass is referred to as an **epoch**. During each epoch, the data is **shuffled** to prevent the model from learning patterns based on the order of the training examples.  Shuffling helps introduce **randomness**, leading to a more robust model.

For each training example, the model performs **forward propagation**, where inputs pass through the network, layer by layer, producing an output. This output is then compared to the actual target value to compute the **loss**.

Next, the model applies **backpropagation** and updates the weights and biases in each layer to reduces the loss.  

This process repeats for **multiple epochs**, allowing the network to refine its parameters gradually. As training progresses, the network learns to make increasingly accurate predictions. However, careful tuning of hyperparameters such as the **learning rate** is crucial to ensure stable and efficient training.  

Training picture

The **learning rate (η)** determines the step size in weight updates. If it is too high, the model might overshoot the optimal values and fail to converge. If it is too low, training becomes slow and might get stuck in a suboptimal solution. Choosing an appropriate learning rate balances **speed** and **stability** in training.

Typical values range from **0.001 to 0.1**, depending on the problem and network size.

The graph below shows how an appropriate learning rate enables the loss to **decrease steadily** at an optimal pace:

Finally, **stochastic gradient descent (SGD)** plays a vital role in training efficiency. Instead of computing weight updates after processing the entire dataset, SGD updates the parameters after each individual example. This makes training **faster** and introduces slight variations in updates, which can help the model escape local minima and reach a better overall solution.

## The fit() Method 

The `fit()` method in the `Perceptron` class is responsible for training the model using **stochastic gradient descent**.
```python
def fit(self, training_data, labels, epochs, learning_rate):
    # Iterating over multiple epochs
    for epoch in range(epochs):
        # Shuffling the data  
        indices = np.random.permutation(training_data.shape[0])
        training_data = training_data[indices]
        labels = labels[indices]
        # Iterating through each training example
        for i in range(training_data.shape[0]):
            inputs = training_data[i, :].reshape(-1, 1)
            target = labels[i, :].reshape(-1, 1)

            # Forward propagation
            output = ...

            # Computing the gradient of the loss function w.r.t. output
            da = ...

            # Backward propagation through all layers
            for layer in self.layers[::-1]:
                da = ...
```

Neural networks are powerful algorithms inspired by the structure of the human brain that are used to solve complex machine learning problems. You will build your own Neural Network from scratch to understand how it works. After this course, you will be able to create neural networks for solving classification and regression problems using the scikit-learn library.

First, we will discuss what a neural network is and how it works. And also consider the scope of its application.

Next, we will try to build our own neural network and see how efficiently it copes with learning. We will also consider a ready-made solution from the scikit-learn library.

Finally, we will give you some additional useful information on how to understand which model to use and what types of neural networks there are. To complete the course, you will be tested on your acquired knowledge.

Introduction to Neural Networks

Model Training

The fit() Method