Model Compilation

After constructing a neural network model in Keras, the next crucial step is model compilation. Model compilation in Keras is the process of configuring the model for training.

It involves specifying the optimizer, loss function, and metrics you want to monitor. This step is necessary because it defines how the model should update during training and how it should evaluate its performance.

Key Components of Model Compilation

Optimizer: Determines how the network will be updated based on the loss function. It implements the specific variant of the gradient descent algorithm (backpropagation step).
Loss Function: Measures how well the model is performing. A model aims to minimize this function.
Metrics: Used to monitor the training and testing steps. Unlike the loss function, metrics are not used for training the model but for evaluating its performance.

The Adam Optimizer

Adam, short for Adaptive Moment Estimation, is one of the most popular optimization algorithms in deep learning. It's an extension of stochastic gradient descent that has been proven to be effective in various types of neural networks.

Adam combines the advantages of two other optimizers: Momentum (helps to navigate along relevant directions and smoothens the journey) and RMSprop (adjusts the learning rate for each parameter).

Adam is often chosen due to its efficiency and minimal requirement for tuning.

Loss Functions

The loss function is a measure of the model's error or inaccuracy. It quantifies how far the model's predictions are from the actual target values.

During training, the primary goal is to minimize this loss function. This is achieved by adjusting the model's weights through backpropagation.

Here's the table of the most popular loss functions:

The choice of the appropriate loss function depends on the nature of the problem (classification, regression, etc.) and the specific requirements of the dataset and task.

Metrics

Metrics are used to evaluate the performance of the model. Unlike loss functions, they are not used for training the model but rather for monitoring during training and testing. Metrics provide insight into how well the model is performing according to specific criteria.

Here's the table of the most popular metrics:

Note
Keras offers a wide array of metrics and loss functions beyond those listed; the ones mentioned are simply the most commonly utilized.

Example

The basic syntax for model compilations is:

Let’s compile the model we created in the previous chapter using the Adam optimizer. Let's assume that we are solving a regression problem where the output must be in the range from 0 to 1.

To achieve this, we must include an additional layer (the output layer) containing a single neuron. This neuron utilizes a sigmoid activation function, ensuring that the final output is constrained within the range of 0 to 1.


              12345678910111213141516171819202122
            
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation, Input
from tensorflow.keras.optimizers import Adam

# Create a model
model = Sequential()

# Old layers
model.add(Input(shape=(100,)))
model.add(Dense(units=64))
model.add(Activation('relu'))

# New layers
model.add(Dense(1))
model.add(Activation('sigmoid'))

# Compile the model
model.compile(optimizer=Adam(),
              loss='mean_squared_error',
              metrics=['mean_squared_error', 'mean_absolute_error'])

model.summary()

In this example, we use:

Optimizer: We are using Adam. This choice suits a wide range of problems and is generally a good starting point.
Loss Function: For a regression problem we use mean_squared_error as the loss function. This is a standard choice for regression tasks as it measures the average of the squares of the errors between predicted and actual values.
Metrics: The metrics included are mean_squared_error and mean_absolute_error. Mean squared error gives an idea of the magnitude of error, while mean absolute error provides a direct interpretation of how far the predictions are from the actual values on average.
Output Layer: The sigmoid activation function is used in the output layer to constrain the output between 0 and 1, which is suitable for the problem statement where the output is expected in this range.

Tudo estava claro?

Seção 1. Capítulo 3