Model Compilation

After constructing a neural network model in Keras, the next crucial step is model compilation.

This step is necessary because it defines how the model should update during training and how it should evaluate its performance.

There are three key components of model compilation:

Optimizer: determines how the network will be updated based on the loss function. It implements the specific variant of the gradient descent algorithm (backpropagation step).
Loss function: measures how well the model is performing. A model aims to minimize this function.
Metrics: used to monitor the training and testing steps. Unlike the loss function, metrics are not used for training the model but for evaluating its performance.

The Adam Optimizer

Adam combines the advantages of two other optimizers: momentum (helps to navigate along relevant directions and smoothens the journey) and RMSprop (adjusts the learning rate for each parameter).

It is often chosen due to its efficiency and minimal requirement for tuning.

Loss Functions

During training, the primary goal is to minimize this loss function. This is achieved by adjusting the model's weights through backpropagation.

Here are some of the most popular loss functions:

The choice of the appropriate loss function depends on the nature of the problem (classification, regression, etc.) and the specific requirements of the dataset and task.

Metrics

Metrics are used to evaluate the performance of the model. Unlike loss functions, they are not used for training the model but rather for monitoring during training and testing. Metrics provide insight into how well the model is performing according to specific criteria.

Here are some of the most popular metrics:

Example

The basic syntax for model compilations is:

python

We'll compile the model we created in the previous chapter using the Adam optimizer. Let's assume that we are solving a classification problem where the output is a probability, so this value must be in the range from 0 to 1.

To achieve this, we must include an additional fully connected layer (the output layer) containing a single neuron. This layer utilizes a sigmoid activation function (created as an Activation layer), ensuring that the final output is constrained within the range of 0 to 1.

A common choice for an optimizer is Adam, so we'll stick with it. To set it as the optimizer for the model, import the Adam class from the tensorflow.keras.optimizers module and pass its instance to the optimizer parameter.

Since the model performs binary classification, 'binary_crossentropy' will be used as the loss function, and 'accuracy' is a good choice as a baseline metric showing the percentage of correct predictions.


              12345678910111213141516171819202122
            
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation, Input
from tensorflow.keras.optimizers import Adam

# Creating a model
model = Sequential()

# Old layers
model.add(Input(shape=(2,)))
model.add(Dense(3))
model.add(Activation('relu'))

# New layers
model.add(Dense(1))
model.add(Activation('sigmoid'))

# Compiling the model
model.compile(optimizer=Adam(),
              loss='binary_crossentropy',
              metrics=['accuracy'])

model.summary()

1. In the provided example, why is the `sigmoid` activation function used in the output layer?

2. How do the roles of loss functions and metrics differ in the context of model compilation and training?

¿Todo estuvo claro?

¡Gracias por tus comentarios!

Sección 1. Capítulo 4

Neural Networks with TensorFlow