Quiz

Adam

SGD

AdaGrad

Nadam

Increases computational cost.

Reduces model capacity.

Enables the model to generalize across tasks.

It allows for the loading of batches concurrently with the processing of the current data.

It prefetches data to GPU memory for faster access.

It increases the size of the dataset.

It reduces the need for shuffling.

By increasing the learning rate after a certain number of epochs.

By keeping the learning rate constant throughout training.

By reducing the learning rate based on a predefined function.

By randomly adjusting the learning rate at each epoch.

Fine-tuning completely replaces a part of a pre-trained model with a new model built from scratch.

Fine-tuning involves unfreezing some of the pre-trained model's layers and training them along with a new classifier on the new dataset.

Fine-tuning only trains the first few layers of the pre-trained model while keeping the rest frozen.

It prevents the optimizer from making any updates to the model weights.

It helps in accelerating the optimizer in the right direction and dampens oscillations.

Momentum makes the learning rate increase exponentially.

It keeps the learning rate constant, independent of the gradient.

It allows the model to leverage knowledge from pre-trained models.

Transfer learning only works with unlimited data.

It guarantees high accuracy on any task.

By resetting the learning rate at each epoch.

By implementing a constant learning rate throughout the training.

By using a moving average of squared gradients to adjust the learning rate.

RMSprop does not address the diminishing learning rates problem.