Optimizer are algorithms used iteratively to update the parameters of a model based on the gradients of the loss function. The aim is to minimize the loss and enhance the functioning of the model.

The commonly used optimizers in deep learning are Stochastic Gradient Descent (SGD), Momentum, Adam, RMSProp, Adadelta, Adagrad, Adam, Adam and Admax. It is necessary to choose an appropriate optimizer. Each optimizer has its own strengths and weaknesses. The choice depends on the specific characteristics of the problem — the size of the dataset and the complexity of the model.