LLMs, as we know, are pretrained for natural language processing (NLP). Such pre-trained models have certain weights assigned to the tokens. The model is further trained to improve its performance for a new or for a specific task. The model can also be trained to adapt to a new domain.

LLMs are fine tuned on a new data set to make them improve for a specific task. The specific task could be translation or summarization or question answering. The pre-trained model has been trained on vast dataset, whereas for fine tuning we will use a smaller dataset relevant to the specific task, say question answering.

There are various techniques of fine tuning. The most common is the use of supervised learning. The model is trained on labelled dataset. In question answering, the dataset would consist of pairs of questions and answers. The model is trained to predict the correct answer of each question.

A model can be fine tuned by training it from scratch. However this is time-consuming and computationally expensive. A model can be fine tuned by freezing some of the layers of the model. Here we will not revise weights of the model. That prevents overfitting. In partial fine tuning, only a subset of parameters are adjusted.

Fine tuning is an effective way to make the model more efficient. However, fine tuning could also lead to over-fitting. Overfitting means the model has mastered the specific details too well, and hence it loses the capacity to generalize for the new data. To prevent over-fitting, we can use smaller learning rate, regularization and early stopping.