Fine Tune GPT Models For Quality Results

Fine-tuning OpenAI API on a specific task gives you a competitive advantage

Photo by Nathanaël Desmeules on unsplash — edited by author to add the graph

GPT3 is a language model with billions of parameters trained on the broad internet data and exceeded its successors’ performance in many benchmarks. It is powerful because it resolves the need for many training data to get satisfactory results using “few-shots” learners.

Fine-tuning is another way to interact with the model by using much more data to train the model; it helps to improve the quality of the results and reduce the latency as you don’t have to send examples with every call.

You can look to model fine-tuning like athletes with the right suit. The athlete will be in the mode for specific activity holding the right tools.

initial cost — image by author

The language models like GPT3 require a massive amount of data for pre-training to be ready for any task. However, the best results come after providing task-specific examples, a few shots, or fine-tuning the model weights on a specific domain or technical topic.

Data is king:

The model is available through a public API, and anyone who calls it with the proper parameters and examples will gain the same results! If you own vast data for your business, it will be a waste not to fine-tune the model to achieve higher quality and faster results, which gives you a competitive advantage.

Update 16 feb 2024: the code in this blog updated to supported latest openai API changes.

Validating The Model Results

sentiment analysis — image by author with icons from the noun project by Jim Lears, Romica and Michele Zamparo

I created a classification fine-tuned GPT3 model for the “Movie Review Sentiment Analysis” kaggle competition and submitted the results to the leader board to validate the fine tune impact.

The training data contains about 156,000 rows, distributed among five sentiments as follows:

sentiment classes bar visual — image by author

The classes are challenging even for humans because they include similar meanings like “negative” and “somewhat negative”. Add to this, kaggle is a competitive place, and usually, it takes weeks of dedication to secure a slot among the top ten.

With GPT3, I managed to get a 0.70865 late submission score which put my model the 3rd place in weekend implementation:

the model score leaderboard’s position — source kaggle.com

The training accuracy tracker for the model can be divided into three areas:

• Significant accuracy increase in the first 6K examples.
• Gradual enhancement until the 500K example.
• Then saturation and almost no enhancement.

GPT3 training fine tune tracker — image by author

The model used Babbage engined; I did not use the most significant OpenAI engine Davinci for cost reasons and did not apply many pre-processing techniques as my focus was on GPT3 evaluation. The model can reach higher score with the right time and cost investment.

While the results generated from a deprecated GPT 3 model, the next section code updated to support latest chatGPT API. Let me know if it produce higher results.

Let’s start coding.

Fine Tuning Steps Using Open AI API

The first step, go to the OpenAI website and create an account, then get the API key:

OpenAI API

Install the python library:

pip install openai

Import openai and set the key:

from openai import OpenAI

client = OpenAI(api_key='')

Training file:

The fine-tuning depends on the JSON file to train the model; it is a JSON format with each example in a separated line. The file should include two values:

• system role is general model instructions.
• user role is the input text.
• assistant role is the model output; you set the class value for classification.

example of jsonl file format — code by github

It is important to end each prompt message for training and testing with a unique value to indicate the end of the input, like “ ->” or “ /n/n###/n/n”.

To upload the training file:

upload file to openai api — code by github

Fine-tuning and prediction:

Call FineTune function with the training file id and the engine name:

create fine tuned model — code by github

Based on the data size, the above statement might take an extended processing time in the background. To track the status of the training, copy the printed “id” value and call the retrieve function until the “status” becomes “succeeded”:

After successful model deployment, you can copy the “fine_tuned_model” value and use it with every inference call:

call the inference function — code by github

Example to call the classifier function :

# this character --> used in the training to indicate end of input
response = gpt3_classifier(input_text + ' -->', fine_tuned_model)
print(response)

Conclusion

GPT 3 is the state-of-the-art model for natural language processing tasks, and it adds value to many business use cases. You can start interacting with the model through OpenAI API with minimum investment. However, adding the effort to fine-tune the model helps get substantial results and improves model quality. The following post will focus on GPT3 model performance tracking and manipulating the hyper-parameters.

This article covered all the steps to fine-tune GPT3, assuming no previous knowledge. However, If you are new to OpenAI API, follow the intro tutorial for in-depth details: Few lines to build advanced OpenAI model

References

• Kaggle data [link].
• Language Models are Few-Shot Learners [link].
• OpenAI fine tuning documentation [link].