In this tutorial, we are going to understand fine-tuning in a practical approach. Let's see some code and then try to understand them.
from openai import OpenAI
client = OpenAI(api_key="sk-W-I-wont-tell-1")
client = OpenAI(api_key="sk-W-I-wont-tell-1") This line creates an instance of the OpenAI class and assigns it to the variable client. The api_key parameter is used to authenticate your requests to the OpenAI API. You should replace "sk-W-I-wont-tell-1" with your actual API key, which you can obtain from the OpenAI website after creating an account.
training_file = client.files.create(
file=open("./training_dataset.jsonl", "rb"),
purpose="fine-tune"
)
training_file
#Output
FileObject(id='file-uevn2DC6rwZnY0v1JtuEnqFm', bytes=40883, created_at=1714361042, filename='training_dataset.jsonl', object='file', purpose='fine-tune', status='processed', status_details=None)
These lines help upload a file to OpenAI's servers to fine-tune a language model. The client.files.create method is used to create a new file upload.
If you visit: https://platform.openai.com/storage/ you should notice your file being uploaded.
Similarly, upload the validation file:
validation_file = client.files.create(
file=open("./validation_dataset.jsonl", "rb"),
purpose="fine-tune"
)
validation_file
Time for the beast to reveal. Now, we can use the fine_tuning.jobs.create to create a fine-tuning job.
fine_tuning = client.fine_tuning.jobs.create(
training_file=training_file.id,
model="gpt-3.5-turbo",
hyperparameters = {"n_epochs":2},
validation_file=validation_file.id
)
fine_tuning
client.fine_tuning.jobs.create is a method provided by the OpenAI Python library to initiate a fine-tuning job. We specify the training file id, model name, validation file id, and the hyperparameters e.g. number of epochs(iterations over the training data).
Now, if you visit: https://platform.openai.com/finetune you should see the fine-tuning process is in progress. This is the time to relax, give some break to your eyes and look 20 feet away for at least 2-3 minutes. Then come back.
If you are too excited like me, then either you can continue looking at the screen or use the below code to get the status of the latest fine-tuning job from the API.
client.fine_tuning.jobs.list(limit=1)
#Output
SyncCursorPage[FineTuningJob](data=[FineTuningJob(id='ftjob-FQactRhfB3LxA20kpLLt6VfD', created_at=1714361064, error=Error(code=None, message=None, param=None), fine_tuned_model=None, finished_at=None, hyperparameters=Hyperparameters(n_epochs=2, batch_size=1, learning_rate_multiplier=2), model='gpt-3.5-turbo-0125', object='fine_tuning.job', organization_id='org-TidatKQQuCG5KXnffDy2UX3w', result_files=[], seed=1890037888, status='validating_files', trained_tokens=None, training_file='file-uevn2DC6rwZnY0v1JtuEnqFm', validation_file='file-deYC7B7CVsxl4FuzlA9d8RPl', integrations=[], user_provided_suffix=None, estimated_finish=None)], object='list', has_more=True)
Keep monitoring the fine-tuning process in the OpenAI platform. Here are some things that you should keep in mind.
Training Loss:
The training loss should gradually decrease as the model learns from the training data. This indicates the model is improving its ability to represent the relationships within the data and make accurate predictions.
Validation Loss:
The validation loss is used to monitor the model's performance on unseen data. Ideally, the validation loss should also decrease over time, but not necessarily at the same rate as the training loss.
A stagnating validation loss, even with decreasing training loss, might suggest overfitting. This means the model is memorizing the training data instead of learning generalizable patterns.
We should consider this points:
- Early Stopping: If the validation loss starts to increase significantly, it's a sign of overfitting. A common technique called early stopping can be used to stop training before this happens. Early stopping monitors the validation loss and halts training when it starts to rise, preventing overfitting.
- Learning Rate Multiplier: The learning rate is a hyperparameter that controls how much the model updates its weights during training. Adjusting the learning rate schedule can sometimes help the model converge to a lower loss. For example, a decaying learning rate can help prevent overfitting in later stages of training.
Once the fine-tuning process is complete, you can use the OpenAI API by specifying the fine-tuned model we just cooked.
import json
from openai import OpenAI
client = OpenAI(api_key="sk-W-I-wont-tell-1")
response = client.chat.completions.create(
model="ft:gpt-3.5-turbo-0125:nofoobar::9JC2TBQr",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": "Hi boss, get the job done by your ego from tomorrow."},
]
)
json.loads(response.choices[0].message.content)