OpenAI announced on 4 April 2024 a series of updates to its fine-tuning API and custom models program, expanding access to model customisation capabilities [source].

The fine-tuning API now supports epoch-based training, allowing developers to specify the number of training cycles rather than relying solely on automatic early stopping. OpenAI also introduced third-party integration support for platforms including Weights & Biases, enabling users to track training metrics and model performance outside OpenAI's dashboard.

The company expanded its Custom Models program, which provides assisted fine-tuning for organisations requiring deeper customisation than the self-service API permits. This program now includes access to additional model variants and dedicated support from OpenAI's technical teams. Pricing remains custom-quoted based on compute requirements and dataset size.

OpenAI stated that fine-tuning is now available for GPT-4 in addition to GPT-3.5 Turbo, though access to GPT-4 fine-tuning requires approval and remains limited to select customers. The company did not specify approval criteria or expected wait times.

The announcement follows reports from developers that fine-tuning jobs occasionally failed without clear error messages, and that model performance after fine-tuning sometimes degraded on tasks outside the training distribution. OpenAI did not address these issues in the blog post.

Fine-tuning allows organisations to adapt foundation models to domain-specific tasks by training on proprietary datasets. The technique can improve accuracy on narrow tasks but may reduce general capability or introduce unexpected behaviours if training data contains biases or errors.