OpenAI announced on 14 December 2021 that it would allow developers to customize GPT-3 models through fine-tuning, enabling applications to achieve better performance with shorter prompts and lower latency [source].

The company stated that fine-tuning had been available in beta and was now generally available for all developers using the API. According to the announcement, fine-tuning allows developers to train GPT-3 on their own data, creating models tailored to specific use cases. OpenAI cited examples including content generation, summarization, and classification tasks where fine-tuned models outperformed base models.

The process requires developers to prepare training data in a specific format—pairs of prompts and completions—then submit it through the API. OpenAI reported that fine-tuning typically requires "a few hundred" examples to see meaningful improvements, though some applications benefit from thousands of examples.

OpenAI stated that fine-tuned models remain private to the developer who created them. The company also noted that fine-tuning costs vary based on the number of training tokens and the base model selected, with pricing details available in the API documentation.

The announcement included guidance on when fine-tuning is appropriate versus when prompt engineering alone suffices. OpenAI recommended starting with prompt design and moving to fine-tuning only when base models fail to meet performance requirements.

This capability represents a shift in how developers can interact with GPT-3, moving from purely prompt-based interactions to models trained on domain-specific data. The announcement did not address potential risks of fine-tuning on biased or problematic datasets, nor did it specify content moderation policies for fine-tuned model outputs.