OpenAI just quietly rolled out GPT-4o Long Output to alpha users. It's an experimental version of GPT-4o that can output up to 64,000 tokens in a single request. That's four times more than their previous limit and even more so when compared to the 4,000 or 8,000 token limits of many of their competitors.
In the world of AI language models, a token is basically a piece of a word. Common words might be a single token, while longer or less common words might be split into multiple tokens. As a rough estimate, one token is about four characters in English.
Context length, on the other hand, refers to how much information the AI can consider at once - both the input it receives and the output it generates. GPT-4o mini has a context window of 128K tokens and supports up to 16K output tokens per request. Essentially, it allows you to use half of the model's entire context for generating responses.
One use case for long output capability is in data transformation tasks. Imagine translating lengthy documents from one language to another or extracting detailed structured data from extensive texts. These are scenarios where you need every input token reflected in the output, making the 64,000 output token limit a game-changer.
These longer completions do come at a slightly higher cost. OpenAI has adjusted the pricing to $6.00 per million input tokens and $18.00 per million output tokens to reflect the increased computational load. Regular GPT-4o is $5.00 per million input tokens and $15.00 per million output tokens.