AI research institute Ai2 has released OLMo 2, the latest in their family of open source language models. Unlike popular open-weights models like Llama and Gemma, OLMo (which stands for Open Language Model) provides the weights, tools, datasets, training recipes, and everything else used to develop these models, making them truly open in every sense.
The model family includes 7B (base, instruct) and 13B (base, instruct) parameter versions. Both show impressive capabilities. The 7B version outperforms Meta's Llama 3.1 8B on English academic benchmarks. The 13B version surpasses Qwen 2.5 7B, despite using less computing power for training.
The new release builds on the foundation set by the first OLMo model released earlier this year. The Ai2 team used an innovative two-stage training approach. They first trained the model on a large dataset of 3.9 trillion tokens. They then refined it with high-quality data from academic content, math workbooks, and instruction sets.
The team paid special attention to training stability. They made key changes to prevent performance drops during long training runs. These changes included improvements to the model's architecture and training process.
This release builds on Ai2's recent work with Tülu 3, their open-source training system. Tülu 3 is a sophisticated post-training process that makes OLMo 2 capable of instruction-following tasks on par with some of the best models in the world.
The complete release includes evaluation frameworks and intermediate checkpoints. These tools allow researchers to understand and improve upon OLMo 2's capabilities.
Researchers and developers can access OLMo 2 through Ai2's playground or download it from Hugging Face. The model's Apache license 2.0 means anyone can study, modify, or build upon it without restrictions.