Cohere for AI Launches Aya Expanse to Help Close the Language Gap

Cohere for AI Launches Aya Expanse to Help Close the Language Gap

Cohere for AI, the research arm of Cohere, has unveiled a new multilingual family of models called Aya Expanse. The models can understand and process 23 languages more effectively than similar tools from Google, Meta, and Mistral – despite often using fewer computing resources.

The company is releasing (open weights) the 8 billion and 32 billion parameter models on Kaggle and Hugging Face. The launch builds on years of research and collaboration, including contributions from over 3,000 researchers from 119 countries, all focused on developing AI that works across a wide range of languages.

The larger Aya Expanse 32B model outperforms competitors like Gemma 2 27B, Mistral 8x22B, and Meta's Llama 3.1 70B—even the latter, a model over twice its size. Aya Expanse 8B also competes effectively in its parameter range, such as against Gemma 2 9B, with win rates surpassing 60%.

Aya Expanse's success lies in its foundation of novel research approaches, from an innovative synthetic data strategy termed "data arbitrage" to advances in preference training and model merging—all aimed at improving multilingual capabilities. Data arbitrage draws on the idea of finding the right "teacher" for each task, using different high-quality teacher models to generate effective synthetic data, reducing the risk of model collapse—a common challenge for low-resource languages.

Cohere's approach to preference training is another key aspect of Aya Expanse's development. Typically, models are fine-tuned based on human feedback (RLHF), but these standards often lean heavily on Western-centric perspectives. Aya Expanse, by contrast, incorporates feedback across multilingual contexts to ensure more inclusive, contextually aware outputs. This adjustment enhances the model's quality and safety, making it more adaptable and sensitive to a diverse global user base.

Model merging, a technique where multiple candidate models are combined during training, further bolsters Aya Expanse's versatility. By merging these candidate models, Cohere aims to create a solution that shines across all supported languages.

For researchers interested in working with these models, both versions are available starting today through Cohere's API platform, as well as on Kaggle and Hugging Face. The company is also offering research grants and compute resources to support further development of multilingual AI systems.

With Aya Expanse, Cohere is aiming for a more inclusive future, where AI models excel in languages that have traditionally been underserved. This could lead to more equitable access to AI capabilities across different languages and cultures worldwide.

Chris McKay is the founder and chief editor of Maginative. His thought leadership in AI literacy and strategic AI adoption has been recognized by top academic institutions, media, and global brands.

Let’s stay in touch. Get the latest AI news from Maginative in your inbox.

Subscribe