French AI startup, Mistral, is making it easier for you to fine-tune their models. Fine-tuning AI models can be a game-changer for businesses and developers. It makes models smarter and more relevant to your specific use case. However, the process can be expensive and technically demanding, putting it out of reach for some organizations. Mistral’s new offerings aim to lower these barriers.
The company has introduced three options for fine-tuning:
- Open-source fine-tuning SDK: Developers who want to fine-tune Mistral's models on their own infrastructure can now use
mistral-finetune, a lightweight codebase optimized for efficient memory usage and performance. Built on the LoRA training paradigm, it enables fine-tuning of all Mistral's open-source models without compromising on speed or resource utilization. - Serverless fine-tuning on la Plateforme: Mistral's AI development platform now offers fine-tuning services that leverage the company's proprietary techniques. These services use LoRA adapters to prevent models from forgetting base knowledge while allowing for efficient deployment. Currently compatible with Mistral 7B and Mistral Small, more models will be supported in the coming weeks.
- Custom training services: For select customers, Mistral provides tailored fine-tuning of its models using the customer's proprietary data. This enables the creation of highly specialized models optimized for specific domains, with advanced techniques like continuous pretraining to incorporate proprietary knowledge into the model weights.
The launch of these fine-tuning capabilities comes amidst Mistral's rapid growth. Over the last several months, Mistral has been rapidly building out its commercial offerings–releasing proprietary models, developer resources, and hosted solutions. The startup released its first model in September 2023, and is reportedly now seeking to raise $600 million at a $6 billion valuation.