Stability AI, the company behind the groundbreaking Stable Diffusion image model, has launched StableLM, a new suite of open-source language models that will drive innovation in text and code generation. The Alpha version of StableLM, released today, includes 3 billion and 7 billion parameter models, with 15 billion to 65 billion parameter versions in the pipeline.
In a blog post announcing the release, Stability AI highlighted the importance of transparent, accessible, and supportive AI technology. By making StableLM freely available under a CC BY-SA-4.0 license, the company aims to democratize access to AI and foster trust among researchers, developers, and end-users alike.
Stability AI's new language models are trained on a dataset containing 1.5 trillion tokens. Despite their relatively small size, the 3 billion and 7 billion parameter StableLM models have demonstrated high performance in conversational and coding tasks, thanks in part to the richness of their training dataset.
In addition to the base StableLM models, Stability AI is releasing research models that have been fine-tuned for instruction. These models use a combination of five recent open-source datasets for conversational agents, including Alpaca, GPT4All, Dolly, ShareGPT, and HH. These fine-tuned models are available under a non-commercial license for research purposes.
The models are available in the company’s GitHub repository. Stability AI said it will publish full technical details on the models and dataset in the future.
Language models will form the backbone of our digital economy, and we want everyone to have a voice in their design
StableLM joins a growing roster of state-of-the-art language models that are revolutionizing the AI landscape, including OpenAI's GPT-4, Meta's LLaMA, Anthropic's Claude and Google's PaLM. Stability AI hopes to differentiate itself and "demonstrate [their] commitment to AI technology that is transparent, accessible, and supportive.” In a statement, the company said "language models will form the backbone of our digital economy, and we want everyone to have a voice in their design.”
By open-sourcing their models, Stability AI enables researchers to scrutinize their performance, develop interpretability techniques, and identify potential risks and safeguards. The company believes that this transparency ensures that organizations can fine-tune the models for their applications without sacrificing control over their AI capabilities or exposing sensitive data.
The company highlights that its models can be run on local devices, thus enabling developers to create independent applications that work with widely available hardware. They believe this approach broadens the economic benefits of AI and facilitates the development of safety and interpretability techniques. Additionally, the company is launching a crowd-sourced RLHF program, and plans to support community efforts like Open Assistant to create an open-source dataset for AI assistants.
With the release of StableLM, Stability AI aims to contribute to the ongoing efforts to democratize AI technology. Focused on promoting transparency, accessibility, and user support, the company envisions a future where AI serves as a complement to human abilities rather than a replacement. The impact of StableLM in achieving these goals remains to be seen, but it marks another important milestone in the pursuit of a more inclusive and ethical AI landscape.