AI21 Releases Jamba 1.5: A Family of Open Models with Long Context and Low Latency
At the heart of these models is a hybrid SSM-Transformer architecture that combines Transformer and Mamba layers with a Mixture-of-Experts approach. This innovative design allows Jamba models to process vast amounts of information with exceptional speed and accuracy.