Microsoft has introduced Phi Silica, a new addition to its Phi family of small language models (SLMs), designed specifically for the Neural Processing Units (NPUs) in the company's recently announced Copilot+ PCs.
Phi Silica is the smallest model in the Phi family, with 3.3 billion parameters. Despite its compact size, it delivers impressive performance, achieving a first token latency of 650 tokens per second while consuming only 1.5 Watts of power. This efficiency ensures that the PC's CPU and GPU remain free for other tasks, enhancing the overall computing experience. Token generation reuses the NPU's KV cache and runs on the CPU, producing approximately 27 tokens per second.
Developers will be able to access the Phi Silica API through the Windows App SDK, along with other AI-powered functionalities such as OCR, Studio Effects, Live Captions, and Recall User Activity APIs. This integration will allow developers to create innovative experiences that leverage the power of AI within the Windows ecosystem. Microsoft plans to release additional APIs, including Vector Embedding, RAG API, and Text Summarization, in the future.
Phi Silica joins the ranks of other models in the Phi-3 series, including Phi-3-mini (3.8B parameters), Phi-3-small (7B parameters), Phi-3-medium (14B parameters), and the recently announced Phi-3-vision (4.2B parameters). However, Phi Silica stands out as the first state-of-the-art SLM to be shipped with Windows, marking a significant milestone in bringing advanced AI capabilities directly to end-users.
The introduction of Phi Silica comes on the heels of Microsoft's Copilot+ PC announcement, which promises to deliver Windows PCs with dedicated AI processors. The first Copilot+ PCs, set to hit the market in mid-June, will feature Qualcomm's Arm-based Snapdragon X Elite and Plus chips. Microsoft, along with major PC manufacturers, is expected to offer these AI-powered laptops throughout the summer. Intel is also developing its own Copilot+ PC processor, codenamed Lunar Lake, which is slated for release in the third quarter of 2024.
As Microsoft continues to invest in the development of small language models and the integration of AI into its Windows platform, Phi Silica represents a significant step forward in making advanced AI capabilities more accessible to developers and end-users alike.