Microsoft has turned to rival cloud provider Oracle to help meet the voracious compute demands for Bing conversational search. The tech giant is utilizing Oracle's Cloud Infrastructure, in addition to its own Azure cloud, to scale up infrastructure for performing inference on AI models that power Bing's natural language search capabilities.
This collaboration highlights the race among tech giants to amass AI compute power as more services come to rely on complex machine learning models. Bing's shift to conversational search has exponentially increased the scale of infrastructure required. Microsoft is leveraging Oracle's specialized AI hardware and network architecture to supplement its own cloud.
Earlier this year, Microsoft unveiled a new natural language search experience for Bing, enabling users to have search the way they talk, and even ask follow-up questions using Bing Chat. Additionally, multiple other AI chatbots such as ChatGPT and Meta Assistant also leverage Microsoft Bing to provide real time web search capabilities. To power these natural conversations, Bing relies on large language models that require massive-scale inference to analyze each user query in real time.
To handle this load, Microsoft is tapping into Oracle's AI-optimized cloud infrastructure. The OCI Supercluster, integral to this operation, boasts scalability up to 4,096 Compute Bare Metal instances with an impressive number of NVIDIA's A100 or H100 GPUs, supported by petabytes of high-performance clustered file system storage.
By combining resources across cloud providers, Microsoft aims to deliver low-latency experiences at a global scale. The collaboration also provides overflow capacity to absorb spikes in demand for Bing's new conversational features.
The partnership highlights how major cloud platforms are adapting to meet the exploding computational requirements of modern AI systems. Both Oracle and Microsoft have invested heavily in specialized hardware, interconnects, and services purpose-built for machine learning.
While the collaboration directly benefits Microsoft's AI aspirations, it also underscores Oracle's strategy to provide differentiated AI offerings. Oracle likely hopes to seduce AI customers away from the two market leaders, Amazon Web Services and Microsoft Azure.
The Bing deal grants Oracle bragging rights as a proven AI cloud option. But Microsoft also stands to benefit from offloading workloads and tapping into Oracle's robust infrastructure.
As AI permeates more software and services, we will likely see more creative partnerships and infrastructure sharing arrangements between tech giants. The AI boom is forcing rapid expansion of data center capacity worldwide, as Microsoft's partnership exemplifies.
Both companies are positioning themselves to capitalize on ballooning demand for AI compute power. For Microsoft, it's a necessity to maintain Bing's capabilities. And for Oracle, it's an opportunity to compete for more pie in the AI cloud market against established rivals.