Microsoft unveils Maia 300 chip, claims 7x token throughput gain

The latest:

Microsoft’s third-generation inference silicon, Maia 300, is built as a single system spanning software, chip, rack, networking and datacenter, the company said, targeting agentic inference at cloud scale. Microsoft put the gain at up to 7x the token throughput of Maia 200 in a full 96-accelerator rack, with more than 2.5x better performance per watt. First operational deployment is scheduled for 2027.

Details:

• The throughput claim: Microsoft said a fully configured rack holding 96 Maia 300 accelerators delivers up to 7 times the token throughput of the previous Maia 200 generation, alongside performance-per-watt efficiency more than 2.5 times higher. The figures are the company’s own, published on the Azure blog rather than by an independent benchmark.
• Memory specs: Each chip carries eight HBM4 stacks totaling 288GB of capacity with 18 terabytes per second of bandwidth, according to Microsoft. That is 2.5 times the bandwidth and 33% more capacity than Maia 200, and roughly double the bandwidth per watt — the metric that matters most for large-model inference.
• Chiplet architecture: The design splits functions across multiple chiplets: compute dies are fabricated on TSMC’s N2P process, while memory controllers and I/O are isolated onto separate dies. Microsoft said the approach lifts floating-point compute performance 36%, adds 2.5 times more HBM bandwidth and 14% additional scale-out bandwidth.
• Rack density: A single rack packs 96 accelerators drawing 167 kilowatts of power, double the density of Maia 200, according to the company. Power density at that level makes cooling and datacenter design part of the product rather than an afterthought, which is why Microsoft frames the chip as a full system.
• Scale-up fabric: Microsoft’s proprietary Ethernet-based scale-up network links accelerators into what the company calls a super-pod, reaching up to 12,288 accelerators. Building on Ethernet rather than a proprietary interconnect is a deliberate divergence from the dominant approach in high-end AI clusters.
• Timeline: First operational rollout of Maia 300 is slated for 2027, Microsoft said. No month or launch region was specified, and the company did not disclose production volumes or how much of its inference fleet the chip is expected to carry at launch.
• What Maia 200 runs: The previous generation is already in production, powering Microsoft Copilot and supporting the team developing the in-house MAI models, according to the company. GitHub Copilot and Microsoft Foundry are set to move onto Maia 200 before the end of the year.
• The authors: The announcement was published on the Azure blog on 12 October 2026, written by Rani Borkar, head of Azure systems and infrastructure, and Saurabh Dighe, vice president for Azure systems planning and engineering.

Background:

Maia is Microsoft’s in-house line of AI accelerators, designed to run inference workloads on Azure. Maia 200 is the current deployed generation; Maia 300 is the third, announced ahead of a 2027 rollout.

Between the lines:

The emphasis on performance per watt and rack density points to the real constraint: not compute scarcity but power and cooling budgets inside datacenters. A 167-kilowatt rack is only economical if the efficiency gains Microsoft cites hold in production. Building the scale-up fabric on Ethernet, and fabricating compute dies at TSMC while isolating memory and I/O, also gives Microsoft more control over cost per token than buying merchant accelerators does.

What’s next

Watch whether GitHub Copilot and Microsoft Foundry complete their move to Maia 200 before year-end, the first independent benchmarks of Maia 300 performance claims, and any disclosure of 2027 deployment scale.

Source: