Microsoft plans to expand Azure Maia to multi-gigawatt deployments starting in 2027, with the goal of making it the main infrastructure for running AI models across its products and services. Production deployments of Maia 300 are planned for the same year. Microsoft designed the new generation for long-context reasoning, latency-sensitive conversations and sustained, multi-step AI agent tasks.
Maia 200 already serves Microsoft Copilot
Maia 200 already handles production workloads including Microsoft Copilot and supports the MAI team as it develops next-generation models. GitHub Copilot and Microsoft Foundry are due to follow by the end of 2026.
Microsoft says Maia 200 runs OpenAI models, its own MAI models and leading open-source model families across these workloads. Deployment is underway in the United States, Europe and Australia. For built-in AI features in Microsoft Teams, Copilot and Cowork, the company reports token output per watt of up to 1.7x that of leading GPU infrastructure.
Multi-step agents and latency-sensitive conversations
Microsoft’s intended workloads span reasoning over long context, responsive conversational services and sustained agent tasks involving multiple steps. For GitHub, the company plans to use both Maia generations to run multi-step coding agents more efficiently.
Other planned uses include serving different model architectures and deployment environments in Microsoft Foundry. Both generations are intended to support inference and data generation for the MAI team. For Dragon Copilot, Microsoft plans to support latency-sensitive interactions that help expand workflows and reduce clinicians’ administrative workload.
Comparison with GPU systems and Maia 200
Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems. That expectation is based on efficiency improvements across the whole system, rather than solely on the performance of an individual accelerator.
Microsoft also claims up to seven times Maia 200’s token throughput at rack level when Maia 300 is configured with a full complement of 96 accelerators.
Less data movement and larger context windows
Microsoft’s approach combines control over how data moves through Maia 300 with AI-assisted optimization of its compute kernels. The software-defined dataflow is designed to keep data closer to where it is processed, reduce transfers and improve the use of compute, memory and networking resources.
The company says Maia 300 also supports advanced low-precision data formats, including enhanced MXFP4. These are intended to reduce memory and bandwidth requirements for both computation and KV-cache storage. Microsoft links those reductions to larger context windows, higher throughput and more efficient model operation within the same power limit.
HBM4 and on-chip working memory
Maia 300 has eight HBM4 memory stacks providing 18.0 TB/s of peak bandwidth at a thermal design power (TDP) of 950 watts. Microsoft says this configuration supports efficient processing of large models and long-context workloads. Specialized compute kernels also use on-chip SRAM as a software-managed scratchpad to reduce pressure on HBM.
From 96 accelerators to an entire data center
According to Microsoft, a Maia 300 rack contains 96 accelerators within a 167-kilowatt power limit, with 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth. The company says this configuration is designed to handle more simultaneous AI interactions and larger workloads per rack.
Microsoft optimized the accelerator interconnects for tensor parallelism. Its custom AI communication transport uses Ethernet-based infrastructure to connect accelerators and racks. The network architecture is designed to scale from a single rack to deployments spanning an entire data center.
