Microsoft AI has completed pre-training of MAI-Base-2, a foundation model with 2.8 trillion total parameters. The model is now undergoing post-training. According to the company, the main change from MAI-Base-1 is the inclusion of image data directly in pre-training, including documents that combine text and images.
Image training data spans charts, spatial tasks and sheet music
Microsoft says it used an encoder-free vision-language model architecture to connect images and language. Its training material takes two forms: image-text pairs and documents in which text and images are interleaved.
The company says the visual corpus covers image captioning, chart understanding and spatial reasoning. Other material addresses computer use and sheet-music reading. The corpus contains 4.5 trillion unique tokens; counting repetitions, vision-language data made up 18% of the full 50-trillion-token training mixture.
A larger corpus and a March 2026 knowledge cutoff
The expansion extends beyond image data. According to Microsoft, the overall corpus grew by roughly 36% from MAI-Base-1, measured in unique training tokens. The company says it processed every token in-house and did not use distillation from other models during pre-training.
Microsoft also expanded its web-content processing, with a March 2026 knowledge cutoff. The company says it used embedding-based sampling to prioritize novel material and reduce duplicate content.
Internal comparisons with DeepSeek, Kimi, Qwen and MiMo
Microsoft evaluated pre-trained models on held-out text using bits-per-byte, or BPB. The metric allows comparisons regardless of each model’s tokenizer, with lower values being better. In the company’s internal evaluations, MAI-Base-2 achieved lower BPB than every other model tested.
The comparison covered checkpoints available before October 2026: DeepSeek-V4-Pro-Base and Kimi K3, alongside Qwen3.8-2.4T-A95B and MiMo-V2.6-Pro-RL. Only post-trained checkpoints were available for the latter three, and Microsoft says post-training may affect their results in these evaluations.
For code evaluation, the company used MAI’s own codebase. It commissioned external vendors to create STEM, mathematics and question-answering tests to reduce overlap with training material. Microsoft says it also removed overlaps with its training data from all evaluation sets.
Estimated efficiency gains over DeepSeek and MAI-Base-1
Using scaling curves from four evaluations, Microsoft estimates that its training recipe is 4.2–12.3 times more FLOPs-efficient than DeepSeek-V4-Pro-Base at reaching the same BPB. Against MAI-Base-1, it reports an estimated efficiency improvement of 6.3–12.7 times. The calculation uses curves that include MAI-Base-2’s final result and compares the compute needed to reach the same BPB, rather than the actual compute consumed by complete training runs.
To transfer findings from smaller experiments, the company used a method called source scaling. In smaller runs, it limits the number of unique samples so that each dataset is repeated as often as it would be during full training. Microsoft says this reduces the risk of choosing settings that work at small scales but lead to overfitting in a larger model.
MoE in every layer, without dropping tokens
MAI-Base-2 uses a mixture-of-experts architecture with 80 billion active parameters. Microsoft says it trained the larger, sparser model on 16,000 GB200 GPUs over 38 days in August and September 2026.
The company says implementation optimizations increased compute utilization in its MoE layers from 24% to 35%. That allowed Microsoft to use MoE in every layer; for MAI-Base-1, alternating MoE with other layers had been more time-efficient.
Multi-round processing handles uneven expert workloads without dropping tokens. In Microsoft’s implementation, an overloaded expert processes excess tokens in additional rounds and can use spare capacity from other experts in its group. Memory usage remains fixed, although allocating less capacity can require more rounds and additional computation.
Microsoft reports more stable training with Muon
Microsoft says the Muon optimizer delivered more stable training and substantial efficiency gains over AdamW as it scaled up the model. After implementation optimizations, Muon accounted for 2.1% of the average training iteration, or 165 milliseconds.
The company also clips activation gradients at selected points in the network. This targets outliers rather than scaling down gradients across an entire batch. Microsoft says that, combined with Muon, the approach keeps the gradient norm within a tighter range than in MAI-Base-1.
Planned licensing through Foundry
Microsoft plans to license the model’s weights to trusted partners through Foundry. It intends to optimize its models and infrastructure together, first for its own products and then for Foundry customers.
