Microsoft's MAI-Base-2 Completes Pre-Training: 2T-Parameter All-MoE Architecture, Trained 38 Days on 16,000 GB200s

What happened

Microsoft's AI research division detailed the MAI-Base-2 base model on Thursday (October 15), SiliconANGLE reported on October 16. Unlike the previous generation's mix of MoE and FFN layers, MAI-Base-2 is made up entirely of MoE layers: 2 trillion parameters in total, with 80 billion activated per inference. Training ran for 38 days on 16,000 Nvidia GB200 chips, on a dataset of 50 trillion tokens — about a fifth of which trained the model's integrated vision module, improving its grasp of scientific charts.

Microsoft says MAI-Base-2 outperformed DeepSeek-V4-Pro-Base — the most capable open-source base model on the market — on internal benchmarks covering science, programming and math questions, with up to 12x hardware efficiency by certain measures. All figures are self-reported by Microsoft and have not been independently verified. During training, Microsoft used gradient clipping to prevent disruptions and the open-source optimizer Muon to speed up the workflow.

MAI-Base-2 is still in post-training. Microsoft says it will license the model's weights to "trusted partners" through Azure rather than releasing them publicly, and will use it to power a new series of models, with more details coming soon; MAI-Base-3 is already in preparation.

Key facts

1. 2T-parameter MoE: All-MoE-layer architecture, 2 trillion parameters total, 80 billion activated per inference
2. 16,000 GB200s x 38 days: 50 trillion training tokens, about a fifth for the integrated vision module
3. Microsoft claims lead over top open base: Beat DeepSeek-V4-Pro-Base on internal benchmarks, up to 12x hardware efficiency by some measures (self-reported, unverified)
4. Still in post-training: Weights not publicly released; licensed to "trusted partners" via Azure
5. MAI-Base-3 in preparation: Microsoft will use MAI-Base-2 to power a new series of models