MiniMax has introduced the MiniMax-M1 model, a new breakthrough in large language models focused on logical reasoning. This model boasts the title of the first open-source large model with a hybrid-attention architecture. MiniMax-M1 is built on a hybrid Mixture-of-Experts (MoE) architecture combined with the Lightning Attention mechanism, representing a major innovation in the efficient processing of long contexts.

The model is based on the previous MiniMax-Text-01 model and contains a total of 456 billion parameters, with 45.9 billion parameters activated per token. This architecture allows the model to natively support a context length of up to 1 million tokens, eight times the context size of the DeepSeek R1 model. The Lightning Attention mechanism also enables efficient scaling of test-time compute.

Technical Innovations: Lightning Attention Mechanism

Lightning Attention is a key innovation of the MiniMax-M1 model that addresses a long-standing problem with traditional transformer architectures. Conventional softmax attention mechanisms suffer from quadratic computational complexity, making continuous scaling of the reasoning process demanding. MiniMax-M1 solves this problem with a hybrid approach in which every seven transnormer blocks with Lightning Attention are followed by one transformer block with softmax attention.

This design theoretically enables reasoning lengths to scale efficiently to hundreds of thousands of tokens. For example, compared to DeepSeek R1, M1 consumes less than 50% of the FLOP (floating-point operations per second) when generating 64K tokens and approximately 25% of the FLOP at a length of 100K tokens. This substantial reduction in computational costs makes M1 significantly more efficient both during inference and during large-scale RL (reinforcement learning) training.

Model Architecture and Parameters

The MiniMax-M1 model uses a sophisticated hybrid architecture based on the Mixture-of-Experts principle with 32 experts. A total of 456 billion parameters, with 45.9 billion parameters activated per token, provides an optimal balance between performance and efficiency. The model natively supports a context length of up to 1 million tokens, representing a significant expansion compared to current open-source models focused on logical reasoning.

The M1 model was developed in three main phases. First, they continued pre-training MiniMax-Text-01 on 7.5T tokens from a carefully curated reasoning-focused corpus. They subsequently performed supervised fine-tuning (SFT) to incorporate certain chain-of-thought (CoT) reasoning patterns, creating a strong foundation for reinforcement learning, which represents a key phase in M1's development.

CISPO Algorithm: A Revolution in Reinforcement Learning

One of MiniMax-M1's most important innovations is a new reinforcement learning algorithm called CISPO (Clipped Importance Sampling Policy Optimization). This algorithm abandons trust-region constraints and instead clips importance sampling weights to stabilize training. The CISPO approach always uses all tokens for gradient calculations, achieving greater efficiency compared to the GRPO and DAPO algorithms.

In a controlled study based on Qwen2.5-32B models, CISPO achieves a twofold speedup compared to DAPO. The algorithm was designed specifically to address problems associated with token clipping in the original PPO/GRPO algorithms. The researchers found that tokens associated with reflective behavior (such as "However", "Recheck", "Wait", "Aha"), which often serve as "forks" in reasoning paths, were typically rare and assigned low probabilities by the base model.

Benchmark Results and Comparisons

The MiniMax-M1 model achieves remarkable results across various benchmarks. In mathematical reasoning, MiniMax-M1-80k scores 86.0% on AIME 2024, ranking second among open-weight models and trailing only the latest DeepSeek-R1-0528 model. In general coding, MiniMax-M1-80k matches Qwen3-235B on LiveCodeBench while outperforming it on FullStackBench.

The model achieves significant success in complex scenarios. Thanks to execution-based software engineering environments during RL, MiniMax-M1-40k and MiniMax-M1-80k achieve strong scores of 55.6% and 56.0% on SWE-bench Verified. By leveraging their 1M-token context window, the M1 models significantly outperform all other open-weight models in long-context understanding and even surpass OpenAI o3 and Claude 4 Opus.

Availability and Future Development

To facilitate collaboration and progress in the field, MiniMax has made its models publicly available on GitHub and Hugging Face. The models are now supported by both the vLLM and Transformers frameworks, with detailed deployment guides. The company also provides a commercial-grade API at minimax.io.

The researchers trained two versions of the MiniMax-M1 models with maximum generation lengths of 40K and 80K tokens, resulting in the MiniMax-M1-40k and MiniMax-M1-80k models. MiniMax-M1-80k outperforms MiniMax-M1-40k on complex mathematical and coding tasks, further demonstrating the benefits of scaling test-time compute.

Looking ahead, as test-time compute continues to scale to support increasingly complex scenarios, the researchers anticipate significant potential for such efficient architectures in addressing real-world challenges. These include automating business workflows and conducting scientific research. Real-world applications particularly require LRMs that function as agents interacting with environments, tools, computers, or other agents, requiring reasoning across tens to hundreds of turns while integrating long-context information from diverse sources.