About MiMo-7B
Overview
MiMo-7B is a family of open-source language models that Xiaomi's LLM-Core-Team trained from scratch for reasoning tasks. The release ships base, supervised fine-tuned, RL-Zero, and RL checkpoints, and the team reports that the final RL model matches OpenAI o1-mini on mathematics and code benchmarks.
The project argues that reinforcement learning only amplifies reasoning ability that already exists in a base model, so Xiaomi tuned pretraining for reasoning as well as post-training. MiMo-7B-Base pretrains on roughly 25 trillion tokens, and its RL variants outperform several 32B-class models on AIME 2024, AIME 2025, and LiveCodeBench.
Key Benefits
- A 7B model that matches OpenAI o1-mini on math and code reasoning benchmarks.
- The base model already outperforms larger 32B models before any reinforcement learning.
- A three-stage pretraining mixture on about 25 trillion tokens raises reasoning pattern density.
- Multi-Token Prediction accelerates inference and reaches roughly 90% speculative decoding acceptance.
- All four checkpoints (Base, SFT, RL-Zero, RL) are published openly on HuggingFace and ModelScope.
- A custom rollout engine cuts RL training time by 2.29x and validation time by 1.96x.
How It Works
Teams download a checkpoint from HuggingFace or ModelScope and launch it with Xiaomi's vLLM fork (the recommended path for MTP), SGLang, or HuggingFace Transformers using trust_remote_code. Xiaomi recommends an empty system prompt and temperature 0.6, and enabling one speculative token lets the MTP layer speed decoding. The May 2025 update, MiMo-7B-RL-0530, scales the SFT dataset to about 6M instances and widens the RL training window to 48K.
Use Cases
Use Cases
- AI researchers studying how pretraining and RL recipes interact in small reasoning models.
- ML engineers deploying a compact reasoning model on vLLM, SGLang, or Transformers.
- Evaluation teams benchmarking math and code accuracy against o1-mini, QwQ-32B-Preview, and R1-Distill variants.
- Application developers who need openly licensed reasoning weights to fine-tune or self-host.
- Research groups reproducing the published technical report and RL training pipeline.
Why Choose This Product
MiMo-7B targets a gap Xiaomi describes in its report: simultaneous gains in math and code were widely considered hard for small models. The release pairs every intermediate checkpoint with a published technical report and deployment scripts, so teams can reproduce or extend the recipe. The README notes that inference is verified only with vLLM, SGLang, and HuggingFace, and that MTP decoding relies on Xiaomi's vLLM fork.
MiMo-7B Pros & Cons
- Open-source checkpoints released under the Apache-2.0 license
- Matches OpenAI o1-mini on math and code at only 7B parameters
- Outperforms several 32B models on AIME 2024, AIME 2025 and LiveCodeBench
- Published technical report with full training and RL recipe
- About 90% acceptance rate from a single MTP speculative decoding layer
- Inference verified only with vLLM, SGLang and HuggingFace engines
- MTP speculative decoding depends on Xiaomi's vLLM fork
Key Features
Reasoning-First Pretraining
MiMo-7B-Base pretrains on approximately 25 trillion tokens using a three-stage data mixture tuned for reasoning pattern density.
Data Filtering Pipeline
Enhanced text extraction toolkits and multi-dimensional filtering increase the density of reasoning patterns in pretraining data.
Synthetic Reasoning Data
Multiple strategies generate massive, diverse synthetic reasoning datasets to supplement the pretraining corpus.
Multi-Token Prediction
An additional MTP training objective improves model performance and accelerates inference, reaching about 90% acceptance with one speculative decoding layer.
Verifiable RL Rewards
The team curates 130K math and code problems that rule-based verifiers check, using only rule-based accuracy rewards to avoid reward hacking.
Difficulty-Driven Code Reward
Fine-grained scores across test cases of varying difficulty give the policy a dense reward signal on challenging code problems.
Data Re-Sampling
Re-sampling easy problems improves rollout sampling efficiency and stabilizes policy updates in late RL phases.
Rollout Engine
A continuous, asynchronous rollout engine with early termination delivers 2.29x faster training and 1.96x faster validation.
Four Open Checkpoints
The release includes MiMo-7B-Base, MiMo-7B-SFT, MiMo-7B-RL-Zero, and MiMo-7B-RL weights on HuggingFace and ModelScope.
Deployment Recipes
Ready-to-run scripts cover Xiaomi's vLLM fork with MTP, SGLang server launch, and HuggingFace Transformers inference.
Frequently asked questions about MiMo-7B
Which models are in the MiMo-7B series?
Xiaomi releases four checkpoints: MiMo-7B-Base, the raw pretrained model; MiMo-7B-SFT, trained from the base model; MiMo-7B-RL-Zero, an RL model trained from base; and MiMo-7B-RL, an RL model trained from the SFT model that matches OpenAI o1-mini on math and code.
How do I run MiMo-7B for inference?
Download a checkpoint from HuggingFace or ModelScope and load it with Xiaomi's vLLM fork (recommended for MTP), SGLang, or HuggingFace Transformers using trust_remote_code. SGLang installs from its main branch and launches with a standard server command, with an optional MTP server mode using EAGLE speculative decoding.
What benchmark scores does MiMo-7B-RL achieve?
The README reports 95.8 on MATH500, 68.2 on AIME 2024, 55.4 on AIME 2025, 57.8 on LiveCodeBench v5, 49.3 on LiveCodeBench v6, and 54.4 on GPQA-Diamond, all Pass@1. The May 2025 MiMo-7B-RL-0530 update raises these to 97.2, 80.1, 70.2, 60.9, 52.2, and 60.6.
Under what license is MiMo released?
The repository is published under the Apache-2.0 license, with model weights hosted on HuggingFace and ModelScope. The README also lists use restrictions covering unlawful, infringing, privacy-violating, military, and malicious cyber uses, and states that the notice does not amend the applicable model licence.
What temperature and prompt settings do you recommend?
Xiaomi recommends an empty system prompt and temperature 0.6. Benchmark evaluations use temperature=0.6, with AIME24, AIME25, LiveCodeBench, GPQA-Diamond, and IF-Eval averaged over multiple repetitions, while MATH500 and SuperGPQA use a single run.
How much data was MiMo-7B trained on?
MiMo-7B-Base pretrains on approximately 25 trillion tokens. For post-training, the team curates 130K math and code problems as RL data, and the May 2025 update scaled the SFT dataset from about 500K to 6M instances.