Contamination MapText / LLM / Model
Phi-mini-MoE / Phi-tiny-MoE (SlimMoE)
- Organisation
- Microsoft
- Family
- Phi · version SlimMoE
- Released
- 2025-06-23 (HF repo creation of microsoft/Phi-mini-MoE-instruct (2025-06-23); SlimMoE arXiv v1 2025-06-23)
- Training cutoff
- 2023-10-01
Evidence
“This is a static model trained on an offline dataset with cutoff date October 2023 for publicly available data.”
huggingface.co · model card · Status · retrieved 2026-10-15 - Checkpoints
- microsoft/Phi-mini-MoE-instruct, microsoft/Phi-tiny-MoE-instruct
- Weights
- Open · MIT
- Links
- PaperModel card
Training data disclosure: Vague
Level determined for 13% of benchmark cells.
- TestTest data seen0
- SeenSeen, split unclear0
- TrainTraining data seen0
- Related data seen0
- NoNot seen2
- ?Unknown13
“Our training data is a subset with 400B tokens of Phi-3 datasets”
Other declared training data
-
Phi-3 datasets (400B-token subset: filtered public documents and synthetic data) (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“The model is trained on Phi-3 synthetic data and filtered public documents, with a focus on high-quality, reasoning-dense content.”
huggingface.co · model card · Model Summary · retrieved 2026-10-15
Benchmark cells
Classic LLM benchmarks
Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail
Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail
Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail
Unknown HumanEval · training data not disclosed in enough detail
Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail
Unknown HellaSwag · training data not disclosed in enough detail
Unknown TruthfulQA · training data not disclosed in enough detail
Unknown WinoGrande · training data not disclosed in enough detail
Recent LLM benchmarks
Unknown GPQA Diamond · training data not disclosed in enough detail
Unknown MMLU-Pro · training data not disclosed in enough detail
Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail
Unknown SWE-bench Verified · training data not disclosed in enough detail
Not seen AIME 2024 · its test items were published from 2024-01-31, after the model's stated training-data cutoff (2023-10-01)
Evidence (4)
“This is a static model trained on an offline dataset with cutoff date October 2023 for publicly available data.”
“2024 AIME I problems and solutions. The test was held on Wednesday, January 31 – Thursday, February 1, 2024.”
“AIME I (main AIME competition date): Thursday, Feb. 1, 2024 between 1:30 pm ET and 5:30 pm ET.”
“AIME II (alternate AIME competition date): Wednesday, Feb. 7, 2024 between 1:30 pm ET and 5:30 pm ET.”
Not seen AIME 2025 · its test items were published from 2025-02-06, after the model's stated training-data cutoff (2023-10-01)
Evidence (3)
“This is a static model trained on an offline dataset with cutoff date October 2023 for publicly available data.”
“2025 AIME I problems and solutions. The test was held on Thursday, February 6, 2025.”
“This year the AIME I will take place on February 6, 2025, while the alternate AIME II will take place on February 12, 2025.”
Unknown MATH-500 · training data not disclosed in enough detail
Notes
Compressed and distilled from the Phi-3.5-MoE / GRIN-MoE base model, then SFT + DPO.
Sources
- Phi-mini-MoE-instruct model card · model card · retrieved 2026-10-15
- Phi-tiny-MoE-instruct model card · model card · retrieved 2026-10-15