Contamination MapText / LLM / Model
Phi-4-reasoning / Phi-4-reasoning-plus
- Organisation
- Microsoft
- Family
- Phi · version 4-reasoning
- Released
- 2025-04-30 (Model card 'Release date: April 30, 2025'; arXiv v1 2025-04-30)
- Training cutoff
- 2025-03-01
Evidence
“Static model trained on an offline dataset with cutoff dates of March 2025 and earlier for publicly available data”
huggingface.co · model card · Model Summary / Status · retrieved 2026-10-15 - Checkpoints
- microsoft/Phi-4-reasoning, microsoft/Phi-4-reasoning-plus
- Weights
- Open · MIT
- Links
- PaperModel card
Training data disclosure: Vague
Level determined for 6% of benchmark cells.
- TestTest data seen0
- SeenSeen, split unclear0
- TrainTraining data seen0
- Related data seen0
- NoNot seen1
- ?Unknown14
“Our prompts are sourced from publicly available websites, existing datasets, and licensed collections, and are further augmented with synthetically generated problems.”
Other declared training data
-
seed prompts from web-based sources, existing datasets and licensed collections (plus synthetic problems) (broad web data: may contain benchmark test sets) · split: unknown · fine-tuning
Evidence
“We begin by collecting a diverse and comprehensive dataset of questions from various web-based sources.”
arxiv.org · paper · Section 2.1 · retrieved 2026-10-15 “Our prompts are sourced from publicly available websites, existing datasets, and licensed collections, and are further augmented with synthetically generated problems.”
arxiv.org · paper · Section 2 · retrieved 2026-10-15 -
o3-mini generated reasoning traces (SFT responses; 16B tokens, ~8.3B unique) (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“The responses that are used exclusively during supervised fine-tuning are synthetically generated using o3-mini”
arxiv.org · paper · Section 2 · retrieved 2026-10-15 “16B tokens, ~8.3B unique tokens”
huggingface.co · model card · Model Summary / Training data · retrieved 2026-10-15 -
GRPO math seed set (72,401 problems; Phi-4-reasoning-plus only) (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“The seed dataset for GRPO consisted of 72,401 mathematical problems (prompts without solutions), from which we subsample 64 problem seeds per RL iteration.”
arxiv.org · paper · Section 4 · retrieved 2026-10-15
Declared exclusions
-
Dataset AIME 2024 · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset MATH (Mathematics Aptitude Test of Heuristics) · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset GPQA (Graduate-Level Google-Proof Q&A) · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset LiveCodeBench (code generation, release_v6) · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset SWE-bench Verified · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset GSM8K (Grade School Math 8K) · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset HellaSwag · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset HumanEval · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset WinoGrande · test portion excluded from fine-tuning data
Evidence
“We pass the full training data through the same rigorous decontamination process used Phi-4 [2] for decontaminating against popular reasoning as well as general-purpose benchmarks including many not discussed in this report.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 “The full list of benchmarks decontaminated against is: AIME-2024, MATH, GPQA, LiveCodeBench, Codeforces, OmniMATH, SWE-Bench Verified, SimpleQA, DROP, AGIEval, ARC-Challenge, ARC-Easy, CommonsenseQA, GPQA, GSM8k, HellaSwag, HumanEval, MBPP, OpenBookQA, PIQA, WinoGrande, ArenaHard, MT-Bench, PhiBench.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15 -
Dataset AIME 2025 · entirely excluded
Evidence
“AIME-2025 was released after the data for Phi-4-reasoning was finalized, so that benchmark remains contamination free.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15
Benchmark cells
Classic LLM benchmarks
Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail
Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail
Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail
Unknown HumanEval · training data not disclosed in enough detail
Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail
Unknown HellaSwag · training data not disclosed in enough detail
Unknown TruthfulQA · training data not disclosed in enough detail
Unknown WinoGrande · training data not disclosed in enough detail
Recent LLM benchmarks
Unknown GPQA Diamond · training data not disclosed in enough detail
Unknown MMLU-Pro · training data not disclosed in enough detail
Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail
Unknown SWE-bench Verified · training data not disclosed in enough detail
Unknown AIME 2024 · training data not disclosed in enough detail
Not seen AIME 2025 · the authors state the dataset was excluded from training
Evidence (1)
“AIME-2025 was released after the data for Phi-4-reasoning was finalized, so that benchmark remains contamination free.”
Unknown MATH-500 · training data not disclosed in enough detail
Notes
Fine-tuned from phi-4 (pre-training data and pre-training decontamination: see phi-4). Phi-4-reasoning-vision-15B (2026) not covered.
Sources
- Phi-4-reasoning Technical Report · paper · retrieved 2026-10-15
- Phi-4-reasoning model card · model card · retrieved 2026-10-15
- Phi-4-reasoning-plus model card · model card · retrieved 2026-10-15