Contamination MapText / LLM / Model
DeepSeek-R1
- Organisation
- DeepSeek-AI
- Family
- DeepSeek · version R1
- Released
- 2025-01-20 (DeepSeek API Docs news entry 'DeepSeek-R1 Release 2025/01/20', https://api-docs.deepseek.com/news/news1226; arXiv v1 submitted 2025-01-22)
- Training cutoff
- 2024-07-01
Evidence
“DeepSeek-V3 base has a knowledge cutoff date of July 2024, predating evaluation benchmarks like CNMO 2024,”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 - Checkpoints
- deepseek-ai/DeepSeek-R1, deepseek-ai/DeepSeek-R1-Zero
- Weights
- Open · MIT
- Links
- PaperModel cardCode
Training data disclosure: Vague
Level determined for 53% of benchmark cells.
- TestTest data seen0
- SeenSeen, split unclear0
- TrainTraining data seen0
- Related data seen0
- NoNot seen8
- ?Unknown7
“Reasoning RL data includes four categories: mathematics, coding, STEM, and logic problems.”
“In total, we collect about 600k reasoning-related training samples.”
Other declared training data
-
plain web pages and e-books (DeepSeek-V3-Base pre-training data, 14.8T tokens) (broad web data: may contain benchmark test sets) · split: unknown
Described in the DeepSeek-R1 paper (same developer). The V3 report adds that math and programming samples were up-weighted.
Evidence
“For the training data of DeepSeek-V3-Base, we exclusively use plain web pages and e-books, without incorporating any synthetic data.”
arxiv.org · paper · Appendix A.1 · retrieved 2026-10-15 “all data used in this phase were naturally occurring and collected through web crawling.”
arxiv.org · paper · Appendix A.1 · retrieved 2026-10-15 -
RL mathematics dataset (26k math exam and competition questions) (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“Mathematics dataset consists of 26k quantitative reasoning questions, including math exam questions and competition problems.”
arxiv.org · paper · Appendix B.3.1, Table 4 · retrieved 2026-10-15 -
RL coding dataset (17k algorithm competition questions, 8k bug-fixing problems from GitHub issues) (not yet matched to a dataset) · split: unknown · fine-tuning
GitHub-issue bug fixing is the same kind of source SWE-bench is built from; the 10-gram decontamination (see exclusions) is the only stated safeguard.
Evidence
“Coding dataset includes 17k algorithm competition questions, along with 8k bug fixing problems.”
arxiv.org · paper · Appendix B.3.1, Table 4 · retrieved 2026-10-15 “The bug-fixing problems are extracted from real-world GitHub issues.”
arxiv.org · paper · Appendix B.3.1 · retrieved 2026-10-15 -
RL STEM dataset (22k multiple-choice physics/chemistry/biology questions) (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“STEM dataset comprises 22k choice questions that cover topics such as physics, chemistry, and biology.”
arxiv.org · paper · Appendix B.3.1, Table 4 · retrieved 2026-10-15 -
RL logic dataset (15k questions, partly sourced from the web) (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“The real-world portion of the dataset comprises a diverse selection of problems sourced from the web, including brain teasers, classical logic puzzles, and knowledge-intensive questions.”
arxiv.org · paper · Appendix B.3.1 · retrieved 2026-10-15 -
cold-start long-CoT data (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“For DeepSeek-R1, we construct and collect a small amount of long CoT data to fine-tune the model as the initial RL actor.”
arxiv.org · paper · Appendix B.3.2 · retrieved 2026-10-15 -
800K supervised data (~600k reasoning samples by rejection sampling, ~200k non-reasoning samples incl. reused DeepSeek-V3 SFT data) (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“In total, we collect about 600k reasoning-related training samples.”
arxiv.org · paper · Appendix B.3.3 · retrieved 2026-10-15 “In the end, we collected a total of approximately 200k training samples that are unrelated to reasoning.”
arxiv.org · paper · Appendix B.3.3 · retrieved 2026-10-15
Declared exclusions
-
Dataset MMLU (Measuring Massive Multitask Language Understanding) · test portion excluded
Evidence
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “Benchmarks We evaluate models on MMLU (Hendrycks et al., 2021), MMLU-Redux (Gema et al., 2025), MMLU-Pro (Wang et al., 2024),”
arxiv.org · paper · Appendix D.1 (Benchmarks) · retrieved 2026-10-15 -
Dataset MMLU-Pro · test portion excluded
Evidence
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “Benchmarks We evaluate models on MMLU (Hendrycks et al., 2021), MMLU-Redux (Gema et al., 2025), MMLU-Pro (Wang et al., 2024),”
arxiv.org · paper · Appendix D.1 (Benchmarks) · retrieved 2026-10-15 -
Dataset GPQA Diamond · test portion excluded
Evidence
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “GPQA Diamond (Rein et al., 2023), SimpleQA (OpenAI, 2024a), C-SimpleQA (He et al., 2024), SWE-Bench Verified (OpenAI, 2024b),”
arxiv.org · paper · Appendix D.1 (Benchmarks) · retrieved 2026-10-15 -
Dataset SWE-bench Verified · test portion excluded
Evidence
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “GPQA Diamond (Rein et al., 2023), SimpleQA (OpenAI, 2024a), C-SimpleQA (He et al., 2024), SWE-Bench Verified (OpenAI, 2024b),”
arxiv.org · paper · Appendix D.1 (Benchmarks) · retrieved 2026-10-15 -
Dataset LiveCodeBench (code generation, release_v6) · test portion excluded
Evidence
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “LiveCodeBench (Jain et al., 2024)”
arxiv.org · paper · Appendix D.1 (Benchmarks) · retrieved 2026-10-15 -
Dataset AIME 2024 · test portion excluded
Evidence
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “American Invitational Mathematics Examination 2024 (AIME 2024) (MAA, 2024).”
arxiv.org · paper · Appendix D.1 (Benchmarks) · retrieved 2026-10-15 -
Dataset MATH-500 · test portion excluded
Evidence
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 “Finally, AIME, MATH-500, and CNMO 2024 comprise mathematics problems that test the model”
arxiv.org · paper · Appendix D.1 (Benchmarks) · retrieved 2026-10-15 -
Dataset AIME 2024 · test portion excluded from fine-tuning data
Evidence
“For post-training, mathematical SFT data and RL training prompts were sourced exclusively from pre-2023 competitions and underwent the same n-gram filtering protocol used in pre-training, ensuring no overlap between training and evaluation data.”
arxiv.org · paper · Appendix D.1 (Decontamination) · retrieved 2026-10-15 -
Dataset AIME 2025 · entirely excluded
Evidence
“can DeepSeek-R1 achieve comparable performance on test sets that were released after its training? To investigate this, we evaluate our model on AIME 2025”
arxiv.org · paper · Appendix E.2 · retrieved 2026-10-15
Benchmark cells
Classic LLM benchmarks
Not seen MMLU (Measuring Massive Multitask Language Understanding) · the authors state its test portion was excluded from training
Evidence (3)
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
“we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
“Benchmarks We evaluate models on MMLU (Hendrycks et al., 2021), MMLU-Redux (Gema et al., 2025), MMLU-Pro (Wang et al., 2024),”
Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail
Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail
Unknown HumanEval · training data not disclosed in enough detail
Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail
Unknown HellaSwag · training data not disclosed in enough detail
Unknown TruthfulQA · training data not disclosed in enough detail
Unknown WinoGrande · training data not disclosed in enough detail
Recent LLM benchmarks
Not seen GPQA Diamond · the authors state its test portion was excluded from training
Evidence (3)
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
“we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
“GPQA Diamond (Rein et al., 2023), SimpleQA (OpenAI, 2024a), C-SimpleQA (He et al., 2024), SWE-Bench Verified (OpenAI, 2024b),”
Not seen MMLU-Pro · the authors state its test portion was excluded from training
Evidence (3)
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
“we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
“Benchmarks We evaluate models on MMLU (Hendrycks et al., 2021), MMLU-Redux (Gema et al., 2025), MMLU-Pro (Wang et al., 2024),”
Not seen LiveCodeBench (code generation, release_v6) · the authors state its test portion was excluded from training
Evidence (3)
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
“we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
“LiveCodeBench (Jain et al., 2024)”
Not seen SWE-bench Verified · the authors state its test portion was excluded from training
Evidence (3)
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
“we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
“GPQA Diamond (Rein et al., 2023), SimpleQA (OpenAI, 2024a), C-SimpleQA (He et al., 2024), SWE-Bench Verified (OpenAI, 2024b),”
Not seen AIME 2024 · the authors state its test portion was excluded from training
Evidence (3)
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
“we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
“American Invitational Mathematics Examination 2024 (AIME 2024) (MAA, 2024).”
Not seen AIME 2025 · the authors state the dataset was excluded from training
Evidence (1)
“can DeepSeek-R1 achieve comparable performance on test sets that were released after its training? To investigate this, we evaluate our model on AIME 2025”
Not seen MATH-500 · the authors state its test portion was excluded from training
Evidence (3)
“To prevent benchmark contamination, we implemented comprehensive decontamination procedures for both pre-training and post-training data.”
“we filtered out any text segments (including web pages and GitHub files) that contained matching 10-gram sequences from evaluation questions or reference solutions.”
“Finally, AIME, MATH-500, and CNMO 2024 comprise mathematics problems that test the model”
Notes
Trained on top of DeepSeek-V3-Base (pre-training data shared with deepseek-v3). The paper also cautions: 'variations of test set questions or discussions of related problems may still exist on websites that were included in the pre-training corpus' (Appendix E.2). Other reported contamination, not shown as cells: evaluation benchmarks (mathematics domain): 10-gram decontamination identified and removed about six million potential pre-training texts in the mathematics domain alone (before training)
Sources
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (v2) · paper · retrieved 2026-10-15
- DeepSeek-R1 model card · model card · retrieved 2026-10-15
- DeepSeek API Docs news · website · retrieved 2026-10-15