Contamination MapText / LLM / Model
Qwen2.5
- Organisation
- Alibaba Cloud (Qwen Team)
- Family
- Qwen · version 2.5
- Released
- 2024-09-19 (Date of the Qwen2.5 announcement blog post, https://qwenlm.github.io/blog/qwen2.5/)
- Checkpoints
- Qwen/Qwen2.5-0.5B, Qwen/Qwen2.5-1.5B, Qwen/Qwen2.5-3B, Qwen/Qwen2.5-7B, Qwen/Qwen2.5-14B, Qwen/Qwen2.5-32B, Qwen/Qwen2.5-72B, Qwen/Qwen2.5-0.5B-Instruct, Qwen/Qwen2.5-1.5B-Instruct, Qwen/Qwen2.5-3B-Instruct, Qwen/Qwen2.5-7B-Instruct, Qwen/Qwen2.5-14B-Instruct, Qwen/Qwen2.5-32B-Instruct, Qwen/Qwen2.5-72B-Instruct
- Weights
- Open · Apache 2.0 (0.5B, 1.5B, 7B, 14B, 32B); Qwen Research License (3B); Qwen License (72B)
- Links
- PaperModel cardCode
Training data disclosure: Vague
Level determined for 20% of benchmark cells.
- TestTest data seen2
- SeenSeen, split unclear0
- TrainTraining data seen0
- Related data seen1
- NoNot seen0
- ?Unknown12
“Building on these techniques, we have developed a larger and higher-quality pre-training dataset, expanding from the 7 trillion tokens used in Qwen2 (Yang et al., 2024a) to 18 trillion tokens.”
“all models are pretrained on our latest large-scale dataset, encompassing up to 18 trillion tokens”
Other declared training data
-
web-scale data (quality-filtered and domain-rebalanced) (broad web data: may contain benchmark test sets) · split: unknown
The report does not name the web sources; 18T tokens total.
Evidence
“Our analysis revealed that domains like e-commerce, social media, and entertainment are significantly overrepresented in web-scale data, often containing repetitive, template-based, or machine-generated content.”
arxiv.org · paper · Section 3.1 (4) Better data mixture · retrieved 2026-10-15 -
Qwen2.5-Math and Qwen2.5-Coder training data (not yet matched to a dataset) · split: unknown
Composition is documented in the Qwen2.5-Math / Qwen2.5-Coder reports, not in this report.
Evidence
“During the pre-training phase of Qwen2.5, we incorporate training data from Qwen2.5-Math (Yang et al., 2024b) and Qwen2.5-Coder (Hui et al., 2024).”
arxiv.org · paper · Section 3.1 (2) · retrieved 2026-10-15 -
synthetic data generated by Qwen2-72B-Instruct and Qwen2-Math-72B-Instruct (not yet matched to a dataset) · split: unknown
Evidence
“To generate high-quality synthetic data, particularly in mathematics, code, and knowledge domains, we leverage both Qwen2-72B-Instruct (Yang et al., 2024a) and Qwen2Math-72B-Instruct (Qwen Team, 2024c).”
arxiv.org · paper · Section 3.1 (3) · retrieved 2026-10-15 -
post-training data (1 million examples for SFT, DPO and GRPO) (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“The post-training data amounts to 1 million examples, across the stage of supervised finetuning (SFT, Ouyang et al., 2022), direct preference optimization (DPO, Rafailov et al., 2023), and group relative policy optimization (GRPO, Shao et al., 2024).”
arxiv.org · paper · Section 1 · retrieved 2026-10-15 -
chain-of-thought data of Qwen2.5-Math (not yet matched to a dataset) · split: unknown · fine-tuning
'public datasets' are not named.
Evidence
“We introduce the chain-of-thought data of Qwen2.5-Math (Yang et al., 2024b), which encompasses a diverse range of query sources, including public datasets, K-12 problem collections, and synthetic problems.”
arxiv.org · paper · Section 4.1 (2) · retrieved 2026-10-15 -
instruction tuning data of Qwen2.5-Coder (incl. code Q&A websites and GitHub snippets) (broad web data: may contain benchmark test sets) · split: unknown · fine-tuning
Evidence
“We expand our instruction dataset by synthesizing new examples from code-related Q&A websites and gathering algorithmic code snippets from GitHub.”
arxiv.org · paper · Section 4.1 (3) · retrieved 2026-10-15
Contamination analyses
Analyses that found benchmark test data in this model's training data, or in a corpus it uses.
-
MATH-500
· pre-training data
Exact completion of 21.2% of MATH-500 problems from a 60% prefix (Table 2).
Evidence
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
arxiv.org third-party study· study · Section 4, Table 2 · retrieved 2026-10-15 -
AIME 2024
· pre-training data
Exact completion of 13.33% of AIME 2024 problems from a 60% prefix (Table 2).
Evidence
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
arxiv.org third-party study· study · Section 4, Table 2 · retrieved 2026-10-15
Benchmark cells
Classic LLM benchmarks
Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail
Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail
MATH (Mathematics Aptitude Test of Heuristics) · a contamination analysis found test examples of the related dataset MATH-500 (Exact completion of 21.2% of MATH-500 problems from a 60% prefix (Table 2).) [third-party study]
Evidence (3)
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
“We therefore evaluate our models only on the remaining 500 held-out problems. We selected these 500 test problems uniformly at random, and we believe they are representative of the test set as a whole.”
“This dataset contains a subset of 500 problems from the MATH benchmark that OpenAI created in their _Let's Verify Step by Step_ paper.”
Unknown HumanEval · training data not disclosed in enough detail
Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail
Unknown HellaSwag · training data not disclosed in enough detail
Unknown TruthfulQA · training data not disclosed in enough detail
Unknown WinoGrande · training data not disclosed in enough detail
Recent LLM benchmarks
Unknown GPQA Diamond · training data not disclosed in enough detail
Unknown MMLU-Pro · training data not disclosed in enough detail
Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail
Unknown SWE-bench Verified · training data not disclosed in enough detail
Test data seen AIME 2024 · a contamination analysis found test examples in the pre-training data (Exact completion of 13.33% of AIME 2024 problems from a 60% prefix (Table 2).) [third-party study]
Evidence (1)
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
Unknown AIME 2025 · training data not disclosed in enough detail
Test data seen MATH-500 · a contamination analysis found test examples in the pre-training data (Exact completion of 21.2% of MATH-500 problems from a 60% prefix (Table 2).) [third-party study]
Evidence (1)
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
Notes
Base and Instruct checkpoints of each size. No training-data cutoff stated. Qwen2.5-Turbo/Plus are API-only and not covered; Qwen2.5-Coder/Math are separate model series. Decontamination statements not shown as cells (they name no benchmark tracked here, or none at all): arc-challenge; gpqa; gsm8k; hellaswag; humaneval; livecodebench; math; mmlu; mmlu-pro; truthfulqa; winogrande.
Sources
- Qwen2.5 Technical Report · paper · retrieved 2026-10-15
- Qwen2.5: A Party of Foundation Models! · website · retrieved 2026-10-15
- Qwen2.5-7B-Instruct model card · model card · retrieved 2026-10-15