Contamination Map Methodology

Text / LLM / Model

Qwen2.5-VL

Organisation
Alibaba Cloud (Qwen Team)
Family
Qwen · version 2.5
Released
2025-01-26 (Date of the Qwen2.5-VL announcement blog post, https://qwenlm.github.io/blog/qwen2.5-vl/ (HF repos Qwen/Qwen2.5-VL-3B/7B-Instruct created 2025-01-26))
Checkpoints
Qwen/Qwen2.5-VL-3B-Instruct, Qwen/Qwen2.5-VL-7B-Instruct, Qwen/Qwen2.5-VL-32B-Instruct, Qwen/Qwen2.5-VL-72B-Instruct
Weights
Open · Apache 2.0 (per the 7B-Instruct model card; licenses of other sizes not checked)
Links
PaperModel cardCode

Training data disclosure: Vague

Level determined for 13% of benchmark cells.

“Compared to Qwen2-VL, we have significantly expanded the volume of our pre-training data, increasing it from 1.2 trillion tokens to approximately 4 trillion tokens.”
arxiv.org · paper · Section 2.2.1 Pre-Training Data · retrieved 2026-10-15
“Our pre-training dataset was constructed through a combination of methods, including cleaning raw web data, synthesizing data, etc.”
arxiv.org · paper · Section 2.2.1 Pre-Training Data · retrieved 2026-10-15

Built on

Everything these base models may have seen is inherited by Qwen2.5-VL.

Other declared training data

Benchmark cells

Classic LLM benchmarks

Unknown MMLU (Measuring Massive Multitask Language Understanding) · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown GSM8K (Grade School Math 8K) · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Related data seen MATH (Mathematics Aptitude Test of Heuristics) · via base model Qwen2.5: a contamination analysis found test examples of the related dataset MATH-500 (Exact completion of 21.2% of MATH-500 problems from a 60% prefix (Table 2).) [third-party study]

Evidence (4)
“while leveraging the pre-trained Qwen2.5 large language model (LLM) (Yang et al., 2024a) as the initialization for the LLM component”
arxiv.org · paper · Section 2.2.2 Training Recipe · retrieved 2026-10-15
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
arxiv.org third-party study· study · Section 4, Table 2 · retrieved 2026-10-15
“We therefore evaluate our models only on the remaining 500 held-out problems. We selected these 500 test problems uniformly at random, and we believe they are representative of the test set as a whole.”
raw.githubusercontent.com · readme · MATH Splits · retrieved 2026-10-15
“This dataset contains a subset of 500 problems from the MATH benchmark that OpenAI created in their _Let's Verify Step by Step_ paper.”
huggingface.co · dataset card · retrieved 2026-10-15

Unknown HumanEval · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown HellaSwag · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown TruthfulQA · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown WinoGrande · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Multimodal LLM benchmarks

Unknown MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning) · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown MMMU-Pro · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown MathVista · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown ChartQA · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown DocVQA · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown AI2D (AI2 Diagrams) · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown CharXiv · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Recent LLM benchmarks

Unknown GPQA Diamond · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown MMLU-Pro · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown LiveCodeBench (code generation, release_v6) · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Unknown SWE-bench Verified · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Test data seen AIME 2024 · via base model Qwen2.5: a contamination analysis found test examples in the pre-training data (Exact completion of 13.33% of AIME 2024 problems from a 60% prefix (Table 2).) [third-party study]

Evidence (2)
“while leveraging the pre-trained Qwen2.5 large language model (LLM) (Yang et al., 2024a) as the initialization for the LLM component”
arxiv.org · paper · Section 2.2.2 Training Recipe · retrieved 2026-10-15
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
arxiv.org third-party study· study · Section 4, Table 2 · retrieved 2026-10-15

Unknown AIME 2025 · unknown for its base model Qwen2.5: training data not disclosed in enough detail

Test data seen MATH-500 · via base model Qwen2.5: a contamination analysis found test examples in the pre-training data (Exact completion of 21.2% of MATH-500 problems from a 60% prefix (Table 2).) [third-party study]

Evidence (2)
“while leveraging the pre-trained Qwen2.5 large language model (LLM) (Yang et al., 2024a) as the initialization for the LLM component”
arxiv.org · paper · Section 2.2.2 Training Recipe · retrieved 2026-10-15
“revealing strong signs of data contamination in the Qwen2.5 series models when evaluated on commonly used benchmarks, such as MATH-500, AMC, and AIME2024.”
arxiv.org third-party study· study · Section 4, Table 2 · retrieved 2026-10-15

Notes

No benchmark is named as training data and no decontamination statement against multimodal benchmarks was found in the report. Training-data description is by category only. Accepts images and videos (model card: 'it is highly capable of analyzing texts, charts, icons, graphics, and layouts within images'). Accepts image input: “it is highly capable of analyzing texts, charts, icons, graphics, and layouts within images”

Sources