Contamination Map Methodology

Text / LLM / Model

Gemini 2.5 (Pro, Flash)

Organisation
Google DeepMind
Family
Gemini · version 2.5
Released
2025-03-25 (blog.google post 'Gemini 2.5: Our newest Gemini model with thinking' (datePublished 2025-03-25))
Training cutoff
2025-01-01
Evidence
“with a cutoff date of June 2024 for 2.0 and January 2025 for 2.5.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15
“Knowledge cutoff January 2025”
ai.google.dev · website · gemini-2.5-pro model page · retrieved 2026-10-15
Checkpoints
gemini-2.5-pro, gemini-2.5-flash
Weights
Closed · proprietary
Links
Paper

Training data disclosure: Vague

Level determined for 4% of benchmark cells.

“Our pre-training dataset is a large-scale, diverse collection of data encompassing a wide range of domains and modalities, which includes publicly available web documents, code (various programming languages), images, audio (including speech and other audio types) and video”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15

Other declared training data

Benchmark cells

Classic LLM benchmarks

Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail

Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail

Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail

Unknown HumanEval · training data not disclosed in enough detail

Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail

Unknown HellaSwag · training data not disclosed in enough detail

Unknown TruthfulQA · training data not disclosed in enough detail

Unknown WinoGrande · training data not disclosed in enough detail

Multimodal LLM benchmarks

Unknown MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning) · training data not disclosed in enough detail

Unknown MMMU-Pro · training data not disclosed in enough detail

Unknown MathVista · training data not disclosed in enough detail

Unknown ChartQA · training data not disclosed in enough detail

Unknown DocVQA · training data not disclosed in enough detail

Unknown AI2D (AI2 Diagrams) · training data not disclosed in enough detail

Unknown CharXiv · training data not disclosed in enough detail

Recent LLM benchmarks

Unknown GPQA Diamond · training data not disclosed in enough detail

Unknown MMLU-Pro · training data not disclosed in enough detail

Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail

Unknown SWE-bench Verified · training data not disclosed in enough detail

Unknown AIME 2024 · training data not disclosed in enough detail

Not seen AIME 2025 · its test items were published from 2025-02-06, after the model's stated training-data cutoff (2025-01-01)

Evidence (4)
“with a cutoff date of June 2024 for 2.0 and January 2025 for 2.5.”
arxiv.org · paper · Section 2.2 · retrieved 2026-10-15
“Knowledge cutoff January 2025”
ai.google.dev · website · gemini-2.5-pro model page · retrieved 2026-10-15
“2025 AIME I problems and solutions. The test was held on Thursday, February 6, 2025.”
web.archive.org · website · retrieved 2026-10-15
“This year the AIME I will take place on February 6, 2025, while the alternate AIME II will take place on February 12, 2025.”
maa.org · website · retrieved 2026-10-15

Unknown MATH-500 · training data not disclosed in enough detail

Notes

Decontamination stated without naming benchmarks: 'in addition to the standard n-gram based decontamination we used in Gemini 1.5, we also employed semantic-similarity and model based decontamination procedures to help mitigate evaluation set leakage.' (Section 3). No positive contamination finding reported. Accepts image input: “The Gemini 2.X series are all built to be natively multimodal, supporting long context inputs of >1” Accepts image input: “Input modalities Text, Image, Video, Audio”

Sources