Contamination Map Methodology

Text / LLM / Model

Gemini 3 Pro

Organisation
Google DeepMind
Family
Gemini · version 3
Released
2025-11-18 (blog.google post 'Gemini 3: Introducing the latest Gemini AI model from Google' (datePublished 2025-11-18); model card: 'Model Release: November 2025')
Training cutoff
2025-01-01
Evidence
“The knowledge cutoff date for Gemini 3 Pro was January 2025.”
storage.googleapis.com · model card · Known Limitations · retrieved 2026-10-15
Checkpoints
gemini-3-pro-preview
Weights
Closed · proprietary
Links
Model card

Training data disclosure: Vague

Level determined for 4% of benchmark cells.

“The pre-training dataset was a large-scale, diverse collection of data encompassing a wide range of domains and modalities, which included publicly-available web-documents, text, code, images, audio (including speech and other audio types) and video.”
storage.googleapis.com · model card · Model Data: Training Dataset · retrieved 2026-10-15
“The training dataset also includes: publicly available datasets that are readily downloadable; data obtained by crawlers; licensed data obtained via commercial licensing agreements;”
storage.googleapis.com · model card · Model Data: Training Dataset · retrieved 2026-10-15

Other declared training data

Benchmark cells

Classic LLM benchmarks

Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail

Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail

Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail

Unknown HumanEval · training data not disclosed in enough detail

Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail

Unknown HellaSwag · training data not disclosed in enough detail

Unknown TruthfulQA · training data not disclosed in enough detail

Unknown WinoGrande · training data not disclosed in enough detail

Multimodal LLM benchmarks

Unknown MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning) · training data not disclosed in enough detail

Unknown MMMU-Pro · training data not disclosed in enough detail

Unknown MathVista · training data not disclosed in enough detail

Unknown ChartQA · training data not disclosed in enough detail

Unknown DocVQA · training data not disclosed in enough detail

Unknown AI2D (AI2 Diagrams) · training data not disclosed in enough detail

Unknown CharXiv · training data not disclosed in enough detail

Recent LLM benchmarks

Unknown GPQA Diamond · training data not disclosed in enough detail

Unknown MMLU-Pro · training data not disclosed in enough detail

Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail

Unknown SWE-bench Verified · training data not disclosed in enough detail

Unknown AIME 2024 · training data not disclosed in enough detail

Not seen AIME 2025 · its test items were published from 2025-02-06, after the model's stated training-data cutoff (2025-01-01)

Evidence (3)
“The knowledge cutoff date for Gemini 3 Pro was January 2025.”
storage.googleapis.com · model card · Known Limitations · retrieved 2026-10-15
“2025 AIME I problems and solutions. The test was held on Thursday, February 6, 2025.”
web.archive.org · website · retrieved 2026-10-15
“This year the AIME I will take place on February 6, 2025, while the alternate AIME II will take place on February 12, 2025.”
maa.org · website · retrieved 2026-10-15

Unknown MATH-500 · training data not disclosed in enough detail

Notes

Model card: Gemini 3 Pro is not a modification of a prior model; the Gemini 3 Pro family (3 Pro Image, 3 Flash, 3.1 Pro, 3.1 Flash Image, 3.1 Flash-Lite, 3.1 Flash Live, 3.5 Flash) is based on it. Training data also includes user data, other Google-acquired/generated data and AI-generated synthetic data. No contamination or decontamination statement in the model card. Other reported contamination, not shown as cells: swe-bench-verified: Third-party (OpenAI) elicitation study found strong SWE-bench Verified contamination examples for Gemini 3 Flash Preview, a model the Gemini 3 Pro card lists as based on Gemini 3 Pro Accepts image input: “Inputs: Text strings (e.g., a question, a prompt, document(s) to be summarized), images, audio, and video files, with a token context window of up to 1M.”

Sources