Contamination Map Methodology

Text / LLM / Model

Gemma 3

Organisation
Google DeepMind
Family
Gemma · version 3
Released
2025-03-12 (Google blog 'Introducing Gemma 3' dated Mar 12, 2025; the 270M size was announced on Aug 14, 2025 (Google Developers Blog))
Training cutoff
2024-08-01
Evidence
“The knowledge cutoff date for the training data was August 2024.”
ai.google.dev · model card · Model Data / Training Dataset · retrieved 2026-10-15
Checkpoints
google/gemma-3-1b-pt, google/gemma-3-4b-pt, google/gemma-3-12b-pt, google/gemma-3-27b-pt, google/gemma-3-1b-it, google/gemma-3-4b-it, google/gemma-3-12b-it, google/gemma-3-27b-it, google/gemma-3-270m, google/gemma-3-270m-it
Weights
Open · Gemma Terms of Use
Links
PaperModel cardCode

Training data disclosure: Vague

Level determined for 4% of benchmark cells.

“These models were trained on a dataset of text data that includes a wide variety of sources.”
ai.google.dev · model card · Model Data / Training Dataset · retrieved 2026-10-15
“We also increase the amount of multilingual data to improve language coverage.”
arxiv.org · paper · Section 2.2 Pre-training, Training data · retrieved 2026-10-15

Other declared training data

Benchmark cells

Classic LLM benchmarks

Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail

Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail

Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail

Unknown HumanEval · training data not disclosed in enough detail

Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail

Unknown HellaSwag · training data not disclosed in enough detail

Unknown TruthfulQA · training data not disclosed in enough detail

Unknown WinoGrande · training data not disclosed in enough detail

Multimodal LLM benchmarks

Unknown MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning) · training data not disclosed in enough detail

Unknown MMMU-Pro · training data not disclosed in enough detail

Unknown MathVista · training data not disclosed in enough detail

Unknown ChartQA · training data not disclosed in enough detail

Unknown DocVQA · training data not disclosed in enough detail

Unknown AI2D (AI2 Diagrams) · training data not disclosed in enough detail

Unknown CharXiv · training data not disclosed in enough detail

Recent LLM benchmarks

Unknown GPQA Diamond · training data not disclosed in enough detail

Unknown MMLU-Pro · training data not disclosed in enough detail

Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail

Unknown SWE-bench Verified · training data not disclosed in enough detail

Unknown AIME 2024 · training data not disclosed in enough detail

Not seen AIME 2025 · its test items were published from 2025-02-06, after the model's stated training-data cutoff (2024-08-01)

Evidence (3)
“The knowledge cutoff date for the training data was August 2024.”
ai.google.dev · model card · Model Data / Training Dataset · retrieved 2026-10-15
“2025 AIME I problems and solutions. The test was held on Thursday, February 6, 2025.”
web.archive.org · website · retrieved 2026-10-15
“This year the AIME I will take place on February 6, 2025, while the alternate AIME II will take place on February 12, 2025.”
maa.org · website · retrieved 2026-10-15

Unknown MATH-500 · training data not disclosed in enough detail

Notes

The paper adds: 'despite the use of decontamination techniques, there is always a risk of contamination of these probes'. The 270M size, released later (Aug 2025), is described in the same model card with the same August 2024 cutoff. Decontamination statements not shown as cells (they name no benchmark tracked here, or none at all): evaluation sets (not named). Accepts image input: “Gemma 3 models are multimodal, handling text and image input and generating text output”

Sources