Contamination MapText / LLM / Model
Gemini 1.5 (Pro, Flash)
- Organisation
- Google DeepMind
- Family
- Gemini · version 1.5
- Released
- 2024-02-15 (blog.google post 'Introducing Gemini 1.5' (datePublished 2024-02-15))
- Training cutoff
- 2023-11-01
Evidence
“Knowledge cutoff November 2023 November 2023 June 2024 June 2024 January 2025 January 2025”
arxiv.org · paper · Gemini 2.5 report, Table 1 (columns: 1.5 Flash, 1.5 Pro, 2.0 Flash-Lite, 2.0 Flash, 2.5 Flash, 2.5 Pro) · retrieved 2026-10-15 - Checkpoints
- gemini-1.5-pro-002, gemini-1.5-flash-002
- Weights
- Closed · proprietary
- Links
- Paper
Training data disclosure: Vague
Level determined for 13% of benchmark cells.
- TestTest data seen0
- SeenSeen, split unclear0
- TrainTraining data seen0
- Related data seen0
- NoNot seen3
- ?Unknown19
“Our pre-training dataset includes data sourced across many different domains, including web documents and code, and incorporates image, audio, and video content.”
Other declared training data
-
web documents and code (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“Our pre-training dataset includes data sourced across many different domains, including web documents and code, and incorporates image, audio, and video content.”
arxiv.org · paper · Section 4 · retrieved 2026-10-15 -
multimodal instruction-tuning data and human preference data (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“we finetuned Gemini 1.5 models on a collection of multimodal data (containing paired instructions and appropriate responses), with further tuning based on human preference data.”
arxiv.org · paper · Section 4 · retrieved 2026-10-15
Benchmark cells
Classic LLM benchmarks
Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail
Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail
Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail
Unknown HumanEval · training data not disclosed in enough detail
Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail
Unknown HellaSwag · training data not disclosed in enough detail
Unknown TruthfulQA · training data not disclosed in enough detail
Unknown WinoGrande · training data not disclosed in enough detail
Multimodal LLM benchmarks
Unknown MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning) · training data not disclosed in enough detail
Unknown MMMU-Pro · training data not disclosed in enough detail
Unknown MathVista · training data not disclosed in enough detail
Unknown ChartQA · training data not disclosed in enough detail
Unknown DocVQA · training data not disclosed in enough detail
Unknown AI2D (AI2 Diagrams) · training data not disclosed in enough detail
Not seen CharXiv · its test items were published from 2024-06-26, after the model's stated training-data cutoff (2023-11-01)
Evidence (3)
“Knowledge cutoff November 2023 November 2023 June 2024 June 2024 January 2025 January 2025”
“arXiv:2406.18521v1 [cs.CL] 26 Jun 2024”
“we release all QA pairs for the validation set and keep the answers to the test set private to prevent data leakage.”
Recent LLM benchmarks
Unknown GPQA Diamond · training data not disclosed in enough detail
Unknown MMLU-Pro · training data not disclosed in enough detail
Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail
Unknown SWE-bench Verified · training data not disclosed in enough detail
Not seen AIME 2024 · its test items were published from 2024-01-31, after the model's stated training-data cutoff (2023-11-01)
Evidence (4)
“Knowledge cutoff November 2023 November 2023 June 2024 June 2024 January 2025 January 2025”
“2024 AIME I problems and solutions. The test was held on Wednesday, January 31 – Thursday, February 1, 2024.”
“AIME I (main AIME competition date): Thursday, Feb. 1, 2024 between 1:30 pm ET and 5:30 pm ET.”
“AIME II (alternate AIME competition date): Wednesday, Feb. 7, 2024 between 1:30 pm ET and 5:30 pm ET.”
Not seen AIME 2025 · its test items were published from 2025-02-06, after the model's stated training-data cutoff (2023-11-01)
Evidence (3)
“Knowledge cutoff November 2023 November 2023 June 2024 June 2024 January 2025 January 2025”
“2025 AIME I problems and solutions. The test was held on Thursday, February 6, 2025.”
“This year the AIME I will take place on February 6, 2025, while the alternate AIME II will take place on February 12, 2025.”
Unknown MATH-500 · training data not disclosed in enough detail
Notes
Decontamination stated without naming benchmarks: 'Gemini 1.5 employed standard n-gram based decontamination procedures to help mitigate this issue, however these n-gram based procedures are imperfect.' (Section 6.1). The report also says controlling HumanEval leakage on webpages and open-source repositories was 'a non-trivial task, even with conservative filtering heuristics', and shows (for Gemini 1.0 Ultra) that one epoch of the HumanEval test split raised scores from 74.4% to 89.0%; that experiment is not a finding about Gemini 1.5's training data. Checkpoints are the IDs in the Gemini 2.5 report Table 2; earlier -001 versions existed. Accepts image input: “since the model is natively multimodal and supports interleaving of data from different modalities, it can support a mix of audio, visual, text, and code inputs in the same input sequence.”
Sources
- Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context (arXiv 2403.05530v5) · paper · retrieved 2026-10-15
- Gemini 2.5 technical report (arXiv 2507.06261v6), Table 1-2 · paper · retrieved 2026-10-15
- Introducing Gemini 1.5 · website · retrieved 2026-10-15