Contamination Map Methodology

Text / LLM / Model

Claude 3.5 Sonnet

Organisation
Anthropic
Family
Claude · version 3.5
Released
2024-06-21 (anthropic.com/news/claude-3-5-sonnet shows Jun 21, 2024 (first API snapshot is named claude-3-5-sonnet-20240620))
Training cutoff
2024-04-01
Evidence
“Training data cut-off Apr 2024 Aug 2023 Aug 2023 Aug 2023”
web.archive.org · website · Model comparison table (columns: Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku); Wayback Machine copy of docs.anthropic.com, 2024-10-31 · retrieved 2026-10-15
Checkpoints
claude-3-5-sonnet-20240620, claude-3-5-sonnet-20241022
Weights
Closed · proprietary
Links
Model card

Training data disclosure: None

Level determined for 9% of benchmark cells.

Benchmark cells

Classic LLM benchmarks

Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail

Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail

Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail

Unknown HumanEval · training data not disclosed in enough detail

Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail

Unknown HellaSwag · training data not disclosed in enough detail

Unknown TruthfulQA · training data not disclosed in enough detail

Unknown WinoGrande · training data not disclosed in enough detail

Multimodal LLM benchmarks

Unknown MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning) · training data not disclosed in enough detail

Unknown MMMU-Pro · training data not disclosed in enough detail

Unknown MathVista · training data not disclosed in enough detail

Unknown ChartQA · training data not disclosed in enough detail

Unknown DocVQA · training data not disclosed in enough detail

Unknown AI2D (AI2 Diagrams) · training data not disclosed in enough detail

Not seen CharXiv · its test items were published from 2024-06-26, after the model's stated training-data cutoff (2024-04-01)

Evidence (3)
“Training data cut-off Apr 2024 Aug 2023 Aug 2023 Aug 2023”
web.archive.org · website · Model comparison table (columns: Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku); Wayback Machine copy of docs.anthropic.com, 2024-10-31 · retrieved 2026-10-15
“arXiv:2406.18521v1 [cs.CL] 26 Jun 2024”
arxiv.org · paper · arXiv v1 header · retrieved 2026-10-15
“we release all QA pairs for the validation set and keep the answers to the test set private to prevent data leakage.”
arxiv.org · paper · Section 3, footnote 4 · retrieved 2026-10-15

Recent LLM benchmarks

Unknown GPQA Diamond · training data not disclosed in enough detail

Unknown MMLU-Pro · training data not disclosed in enough detail

Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail

Unknown SWE-bench Verified · training data not disclosed in enough detail

Unknown AIME 2024 · training data not disclosed in enough detail

Not seen AIME 2025 · its test items were published from 2025-02-06, after the model's stated training-data cutoff (2024-04-01)

Evidence (3)
“Training data cut-off Apr 2024 Aug 2023 Aug 2023 Aug 2023”
web.archive.org · website · Model comparison table (columns: Claude 3.5 Sonnet, Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku); Wayback Machine copy of docs.anthropic.com, 2024-10-31 · retrieved 2026-10-15
“2025 AIME I problems and solutions. The test was held on Thursday, February 6, 2025.”
web.archive.org · website · retrieved 2026-10-15
“This year the AIME I will take place on February 6, 2025, while the alternate AIME II will take place on February 12, 2025.”
maa.org · website · retrieved 2026-10-15

Unknown MATH-500 · training data not disclosed in enough detail

Notes

The model card addendum contains evaluations and safety results but no training-data description and no contamination statement. The cutoff comes from an archived copy of Anthropic's own docs; current Anthropic pages no longer list Claude 3.5 Sonnet. The upgraded 2024-10-22 snapshot is listed in the same column with the same Apr 2024 cutoff. Accepts image input: “Claude 3.5 Sonnet Our most intelligent model Text and image input Text output”

Sources