Contamination MapText / LLM foundation models
Which benchmark datasets could each model have seen during training? Every coloured square links to a verbatim quote from the model's paper, model card or repository. Grey means not enough has been disclosed to tell.
Contamination map
- TestTest data seen54
- SeenSeen, split unclear8
- TrainTraining data seen15
- Related data seen20
- NoNot seen84
- ?Unknown1435
Rows are models, grouped by family; columns are benchmark datasets. Hover a square for the reason, click it for the evidence. Empty outlines mark benchmarks a model cannot take (e.g. image benchmarks for text-only models).
Benchmarks
-
Classic LLM benchmarks
8 datasets
-
Recent LLM benchmarks
7 datasets
-
Multimodal LLM benchmarks
7 datasets
Models
| Model | Organisation | Released | Disclosure | Determined |
|---|---|---|---|---|
| BLOOM | BigScience | 2022-07-11 | Partial list | 0% |
| Claude 3 (Opus, Sonnet, Haiku) | Anthropic | 2024-03-04 | Vague | 13% |
| Claude 3.5 Sonnet | Anthropic | 2024-06-21 | None | 9% |
| Claude 3.7 Sonnet | Anthropic | 2025-02-24 | Vague | 4% |
| Claude Fable 5 / Claude Mythos 5 | Anthropic | 2026-06-09 | Partial list | 0% |
| Claude Mythos Preview | Anthropic | 2026-04-07 | Partial list | 18% |
| Claude Opus 4 and Claude Sonnet 4 | Anthropic | 2025-05-22 | Vague | 0% |
| Claude Opus 4.5 | Anthropic | 2025-11-24 | Vague | 13% |
| Claude Opus 5 | Anthropic | 2026-07-24 | Partial list | 0% |
| Claude Sonnet 5 | Anthropic | 2026-06-30 | Partial list | 0% |
| DCLM-7B (DCLM-Baseline-7B) | Apple / DataComp-LM (DCLM) team | 2024-07-11 | Full list | 53% |
| DCLM-7B-IT (DCLM-IT-7B) | Apple / DataComp-LM (DCLM) team | 2024-06-15 | Partial list | 26% |
| DeepSeek-R1 | DeepSeek-AI | 2025-01-20 | Vague | 53% |
| DeepSeek-R1-0528 | DeepSeek-AI | 2025-05-28 | None | 0% |
| DeepSeek-R1-Distill | DeepSeek-AI | 2025-01-20 | Vague | 0% |
| DeepSeek-V3 | DeepSeek-AI | 2024-12-26 | Vague | 6% |
| DeepSeek-V3-0324 | DeepSeek-AI | 2025-03-25 | None | 0% |
| DeepSeek-V3.1 | DeepSeek-AI | 2025-08-21 | Vague | 0% |
| DeepSeek-V3.2 | DeepSeek-AI | 2025-12-01 | Vague | 0% |
| DeepSeek-V3.2-Exp | DeepSeek-AI | 2025-09-29 | Vague | 0% |
| DeepSeek-V4 (Pro / Flash) | DeepSeek-AI | 2026-04-24 | Vague | 0% |
| DeepSeek-V4.1-Flash | DeepSeek-AI | 2026-09-10 | Vague | 0% |
| Gemini 1.5 (Pro, Flash) | Google DeepMind | 2024-02-15 | Vague | 13% |
| Gemini 2.5 (Pro, Flash) | Google DeepMind | 2025-03-25 | Vague | 4% |
| Gemini 3 Pro | Google DeepMind | 2025-11-18 | Vague | 4% |
| Gemini 4 Argon | Google DeepMind | 2026-10-13 | Vague | 0% |
| Gemma 2 | Google DeepMind | 2024-06-27 | Vague | 0% |
| Gemma 3 | Google DeepMind | 2025-03-12 | Vague | 4% |
| Gemma 3n | Google DeepMind | 2025-06-26 | Vague | 9% |
| Gemma 4 | Google DeepMind | 2026-04-02 | Vague | 4% |
| GPT-4 | OpenAI | 2023-03-14 | Partial list | 59% |
| GPT-4o | OpenAI | 2024-05-13 | Vague | 13% |
| GPT-5 | OpenAI | 2025-08-07 | Vague | 4% |
| GPT-5.2 | OpenAI | 2025-12-11 | Partial list | 4% |
| GPT-6 Astra | OpenAI | 2026-09-03 | Partial list | 0% |
| gpt-oss | OpenAI | 2025-08-05 | Vague | 6% |
| InternVL3 | Shanghai AI Laboratory (OpenGVLab) | 2025-04-10 | Partial list | 31% |
| InternVL3.5 | Shanghai AI Laboratory (OpenGVLab) | 2025-08-25 | Partial list | 18% |
| Llama 2 | Meta | 2023-07-18 | Vague | 40% |
| Llama 3.1 | Meta | 2024-07-23 | Vague | 66% |
| Llama 3.2 (1B, 3B) | Meta | 2024-09-25 | Vague | 13% |
| Llama 3.2-Vision (11B, 90B) | Meta | 2024-09-25 | Vague | 13% |
| Llama 3.3 70B | Meta | 2024-12-04 | Vague | 13% |
| Llama 4 (Scout, Maverick) | Meta | 2025-04-05 | Vague | 4% |
| Meta Llama 3 | Meta | 2024-04-18 | Vague | 13% |
| LLaVA-OneVision | LLaVA-NeXT team (LMMs-Lab), with ByteDance | 2024-08-06 | Partial list | 18% |
| LLaVA-OneVision-1.5 | LLaVA-OneVision Community Contributors (EvolvingLMMs-Lab) | 2025-09-28 | Partial list | 18% |
| K2-Horizon | MBZUAI Institute of Foundation Models (IFM) / LLM360 | 2026-09-01 | Partial list | 0% |
| K2-V2 | LLM360 / MBZUAI Institute of Foundation Models (IFM) | 2025-12-05 | Full list | 6% |
| LLM360 K2-65B (K2 Diamond) | LLM360 (MBZUAI / Petuum) | 2024-04-17 | Partial list | 20% |
| LLM360 Amber | LLM360 (MBZUAI / Petuum) | 2023-12-11 | Full list | 0% |
| Ministral 3 | Mistral AI | 2025-12-02 | Vague | 0% |
| Mistral 7B | Mistral AI | 2023-09-27 | Vague | 0% |
| Mistral Large 3 | Mistral AI | 2025-12-02 | None | 0% |
| Mistral Medium 3.5 | Mistral AI | 2026-05-22 | None | 4% |
| Mistral Small 4 | Mistral AI | 2026-03-16 | None | 4% |
| Mixtral 8x7B | Mistral AI | 2023-12-11 | Vague | 0% |
| Molmo (0924) | Allen Institute for AI (Ai2) | 2024-09-25 | Partial list | 18% |
| Molmo2 | Allen Institute for AI (Ai2) | 2025-12-11 | Partial list | 18% |
| OpenAI o1 | OpenAI | 2024-12-05 | Vague | 13% |
| OpenAI o3 | OpenAI | 2025-04-16 | Vague | 13% |
| OLMo (February 2024) | Allen Institute for AI (AI2) | 2024-02-01 | Full list | 26% |
| OLMo 2 (November 2024) | Allen Institute for AI (Ai2) | 2024-11-26 | Full list | 60% |
| Olmo 3 | Allen Institute for AI (Ai2) | 2025-11-20 | Full list | 6% |
| Olmo 3.1 | Allen Institute for AI (Ai2) | 2025-12-12 | Full list | 6% |
| Olmo Hybrid 7B | Allen Institute for AI (Ai2) | 2026-03-05 | Full list | 6% |
| Phi-3 (mini / small / medium) | Microsoft | 2024-04-22 | Vague | 13% |
| Phi-3.5 (mini / MoE) | Microsoft | 2024-08-16 | Vague | 13% |
| Phi-4 | Microsoft | 2024-12-12 | Vague | 60% |
| Phi-4-mini | Microsoft | 2025-02-19 | Vague | 6% |
| Phi-4-mini-flash-reasoning | Microsoft | 2025-06-19 | Vague | 6% |
| Phi-4-mini-reasoning | Microsoft | 2025-04-30 | Partial list | 6% |
| Phi-4-reasoning / Phi-4-reasoning-plus | Microsoft | 2025-04-30 | Vague | 6% |
| Phi-mini-MoE / Phi-tiny-MoE (SlimMoE) | Microsoft | 2025-06-23 | Vague | 13% |
| Pixtral 12B | Mistral AI | 2024-09-17 | Vague | 0% |
| Pixtral Large | Mistral AI | 2024-11-18 | Vague | 0% |
| Pythia | EleutherAI | 2023-04-03 | Full list | 20% |
| Pythia (deduplicated) | EleutherAI | 2023-04-03 | Full list | 20% |
| Qwen2.5 | Alibaba Cloud (Qwen Team) | 2024-09-19 | Vague | 20% |
| Qwen2.5-VL | Alibaba Cloud (Qwen Team) | 2025-01-26 | Vague | 13% |
| Qwen3 | Alibaba Cloud (Qwen Team) | 2025-04-29 | Vague | 0% |
| Qwen3-2507 (Instruct-2507 / Thinking-2507) | Alibaba Cloud (Qwen Team) | 2025-07-21 | None | 0% |
| Qwen3-Next | Alibaba Cloud (Qwen Team) | 2025-09-09 | Vague | 0% |
| Qwen3-VL | Alibaba Cloud (Qwen Team) | 2025-09-22 | Vague | 0% |
| Qwen3.5 | Alibaba Cloud (Qwen Team) | 2026-02-16 | Vague | 0% |
| Qwen3.6 | Alibaba Cloud (Qwen Team) | 2026-04-15 | None | 0% |
| Qwen3.8 | Alibaba Cloud (Qwen Team) | 2026-08-05 | None | 0% |
| Qwen3.8-Flash-Next | Alibaba Cloud (Qwen Team) | 2026-08-24 | None | 0% |
| SmolLM2 | Hugging Face | 2024-10-30 | Full list | 46% |
| SmolLM3 | Hugging Face | 2025-07-08 | Full list | 0% |
Datasets
- AI2D (AI2 Diagrams)
- AIME 2024
- AIME 2025
- ARC-Challenge (AI2 Reasoning Challenge, Challenge Set)
- ChartQA
- CharXiv
- DocVQA
- GPQA (Graduate-Level Google-Proof Q&A)
- GPQA Diamond
- GSM8K (Grade School Math 8K)
- HellaSwag
- HumanEval
- LiveCodeBench (code generation, release_v6)
- MATH (Mathematics Aptitude Test of Heuristics)
- MATH-500
- MathVista
- MMLU (Measuring Massive Multitask Language Understanding)
- MMLU-Pro
- MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning)
- MMMU-Pro
- SWE-bench
- SWE-bench Verified
- TruthfulQA
- WinoGrande