Contamination Map Methodology

Text / LLM / Dataset

ChartQA

Questions about real charts crawled from Statista, Pew Research, Our World in Data and OECD (20,882 charts). Two parts: ChartQA-H with 9,608 human-written questions (AMT) and ChartQA-M with 23,111 questions generated by T5 from Statista's human-written summaries. Random train/val/test split; test = 1,250 human + 1,250 machine-generated questions (2,500), which is the evaluated split (relaxed accuracy). Test questions and answers are public (vis-nlp/ChartQA GitHub 'ChartQA Dataset/test' with test_human.json and test_augmented.json; HF copies).

Also known as
ChartQA, ChartQA (test), chartqa, ChartQA-H, ChartQA-M, HuggingFaceM4/ChartQA
Domain
chart question answering
License
GPL-3.0 (GitHub repository license; HuggingFaceM4/ChartQA card metadata gpl-3.0)
Homepage
github.com

Exposure by model

BenchmarkModelLevelReason
Multimodal LLM benchmarks Claude 3.5 Sonnet Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude 3.7 Sonnet Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude 3 (Opus, Sonnet, Haiku) Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude Opus 4 and Claude Sonnet 4 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude Fable 5 / Claude Mythos 5 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude Mythos Preview Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude Opus 4.5 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude Opus 5 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Claude Sonnet 5 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Gemini 1.5 (Pro, Flash) Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Gemini 2.5 (Pro, Flash) Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Gemini 3 Pro Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Gemini 4 Argon Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Gemma 3 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Gemma 3n Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Gemma 4 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks GPT-4 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks GPT-4o Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks GPT-5.2 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks GPT-5 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks InternVL3.5 Seen, split unclear in fine-tuning data: the dataset was used; the source does not say which part
Multimodal LLM benchmarks InternVL3 Seen, split unclear the dataset was used; the source does not say which part
Multimodal LLM benchmarks Llama 3.2-Vision (11B, 90B) Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Llama 4 (Scout, Maverick) Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks LLaVA-OneVision-1.5 Seen, split unclear in fine-tuning data: the dataset was used; the source does not say which part
Multimodal LLM benchmarks LLaVA-OneVision Training data seen in fine-tuning data: only the training split was used, not the benchmark's test data
Multimodal LLM benchmarks Mistral Large 3 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Mistral Medium 3.5 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Mistral Small 4 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Molmo (0924) Training data seen in fine-tuning data: only the training split was used, not the benchmark's test data; the test portion was explicitly excluded
Multimodal LLM benchmarks Molmo2 Training data seen in fine-tuning data: only the training split was used, not the benchmark's test data
Multimodal LLM benchmarks OpenAI o1 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks OpenAI o3 Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Pixtral 12B Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Pixtral Large Unknown training data not disclosed in enough detail
Multimodal LLM benchmarks Qwen2.5-VL Unknown unknown for its base model Qwen2.5: training data not disclosed in enough detail
Multimodal LLM benchmarks Qwen3-VL Unknown unknown for its base model Qwen3: training data not disclosed in enough detail
Multimodal LLM benchmarks Qwen3.5 Unknown training data not disclosed in enough detail

Notes

Released March 2022 (arXiv v1 19 Mar 2022; GitHub repo created 2022-03-07); the chart images themselves come from public websites and predate the questions. published_from left null (pre-2023 benchmark). Its train split is widely used in VLM instruction tuning, which is a train-split (not test) exposure.

Sources