Text / LLM / Model

Molmo2

Organisation
Allen Institute for AI (Ai2)
Family
Molmo · version 2
Released
2025-12-11 (Ai2 blog post 'Molmo 2' dated December 11, 2025 (https://allenai.org/blog/molmo2); arXiv v1 2026-01-15)
Checkpoints
allenai/Molmo2-4B, allenai/Molmo2-8B, allenai/Molmo2-O-7B
Weights
Open · Apache 2.0
Links

Training data disclosure: Partial list

Level determined for 18% of benchmark cells.

“We train on the Molmo2 data, image data from PixMo, and various open-source datasets.”
arxiv.org · paper · Section 3.2 Training · retrieved 2026-10-15
“We create nine new datasets (in pink) to train Molmo2.”
arxiv.org · paper · Table 1 caption · retrieved 2026-10-15

Built on

Everything these base models may have seen is inherited by Molmo2.

Other declared training data

Benchmark cells

Classic LLM benchmarks

Unknown MMLU (Measuring Massive Multitask Language Understanding) · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown GSM8K (Grade School Math 8K) · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown MATH (Mathematics Aptitude Test of Heuristics) · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown HumanEval · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown HellaSwag · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown TruthfulQA · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown WinoGrande · unknown for its base model Qwen3: training data not disclosed in enough detail

Multimodal LLM benchmarks

Unknown MMMU (Massive Multi-discipline Multimodal Understanding and Reasoning) · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown MMMU-Pro · unknown for its base model Qwen3: training data not disclosed in enough detail

Related data seen MathVista · in fine-tuning data: related dataset ChartQA was used

Evidence (6)
“open-source image datasets [48, 130, 101, 102, 103, 105, 65, 125, 94, 95, 14, 2, 61, 62, 107] following Molmo with CoSyn [172] instead of PixMo-Docs”
arxiv.org · paper · Table 1 (Image QA group) · retrieved 2026-10-15
“"chart_qa_weighted",”
raw.githubusercontent.com · code · IMAGE_ACADEMIC_DATASETS · retrieved 2026-10-15
“split="train",”
raw.githubusercontent.com · code · launch_scripts/sft.py, DataLoaderConfig · retrieved 2026-10-15
“It consists of 6,141 examples, derived from 28 existing multimodal datasets involving mathematics and 3 newly created datasets”
arxiv.org · paper · Abstract · retrieved 2026-10-15
“ChartQA (2022)”
arxiv.org · paper · Table 5 (Appendix C) · retrieved 2026-10-15
“"dataset": "ChartQA",”
huggingface.co · dataset card · source.json · retrieved 2026-10-15

Training data seen ChartQA · in fine-tuning data: only the training split was used, not the benchmark's test data

Evidence (3)
“open-source image datasets [48, 130, 101, 102, 103, 105, 65, 125, 94, 95, 14, 2, 61, 62, 107] following Molmo with CoSyn [172] instead of PixMo-Docs”
arxiv.org · paper · Table 1 (Image QA group) · retrieved 2026-10-15
“"chart_qa_weighted",”
raw.githubusercontent.com · code · IMAGE_ACADEMIC_DATASETS · retrieved 2026-10-15
“split="train",”
raw.githubusercontent.com · code · launch_scripts/sft.py, DataLoaderConfig · retrieved 2026-10-15

Training data seen DocVQA · in fine-tuning data: only the training split was used, not the benchmark's test data

Evidence (3)
“open-source image datasets [48, 130, 101, 102, 103, 105, 65, 125, 94, 95, 14, 2, 61, 62, 107] following Molmo with CoSyn [172] instead of PixMo-Docs”
arxiv.org · paper · Table 1 (Image QA group) · retrieved 2026-10-15
“"doc_qa",”
raw.githubusercontent.com · code · IMAGE_ACADEMIC_DATASETS · retrieved 2026-10-15
“split="train",”
raw.githubusercontent.com · code · launch_scripts/sft.py, DataLoaderConfig · retrieved 2026-10-15

Training data seen AI2D (AI2 Diagrams) · in fine-tuning data: only the training split was used, not the benchmark's test data

Evidence (3)
“open-source image datasets [48, 130, 101, 102, 103, 105, 65, 125, 94, 95, 14, 2, 61, 62, 107] following Molmo with CoSyn [172] instead of PixMo-Docs”
arxiv.org · paper · Table 1 (Image QA group) · retrieved 2026-10-15
“"ai2_diagram_v2_mix_transparent",”
raw.githubusercontent.com · code · IMAGE_ACADEMIC_DATASETS · retrieved 2026-10-15
“split="train",”
raw.githubusercontent.com · code · launch_scripts/sft.py, DataLoaderConfig · retrieved 2026-10-15

Unknown CharXiv · unknown for its base model Qwen3: training data not disclosed in enough detail

Recent LLM benchmarks

Unknown GPQA Diamond · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown MMLU-Pro · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown LiveCodeBench (code generation, release_v6) · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown SWE-bench Verified · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown AIME 2024 · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown AIME 2025 · unknown for its base model Qwen3: training data not disclosed in enough detail

Unknown MATH-500 · unknown for its base model Qwen3: training data not disclosed in enough detail

Notes

Paper notes Molmo2 lags on reasoning benchmarks (MathVista, MMMU), 'possibly due to the lack of multi-modal reasoning training data'. A decontamination step is reported only for MOSE videos against the MeViS validation set (video tracking), not for the multimodal benchmarks here. Accepts image input: “Molmo2 is a family of open vision-language models developed by the Allen Institute for AI (Ai2) that support image, video and multi-image understanding and grounding.”

Sources