Text / LLM / Model
Olmo Hybrid 7B
- Organisation
- Allen Institute for AI (Ai2)
- Family
- OLMo
- Released
- 2026-03-05 (Ai2 blog 'Introducing Olmo Hybrid' dated March 5, 2026 (https://allenai.org/blog/olmohybrid); arXiv 2604.03444 v1 3 Apr 2026)
- Training cutoff
- 2024-12-01
Evidence
“Date cutoff:** Dec. 2024.”
huggingface.co · model card · Model Description · retrieved 2026-10-15 - Checkpoints
- allenai/Olmo-Hybrid-7B, allenai/Olmo-Hybrid-Think-SFT-7B, allenai/Olmo-Hybrid-Instruct-SFT-7B, allenai/Olmo-Hybrid-Instruct-DPO-7B
- Weights
- Open · apache-2.0
- Links
- PaperModel cardCode
Training data disclosure: Full list
Level determined for 6% of benchmark cells.
- TestTest data seen0
- SeenSeen, split unclear0
- TrainTraining data seen0
- Related data seen0
- NoNot seen1
- ?Unknown14
“Additionally, we use the improved data mix of Olmo 3 32B instead of the Olmo 3 7B mix.”
“The Instruct SFT and DPO data are identical to the Olmo 3 7B and 32B Instruct SFT and DPO variants.”
Training corpora
-
Dolma 3 Mix
· 6 sources (complete list) · pre-training
· homepage
Evidence that Olmo Hybrid 7B uses it
“Dataset: [dolma3_6T_mix](https://huggingface.co/datasets/allenai/dolma3_mix-6T)”
huggingface.co · model card · Stage 1 · retrieved 2026-10-15 Contents of this corpus
-
common_crawl (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“common_crawl | web pages | 4.51T”
huggingface.co · dataset card · Source Sizes · retrieved 2026-10-15 “Text extraction We start with 104 dumps from the CommonCrawl corpus, with a cutoff date of December”
arxiv.org · paper · Section 3.4 · retrieved 2026-10-15 -
olmocr_science_pdfs (broad web data: may contain benchmark test sets) · split: unknown
Crawled PDFs (238M unique PDFs, cutoff Dec 2024) converted with olmOCR.
Evidence
“olmocr_science_pdfs | academic papers | 805B”
huggingface.co · dataset card · Source Sizes · retrieved 2026-10-15 -
stack_edu (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“stack_edu | code | 409B”
huggingface.co · dataset card · Source Sizes · retrieved 2026-10-15 “For code data, we use Stack-Edu (Allal et al., 2025), an improved curation of GitHub repositories from”
arxiv.org · paper · Section 3.4 · retrieved 2026-10-15 -
finemath-3plus (broad web data: may contain benchmark test sets) · split: unknown
FineMath: Common Crawl math pages.
Evidence
“finemath-3plus | mathematics | 151B”
huggingface.co · dataset card · Source Sizes · retrieved 2026-10-15 “This data has a cutoff date of September 2024.”
arxiv.org · paper · Section 3.4 · retrieved 2026-10-15 -
rpj-proofpile-arxiv (not yet matched to a dataset) · split: unknown
Evidence
“rpj-proofpile-arxiv | research papers | 50.9B”
huggingface.co · dataset card · Source Sizes · retrieved 2026-10-15 -
dolma1_7-wiki-en (not yet matched to a dataset) · split: unknown
Evidence
“dolma1_7-wiki-en | encyclopedic | 2.51B”
huggingface.co · dataset card · Source Sizes · retrieved 2026-10-15
-
common_crawl (broad web data: may contain benchmark test sets) · split: unknown
-
Dolma 3 Dolmino Mix
· 24 sources (complete list) · pre-training
· homepage
Evidence that Olmo Hybrid 7B uses it
“we conduct two separate mid-training runs on different 100B token subsets of”
arxiv.org · paper · Section 3 · retrieved 2026-10-15 “Dataset: [dolma3-dolmino-mix-1025](https://huggingface.co/datasets/allenai/dolma3_dolmino_mix-100B-1025)”
huggingface.co · model card · Stage 2 · retrieved 2026-10-15 Contents of this corpus
-
TinyMATH Mind (not yet matched to a dataset) · split: unknown
Synthetic problems generated from the MATH training set.
Evidence
“TinyMATH Mind | Math (synth) | 898M (0.9%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 “TinyMATH For each of the 7500 examples in the MATH training set, we generate 100 new, similar”
arxiv.org · paper · Section 3.5 · retrieved 2026-10-15 -
TinyMATH PoT (not yet matched to a dataset) · split: unknown
Synthetic, derived from the MATH training set.
Evidence
“TinyMATH PoT | Math (synth) | 241M (0.24%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
CraneMath (not yet matched to a dataset) · split: unknown
Evidence
“CraneMath | Math (synth) | 5.62B (5.63%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
MegaMatt (not yet matched to a dataset) · split: unknown
Evidence
“MegaMatt | Math (synth) | 1.73B (1.73%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Dolmino Math (not yet matched to a dataset) · split: unknown
OLMo 2 Dolmino math subset (which contained GSM8K train and GSM8K-derived data), re-filtered by Olmo 3 decontamination, which removes matches of any benchmark split.
Evidence
“Dolmino Math | Math (synth) | 10.7B (10.7%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 “The version we use differs from the original only in additional filtering for decontamination.”
arxiv.org · paper · Section 3.5 · retrieved 2026-10-15 -
StackEdu (FIM) (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“StackEdu (FIM) | Code | 10.0B (10.0%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
CraneCode (not yet matched to a dataset) · split: unknown
Evidence
“CraneCode | Python (synth) | 10.0B (10.0%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Reddit To Flashcards (not yet matched to a dataset) · split: unknown
Evidence
“Reddit To Flashcards | QA (synth) | 5.90B (5.9%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Wiki To RCQA (not yet matched to a dataset) · split: unknown
Evidence
“Wiki To RCQA | QA (synth) | 3.0B (3.0%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Nemotron Synth QA (not yet matched to a dataset) · split: unknown
Evidence
“Nemotron Synth QA | QA (synth) | 5.0B (5.0%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Math Meta-Reasoning (not yet matched to a dataset) · split: unknown
Evidence
“Math Meta-Reasoning | Thinking (synth) | 381M (0.38%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Code Meta-Reasoning (not yet matched to a dataset) · split: unknown
Evidence
“Code Meta-Reasoning | Thinking (synth) | 459M (0.46%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Program-Verifiable (not yet matched to a dataset) · split: unknown
Evidence
“Program-Verifiable | Thinking (synth) | 159M (0.16%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
OMR Rewrite FullThoughts (not yet matched to a dataset) · split: unknown
Evidence
“OMR Rewrite FullThoughts | Thinking (synth) | 850M (0.85%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
QWQ Reasoning Traces (not yet matched to a dataset) · split: unknown
Evidence
“QWQ Reasoning Traces | Thinking (synth) | 1.87B (1.87%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
General Reasoning Mix (not yet matched to a dataset) · split: unknown
Evidence
“General Reasoning Mix | Thinking (synth) | 1.87B (1.87%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Gemini Reasoning Traces (not yet matched to a dataset) · split: unknown
Evidence
“Gemini Reasoning Traces | Thinking (synth) | 246M (0.25%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Llama Nemotron Reasoning Traces (not yet matched to a dataset) · split: unknown
Evidence
“Llama Nemotron Reasoning Traces | Thinking (synth) | 1.25B (1.25%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
OpenThoughts2 Reasoning Traces (not yet matched to a dataset) · split: unknown
Evidence
“OpenThoughts2 Reasoning Traces | Thinking (synth) | 1.25B (1.25%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Tulu 3 SFT (not yet matched to a dataset) · split: unknown
Evidence
“Tulu 3 SFT | Instruction (synth) | 1.1B (1.1%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Dolmino 1 Flan (not yet matched to a dataset) · split: unknown
Evidence
“Dolmino 1 Flan | Instruction (synth) | 5.0B (5.0%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
OLMOCR Science PDFs (High Q.) (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“OLMOCR Science PDFs (High Q.) | PDFs | 4.99B (5.0%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
STEM-Heavy Crawl (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“STEM-Heavy Crawl | Web pages | 4.99B (5.0%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Common Crawl (High Q.) (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“Common Crawl (High Q.) | Web pages | 22.4B (22.5%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15
-
TinyMATH Mind (not yet matched to a dataset) · split: unknown
-
Dolma 3 Longmino Mix
· 2 sources (complete list) · pre-training
· homepage
Evidence that Olmo Hybrid 7B uses it
“100B tokens of Dolma 3 Longmino Mix.”
arxiv.org · paper · Section 3 · retrieved 2026-10-15 Contents of this corpus
-
LC-s2pdf (long PDFs, incl. synthetic REX/CWE variants) (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“LC-s2pdf 32k-64k | PDFs | 4.81B (9.63%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15 -
Midtraining Data (not yet matched to a dataset) · split: unknown
Drawn from the Dolma 3 Dolmino pool.
Evidence
“Midtraining Data | Mix | 33.0B (66.1%)”
huggingface.co · dataset card · Dataset Sources · retrieved 2026-10-15
-
LC-s2pdf (long PDFs, incl. synthetic REX/CWE variants) (broad web data: may contain benchmark test sets) · split: unknown
Other declared training data
-
Dolci-Think-SFT-Olmo-Hybrid (not yet matched to a dataset) · split: unknown · fine-tuning
Olmo 3 Instruct SFT prompts with new thinking traces (DR Tulu, GPT-4.1) plus function-calling data.
Evidence
“The new Think SFT dataset is available at: https://hf.co/datasets/allenai/Dolci-Think-SFT-Olmo-Hybrid”
arxiv.org · paper · Section 5.3, footnote 6 · retrieved 2026-10-15 -
Dolci-Instruct-SFT (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“The Instruct SFT and DPO data are identical to the Olmo 3 7B and 32B Instruct SFT and DPO variants.”
arxiv.org · paper · Section 5.3 · retrieved 2026-10-15 -
Dolci-Instruct-DPO (not yet matched to a dataset) · split: unknown · fine-tuning
Evidence
“The Instruct SFT and DPO data are identical to the Olmo 3 7B and 32B Instruct SFT and DPO variants.”
arxiv.org · paper · Section 5.3 · retrieved 2026-10-15
Benchmark cells
Classic LLM benchmarks
Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail
Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail
Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail
Unknown HumanEval · training data not disclosed in enough detail
Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail
Unknown HellaSwag · training data not disclosed in enough detail
Unknown TruthfulQA · training data not disclosed in enough detail
Unknown WinoGrande · training data not disclosed in enough detail
Recent LLM benchmarks
Unknown GPQA Diamond · training data not disclosed in enough detail
Unknown MMLU-Pro · training data not disclosed in enough detail
Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail
Unknown SWE-bench Verified · training data not disclosed in enough detail
Unknown AIME 2024 · training data not disclosed in enough detail
Not seen AIME 2025 · its test items were published from 2025-02-06, after the model's stated training-data cutoff (2024-12-01)
Evidence (3)
“Date cutoff:** Dec. 2024.”
“2025 AIME I problems and solutions. The test was held on Thursday, February 6, 2025.”
“This year the AIME I will take place on February 6, 2025, while the alternate AIME II will take place on February 12, 2025.”
Unknown MATH-500 · training data not disclosed in enough detail
Notes
Same data pipeline as Olmo 3 (Dolma 3 + Dolci); post-trained models are preliminary (no RL stage). Decontamination is inherited from the Olmo 3 data (midtraining pool and Dolci post-training data were decontaminated in Olmo 3), but the Hybrid paper does not restate it, so no exclusions are recorded here.
Sources
- Olmo Hybrid blog post · website · retrieved 2026-10-15
- Olmo-Hybrid-7B card · model card · retrieved 2026-10-15
- Olmo Hybrid: From Theory to Practice and Back · paper · retrieved 2026-10-15