SmolLM3

Training data disclosure: Full list

Level determined for 0% of benchmark cells.

• TestTest data seen0
• SeenSeen, split unclear0
• TrainTraining data seen0
• Rel.Related data seen0
• NoNot seen0
• ?Unknown15

Training corpora

• FineWeb-Edu · 1 source (complete list) · pre-training · homepage
  Evidence that SmolLM3 uses it

  “Web: 85% (12% multilingual) - FineWeb-Edu, DCLM, FineWeb2 and FineWeb2-HQ”

  Contents of this corpus

  • FineWeb-Edu (filtered CommonCrawl dumps since 2013) (broad web data: may contain benchmark test sets) · split: unknown
    Evidence

    “Along with the dataset, which includes all filtered CommonCrawl dumps since 2013, we also release the educational classifier used for the filtering”

    “We then built 📚 FineWeb-Edu by filtering out samples with scores lower than 3.”

• SmolTalk2 · 16 sources (complete list) · fine-tuning · homepage
  Evidence that SmolLM3 uses it

  “This dataset contains three subsets (Mid, SFT, Preference) that correspond to the three phases of Post-Training for [SmolLM3-3B]”

  “Post-training included midtraining on 140B reasoning tokens followed by supervised fine-tuning and alignment via Anchored Preference Optimization (APO).”

  Contents of this corpus

  • Llama-Nemotron-Post-Training-Dataset (Mid, reasoning_r1) (not yet matched to a dataset) · split: unknown · fine-tuning

    Reasoning mid-training (SmolLM3 post-training phase 1).

    Evidence

    “[Llama-Nemotron-Post-Training-Dataset](https://huggingface.co/datasets/nvidia/Llama-Nemotron-Post-Training-Dataset): 3.64M rows”

  • OpenThoughts3-1.2M (not yet matched to a dataset) · split: unknown · fine-tuning

    Used in mid-training and in SFT (think and no_think variants).

    Evidence

    “[OpenThoughts3-1.2M](https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M): 1.14M rows.”

  • SmolTalk (re-generated/translated subsets: multilingual-8languages, everyday-conversations, systemchats-30k, smol-magpie-ultra, explore-instruct-rewriting, smol-rewrite, smol-summarize) (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “(multilingual-8languages): 244736 rows generated with Qwen3-32B with the prompts in SmolTalk.”

  • LongAlign-64k-context-lang-annotated (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “LongAlign-64k-context-lang-annotated: 7526 rows generated with Qwen3-32B with the prompts in LongAlign-64k.”

  • smolagents-toolcalling-traces (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “smolagents-toolcalling-traces: 9079 rows.”

  • Multi-Turn IF (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “Multi-Turn IF: 28217 rows.”

  • s1k-1.1 (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “[s1k-1.1](https://huggingface.co/datasets/open-r1/s1K-1.1): 835 rows.”

  • Aya dataset (Qwen3-32B think) (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “[Aya](https://huggingface.co/datasets/CohereLabs/aya_dataset): 15222 rows generated with Qwen3-32B with the prompts in Aya.”

  • Table-GPT (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “[Table-GPT](https://huggingface.co/datasets/LipengCS/Table-GPT): 13201 rows generated with Qwen3-32B with the prompts in Table-GPT.”

  • Mixture of Thoughts (science) (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “[Mixture of Thoughts](https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts) (science): 86110 rows where we remove the reasoning trace.”

  • Tulu 3 SFT Personas IF (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “[Tulu 3 SFT Personas IF](https://huggingface.co/datasets/allenai/tulu-3-sft-personas-instruction-following): 29970 rows.”

  • hermes-function-calling-v1 (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “[hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1): 8961 rows.”

  • OpenHermes-2.5 (not yet matched to a dataset) · split: unknown · fine-tuning

    Includes MetaMathQA (GSM8K/MATH-train-derived) per SmolLM2 paper.

    Evidence

    “[OpenHermes-2.5](https://huggingface.co/datasets/teknium/OpenHermes-2.5): 384900 rows.”

  • xlam-traces (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “xlam-traces_no_think”

  • Tulu 3 8B Preference Mixture (no_think) (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “[Tulu 3 8B Preference Mixture (`no_think`)](https://huggingface.co/datasets/allenai/llama-3.1-tulu-3-8b-preference-mixture): 231k rows.”

  • Tulu 3 8B Preference Mixture (think, Qwen3-32B chosen / Qwen3-0.6B rejected) (not yet matched to a dataset) · split: unknown · fine-tuning
    Evidence

    “Tulu 3 8B Preference Mixture (`think`): 216k rows where we generate the chosen responses with Qwen3-32B and the rejected responses with Qwen3-0.6B.”

Other declared training data

• FineWeb2 and FineWeb2-HQ (broad web data: may contain benchmark test sets) · split: unknown
  Evidence

  “Web: 85% (12% multilingual) - FineWeb-Edu, DCLM, FineWeb2 and FineWeb2-HQ”

• The Stack v2 (16 programming languages) (broad web data: may contain benchmark test sets) · split: unknown
  Evidence

  “Code: 12% - The Stack v2 (16 programming languages), StarCoder2 pull requests, Jupyter and Kaggle notebooks, GitHub issues, and StackExchange.”

• StarCoder2 pull requests, Jupyter and Kaggle notebooks, GitHub issues (broad web data: may contain benchmark test sets) · split: unknown
  Evidence

  “Code: 12% - The Stack v2 (16 programming languages), StarCoder2 pull requests, Jupyter and Kaggle notebooks, GitHub issues, and StackExchange.”

• StackExchange (not yet matched to a dataset) · split: unknown
  Evidence

  “Code: 12% - The Stack v2 (16 programming languages), StarCoder2 pull requests, Jupyter and Kaggle notebooks, GitHub issues, and StackExchange.”

• FineMath3+ and InfiWebMath3+ (broad web data: may contain benchmark test sets) · split: unknown
  Evidence

  “Math: 3% - FineMath3+ and InfiWebMath3+”

• Stack-Edu (broad web data: may contain benchmark test sets) · split: unknown
  Evidence

  “Code: 15% - Adding Stack-Edu”

• FineMath4+, InfiWebMath4+ and MegaMath (Qwen Q&A, Pro synthetic rewrites, text-code interleaved blocks) (broad web data: may contain benchmark test sets) · split: unknown
  Evidence

  “Math: 10% - Introducing FineMath4+, InfiWebMath4+, and MegaMath (including Qwen Q&A, Pro synthetic rewrites, and text-code interleaved blocks)”

• OpenMathReasoning (and other instruction/reasoning datasets in decay) (not yet matched to a dataset) · split: unknown

  Stage 3 config also lists openmathinstruct-2, open-codereasoning-4k, natural_reasoning.

  Evidence

  “Math: 13% - upsampling math data and introducing instruction and reasoning datasets such as OpenMathReasoning”

• cosmopedia2 (not yet matched to a dataset) · split: unknown

  Cosmopedia v2 (smollm-corpus component).

  Evidence

  “- /scratch/smollm3-data-part1/cosmopedia2”

• dolmino_math_synth_gsm_gsm8k (not yet matched to a dataset) · split: unknown

  Name indicates GSM8K-derived synthetic math from Dolmino; the config does not state which GSM8K split it derives from (see open_questions).

  Evidence

  “- /scratch/smollm3-data-part1/dolmino_math_synth_gsm_gsm8k”

• tiny-gsm-mind (problem-solving, 2students) (not yet matched to a dataset) · split: unknown

  TinyGSM-based synthetic math conversations; relation to GSM8K items not stated.

  Evidence

  “- /scratch/smollm3-data-part1/tiny-gsm-mind-problem-solving”

• openmathinstruct-2 (not yet matched to a dataset) · split: unknown
  Evidence

  “- /scratch/smollm3-data-part1/openmathinstruct-2”

• pes2o (not yet matched to a dataset) · split: unknown
  Evidence

  “- /scratch/smollm3-data-part1/pes2o”

• wiki (not yet matched to a dataset) · split: unknown
  Evidence

  “- /scratch/smollm3-data-part1/wiki”

• DCLM (broad web data: may contain benchmark test sets) · split: unknown

  Blog says 'DCLM'; that it is DCLM-Baseline specifically is not stated.

  Evidence

  “Web: 85% (12% multilingual) - FineWeb-Edu, DCLM, FineWeb2 and FineWeb2-HQ”

Benchmark cells

Classic LLM benchmarks

Recent LLM benchmarks

Notes

11.2T tokens in three stages + 100B-token context extension; post-training: 140B-token reasoning mid-training, SFT (1.8B tokens, 4 epochs) and APO (SmolTalk2). The cutoff comes from the default system metadata in the chat template. Decontamination statements not shown as cells (they name no benchmark tracked here, or none at all): evaluation benchmarks (unspecified) - SmolTalk2 Mid/SFT/Preference.

Sources

• SmolLM3: smol, multilingual, long-context reasoner (HF blog, raw markdown) · website · retrieved 2026-10-15
• SmolLM3-3B-Base · model card · retrieved 2026-10-15
• SmolLM3-3B · model card · retrieved 2026-10-15
• SmolLM3-3B chat template · code · retrieved 2026-10-15
• SmolLM3 training configs · dataset card · retrieved 2026-10-15
• stage1_8T.yaml · code · retrieved 2026-10-15
• stage3_9T_11T.yaml · code · retrieved 2026-10-15
• SmolTalk2 · dataset card · retrieved 2026-10-15