Contamination Map Methodology

Text / LLM / Model

DCLM-7B-IT (DCLM-IT-7B)

Organisation
Apple / DataComp-LM (DCLM) team
Family
DCLM
Released
2024-06-15 (HF repo creation of mlfoundations/dclm-7b-it (createdAt 2024-06-15, https://huggingface.co/api/models/mlfoundations/dclm-7b-it); public availability date not verified.)
Checkpoints
mlfoundations/dclm-7b-it
Weights
Open · Apple Sample Code License (apple-ascl)
Links
PaperModel cardCode

Training data disclosure: Partial list

Level determined for 26% of benchmark cells.

“DCLM-IT-7B is a 7 billion parameter language model trained on the DCLM-Baseline dataset and then further finetuned on our DCLM-IT finetuning mixture.”
huggingface.co · model card · Model Card for DCLM-7B-IT · retrieved 2026-10-15
“we curate a custom dataset, DCLM-IT, by combining some of the best instruction-tuning datasets including UltraFeedback [49], Tulu-v2 SFT [82], CodeFeedback [201], OH-2.5, Nectar [204], NoRobots [139], WildChat [199], WebInstruct [193], and StarCoder2-Self-OSS-Instruct [177].”
arxiv.org · paper · Appendix P · retrieved 2026-10-15

Training corpora

Other declared training data

Contamination analyses

Analyses that found benchmark test data in this model's training data, or in a corpus it uses.

Benchmark cells

Classic LLM benchmarks

Test data seen MMLU (Measuring Massive Multitask Language Understanding) · a contamination analysis found test examples in the pre-training data (DCLM's MMLU overlap rule (question last sentence + one answer option on the same page; recall-oriented, likely overestimate) flagged 0.007% of DCLM-Baseline documents (vs 0.001% Dolma-V1.7, 0.009% FineWeb-Edu). Removing the flagged overlaps did not lower MMLU accuracy at 7B-2x scale (51.8 -> 52.7).) (via DCLM-Baseline 1.0)

Evidence (3)
“DCLM-IT-7B is a 7 billion parameter language model trained on the DCLM-Baseline dataset and then further finetuned on our DCLM-IT finetuning mixture.”
huggingface.co · model card · Model Card for DCLM-7B-IT · retrieved 2026-10-15
“the percentage of documents in DCLM- BASELINE that are flagged by our MMLU-specific decontamination is roughly similar to that for other high-performing contemporary datasets”
arxiv.org · paper · Appendix O, Table 25 · retrieved 2026-10-15
“Note that because our flagging rule prioritizes recall over precision, these numbers are likely to be overestimates of the true contamination rates.”
arxiv.org · paper · Appendix O, Table 25 caption · retrieved 2026-10-15

Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail

Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail

Unknown HumanEval · training data not disclosed in enough detail

Test data seen ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · a contamination analysis found test examples in the pre-training data (32.6% of ARC-Challenge test questions are 'dirty' in DCLM-baseline (17x more than in Pile-train).) [third-party study] (via DCLM-Baseline 1.0)

Evidence (3)
“DCLM-IT-7B is a 7 billion parameter language model trained on the DCLM-Baseline dataset and then further finetuned on our DCLM-IT finetuning mixture.”
huggingface.co · model card · Model Card for DCLM-7B-IT · retrieved 2026-10-15
“When checking against DCLMbaseline, MMLU has 27.70% dirty entries, MMLUPro has 16.20%, ARC-Challenge has 32.6%, and ARC-Easy has 32.3%.”
arxiv.org third-party study· study · Section 4.2 · retrieved 2026-10-15
“For example, ARC-Challenge has 17× more and MMLU has 1.1× more dirty entries.”
arxiv.org third-party study· study · Section 4.2 · retrieved 2026-10-15

Test data seen HellaSwag · a contamination analysis found test examples in the pre-training data (Overlaps between HellaSwag test examples and DCLM-Baseline pages were detected and removed for an ablation (fraction not reported in text); removal did not lower HellaSwag accuracy (77.9 -> 78.4). In the 10-gram token-overlap analysis HellaSwag is the notable exception with a performance gap between full and 'clean' subsets (Figure 12).) (via DCLM-Baseline 1.0)

Evidence (3)
“DCLM-IT-7B is a 7 billion parameter language model trained on the DCLM-Baseline dataset and then further finetuned on our DCLM-IT finetuning mixture.”
huggingface.co · model card · Model Card for DCLM-7B-IT · retrieved 2026-10-15
“Specifically, we attempt to remove examples from these two test sets that exist in DCLM- BASELINE.”
arxiv.org · paper · Section 4.6 · retrieved 2026-10-15
“While Hellaswag appears to be a notable exception, we showed in Section 4.6 that removing contaminated Hellaswag examples from our training set does not decrease performance.”
arxiv.org · paper · Appendix O · retrieved 2026-10-15

Unknown TruthfulQA · training data not disclosed in enough detail

Unknown WinoGrande · training data not disclosed in enough detail

Recent LLM benchmarks

Unknown GPQA Diamond · training data not disclosed in enough detail

Test data seen MMLU-Pro · a contamination analysis found test examples in the pre-training data (16.20% of (downsampled) MMLU-Pro test questions are 'dirty' in DCLM-baseline.) [third-party study] (via DCLM-Baseline 1.0)

Evidence (2)
“DCLM-IT-7B is a 7 billion parameter language model trained on the DCLM-Baseline dataset and then further finetuned on our DCLM-IT finetuning mixture.”
huggingface.co · model card · Model Card for DCLM-7B-IT · retrieved 2026-10-15
“When checking against DCLMbaseline, MMLU has 27.70% dirty entries, MMLUPro has 16.20%, ARC-Challenge has 32.6%, and ARC-Easy has 32.3%.”
arxiv.org third-party study· study · Section 4.2 · retrieved 2026-10-15

Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail

Unknown SWE-bench Verified · training data not disclosed in enough detail

Unknown AIME 2024 · training data not disclosed in enough detail

Unknown AIME 2025 · training data not disclosed in enough detail

Unknown MATH-500 · training data not disclosed in enough detail

Notes

Instruction-tuned on DCLM-IT (~4M instances, ~8B tokens, 10 epochs). Pretraining data identical to dclm-7b (StarCoder and ProofPile2 entries and their third-party findings apply too). Paper says no alignment (PPO/DPO) was applied.

Sources