Contamination Map Methodology

Text / LLM / Model

LLM360 K2-65B (K2 Diamond)

Organisation
LLM360 (MBZUAI / Petuum)
Family
LLM360 K2 · version 65B
Released
2024-04-17 (HF createdAt of IFM/K2 (https://huggingface.co/api/models/IFM/K2); paper arXiv v1 2025-01-13)
Checkpoints
IFM/K2, IFM/K2-Chat
Weights
Open · apache-2.0
Links
PaperModel cardCode

Training data disclosure: Partial list

Level determined for 20% of benchmark cells.

“K2 is fully transparent, meaning we've open-sourced all artifacts, including code, data, model checkpoints, intermediate results, and more.”
huggingface.co · model card · retrieved 2026-10-15
“• Data: The exact sequence of training data, segmented into chunks corresponding to each checkpoint.”
arxiv.org · paper · Section 1, release artifacts · retrieved 2026-10-15
“Safety & Alignment Data | 99,976,621”
huggingface.co · model card · K2-Chat-060124, Datasets and Mix · retrieved 2026-10-15

Training corpora

Other declared training data

Contamination analyses

Analyses that found benchmark test data in this model's training data, or in a corpus it uses.

Benchmark cells

Classic LLM benchmarks

Test data seen MMLU (Measuring Massive Multitask Language Understanding) · a contamination analysis found test examples in the pre-training data (MMLU test questions are also substantially 'dirty' in Pile-train: DCLM-baseline (27.70%) has only 1.1x more dirty entries than Pile-train. Exact Pile-train rate is in Table 2 (not reliably extracted from the PDF text).) [third-party study] (via The Pile)

Evidence (4)
“pubmed-abstracts (from the Pile) | 4.77B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“uspto (from the Pile) | 4.77B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“pubmed-central (from the Pile) | 26B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“Compared with Pile-train, most benchmarks show a higher dirty rate on DCLM-baseline. For example, ARC-Challenge has 17× more and MMLU has 1.1× more dirty entries.”
arxiv.org third-party study· study · Section 4.2 · retrieved 2026-10-15

Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail

Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail

Test data seen HumanEval · a contamination analysis found test examples in the pre-training data (Riddell et al. (2024): 12.2% of HumanEval gold solutions have a perfect (surface- or semantic-level) match in the Pile's GitHub split; CONDA 2024 contamination report also lists openai_humaneval as found (>0%) in EleutherAI/pile. (WIMBD's exact-match input-field check reports 0.00% for openai-humaneval in the Pile, Table 16.)) [third-party study] (via The Pile)

Evidence (6)
“pubmed-abstracts (from the Pile) | 4.77B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“uspto (from the Pile) | 4.77B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“pubmed-central (from the Pile) | 26B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“Results in Fig. 1 show that 12.2% of the solutions in HumanEval have been seen by models trained on the P ILE”
arxiv.org third-party study· study · Section 4.1 · retrieved 2026-10-15
“We use the GitHub split of the training dataset, which has a raw size of 95.16 GiB.”
arxiv.org third-party study· study · Section 3.1 · retrieved 2026-10-15
“amazon_reviews_multi (Keung et al., 2020), mbpp, openai_humaneval (Chen et al., 2021), liar”
arxiv.org third-party study· study · Table 1 (row EleutherAI/pile) · retrieved 2026-10-15

Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail

Unknown HellaSwag · training data not disclosed in enough detail

Unknown TruthfulQA · training data not disclosed in enough detail

Unknown WinoGrande · training data not disclosed in enough detail

Recent LLM benchmarks

Unknown GPQA Diamond · training data not disclosed in enough detail

Related data seen MMLU-Pro · a contamination analysis found test examples of the related dataset MMLU (Measuring Massive Multitask Language Understanding) (MMLU test questions are also substantially 'dirty' in Pile-train: DCLM-baseline (27.70%) has only 1.1x more dirty entries than Pile-train. Exact Pile-train rate is in Table 2 (not reliably extracted from the PDF text).) [third-party study] (via The Pile)

Evidence (6)
“pubmed-abstracts (from the Pile) | 4.77B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“uspto (from the Pile) | 4.77B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“pubmed-central (from the Pile) | 26B”
huggingface.co · dataset card · Stage 1 table · retrieved 2026-10-15
“Compared with Pile-train, most benchmarks show a higher dirty rate on DCLM-baseline. For example, ARC-Challenge has 17× more and MMLU has 1.1× more dirty entries.”
arxiv.org third-party study· study · Section 4.2 · retrieved 2026-10-15
“**Original MMLU Questions:** Part of the dataset comes from the original MMLU dataset. We remove the trivial and ambiguous questions.”
huggingface.co · dataset card · 2. Dataset Summary · retrieved 2026-10-15
“Additionally, MMLU-Pro eliminates the trivial and noisy questions in MMLU.”
arxiv.org · paper · Abstract · retrieved 2026-10-15

Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail

Unknown SWE-bench Verified · training data not disclosed in enough detail

Unknown AIME 2024 · training data not disclosed in enough detail

Unknown AIME 2025 · training data not disclosed in enough detail

Unknown MATH-500 · training data not disclosed in enough detail

Notes

Pre-training data fully released (k2-data). Fine-tuning data recorded for two K2-Chat releases: K2-Chat-060124 (MathInstruct, OpenHermes-2.5, FLAN_3M, philosophy encyclopedia, TinyStories, safety data) and the 10/31/24 update (adds Infinity-Instruct, JiuZhang3.0-Corpus-SFT, glaive function calling). Disclosure partial_list because the in-house safety prompts and self-synthesized math data are described but not shown to be released.

Sources