Contamination MapText / LLM / Model
LLM360 Amber
- Organisation
- LLM360 (MBZUAI / Petuum)
- Family
- LLM360 Pebble · version 7B
- Released
- 2023-12-11 (arXiv v1 submission date of 2312.06550 (https://arxiv.org/abs/2312.06550); HF createdAt: IFM/Amber 2023-12-07, IFM/AmberChat 2023-11-30, IFM/AmberSafe 2023-12-15 (https://huggingface.co/api/models/IFM/Amber))
- Checkpoints
- IFM/Amber, IFM/AmberChat, IFM/AmberSafe
- Weights
- Open · apache-2.0
- Links
- PaperModel cardCode
Training data disclosure: Full list
Level determined for 0% of benchmark cells.
- TestTest data seen0
- SeenSeen, split unclear0
- TrainTraining data seen0
- Related data seen0
- NoNot seen0
- ?Unknown15
“360 model checkpoints and the full data sequence are available under the Apache 2.0 license.”
“We simply put together all the original aforementioned datasets (without any further cleaning, filtering, or sub-sampling), conduct a global permutation, and partition them evenly into 360 data chunks.”
“WizardLM/WizardLM_evol_instruct_V2_196k | 143k”
Training corpora
-
AmberDatasets (Amber-Data)
· 7 sources (complete list) · pre-training
· homepage
Evidence that LLM360 Amber uses it
“This dataset contains the fully prepared data sequence used to train Amber, an LLM360 model.”
huggingface.co · dataset card · retrieved 2026-10-15 Contents of this corpus
-
Arxiv (not yet matched to a dataset) · split: unknown
Arxiv, Book, StackExchange and Wikipedia come from RedPajama-v1 (paper Section 4.1.1).
Evidence
“Arxiv | 30.00”
huggingface.co · dataset card · DataMix table · retrieved 2026-10-15 “our pretraining data is a mixture of RefinedWeb, StarCoder, and RedPajama-v1.”
arxiv.org · paper · Section 4.1.1 · retrieved 2026-10-15 -
Book (not yet matched to a dataset) · split: unknown
Evidence
“Book | 28.86”
huggingface.co · dataset card · DataMix table · retrieved 2026-10-15 -
C4 (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“C4 | 197.67”
huggingface.co · dataset card · DataMix table · retrieved 2026-10-15 “A slight difference with OpenLLaMA-v2 is our inclusion of C4”
arxiv.org · paper · Section 4.1.1 · retrieved 2026-10-15 -
Refined-Web (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“Refined-Web | 665.01”
huggingface.co · dataset card · DataMix table · retrieved 2026-10-15 -
StarCoder (broad web data: may contain benchmark test sets) · split: unknown
Evidence
“StarCoder | 291.92”
huggingface.co · dataset card · DataMix table · retrieved 2026-10-15 -
StackExchange (not yet matched to a dataset) · split: unknown
Evidence
“StackExchange | 21.75”
huggingface.co · dataset card · DataMix table · retrieved 2026-10-15 -
Wikipedia (not yet matched to a dataset) · split: unknown
Evidence
“Wikipedia | 23.90”
huggingface.co · dataset card · DataMix table · retrieved 2026-10-15
-
Arxiv (not yet matched to a dataset) · split: unknown
Other declared training data
-
WizardLM/WizardLM_evol_instruct_V2_196k (not yet matched to a dataset) · split: unknown · fine-tuning
AmberChat SFT data (143k rows).
Evidence
“WizardLM/WizardLM_evol_instruct_V2_196k | 143k”
huggingface.co · model card · AmberChat DataMix table · retrieved 2026-10-15 “A MBER C HAT is trained on the evolved instruct training data as used by WizardLM [36].”
arxiv.org · paper · Section 4.1.3 · retrieved 2026-10-15 -
icybee/share_gpt_90k_v1 (not yet matched to a dataset) · split: unknown · fine-tuning
AmberChat SFT data per model card; the paper instead says AmberSafe was trained on ShareGPT 90K (see open_questions).
Evidence
“icybee/share_gpt_90k_v1 | 90k”
huggingface.co · model card · AmberChat DataMix table · retrieved 2026-10-15 -
ShareGPT 90K (not yet matched to a dataset) · split: unknown · fine-tuning
AmberSafe per paper; probably the same data as icybee/share_gpt_90k_v1.
Evidence
“A MBER S AFE is trained on ShareGPT 90K6 , and further optimized on the SafeRLHF”
arxiv.org · paper · Section 4.1.3 · retrieved 2026-10-15 -
PKU-Alignment/PKU-SafeRLHF (not yet matched to a dataset) · split: unknown · fine-tuning
AmberSafe DPO preference data, filtered to pairs with one safe and one unsafe response.
Evidence
“PKU-Alignment/PKU-SafeRLHF](https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF) | 330k”
huggingface.co · model card · AmberSafe DataMix table · retrieved 2026-10-15
Benchmark cells
Classic LLM benchmarks
Unknown MMLU (Measuring Massive Multitask Language Understanding) · training data not disclosed in enough detail
Unknown GSM8K (Grade School Math 8K) · training data not disclosed in enough detail
Unknown MATH (Mathematics Aptitude Test of Heuristics) · training data not disclosed in enough detail
Unknown HumanEval · training data not disclosed in enough detail
Unknown ARC-Challenge (AI2 Reasoning Challenge, Challenge Set) · training data not disclosed in enough detail
Unknown HellaSwag · training data not disclosed in enough detail
Unknown TruthfulQA · training data not disclosed in enough detail
Unknown WinoGrande · training data not disclosed in enough detail
Recent LLM benchmarks
Unknown GPQA Diamond · training data not disclosed in enough detail
Unknown MMLU-Pro · training data not disclosed in enough detail
Unknown LiveCodeBench (code generation, release_v6) · training data not disclosed in enough detail
Unknown SWE-bench Verified · training data not disclosed in enough detail
Unknown AIME 2024 · training data not disclosed in enough detail
Unknown AIME 2025 · training data not disclosed in enough detail
Unknown MATH-500 · training data not disclosed in enough detail
Notes
7B LLaMA-architecture model trained on 1.26T tokens; 360 intermediate checkpoints. AmberChat (SFT) and AmberSafe (DPO on AmberChat) were released within about two weeks of the base model (HF repos created 2023-11-30 to 2023-12-15), so their data is recorded here as fine_tuning. HF org LLM360 now redirects to IFM.
Sources
- LLM360: Towards Fully Transparent Open-Source LLMs (arXiv 2312.06550v1) · paper · retrieved 2026-10-15
- IFM/Amber model card · model card · retrieved 2026-10-15
- IFM/AmberChat model card · model card · retrieved 2026-10-15
- IFM/AmberSafe model card · model card · retrieved 2026-10-15
- IFM/AmberDatasets dataset card · dataset card · retrieved 2026-10-15