Contamination MapTime series / Benchmark
GIFT-Eval
General Time Series Forecasting Model Evaluation benchmark (Salesforce AI Research): 23 datasets (28 HF folders, M4 split by frequency), 144,000 series, 177 million points, 7 domains, 10 frequencies, 97 dataset/frequency/term configurations with short, medium (x10) and long (x15) prediction lengths; multivariate datasets are flattened for univariate models. Ships with a separate non-leaking pre-training corpus (GiftEvalPretrain).
Test split: Per dataset, the final portion of each series is the test split and the rest is training data published in the same HF dataset (Salesforce/GiftEval, the 'train/test' dataset). Paper: 'We structure the evaluation component of our benchmark by dedicating the final 10% of each dataset in train/test portion to testing, with the rest allocated for training.' and 'A non-overlapping rolling evaluation method is employed, setting a predetermined number of windows in the test split, each equal to the dataset's prediction length.' 'The final window of the training data serves as validation for tuning deep learning model hyperparameters.' (Section 3.2; https://arxiv.org/pdf/2410.10393v2). Code (src/gift_eval/data.py): 'TEST_SPLIT = 0.1', 'MAX_WINDOW = 20', 'w = math.ceil(TEST_SPLIT * self._min_series_length / self.prediction_length)', 'return min(max(1, w), MAX_WINDOW)': the test split is the last windows x prediction_length points, with windows capped at 20, so for long series it is less than 10% of the series; validation is the one window before it. README: train splits 'are carefully constructed using earlier horizons that do not overlap with the test set'.
Contamination matrix
- TestTest data seen12
- SeenSeen, split unclear212
- TrainTraining data seen109
- Related data seen19
- NoNot seen629
- ?Unknown279
Hover a cell for the reason; click it for the evidence.
Notes
Leaderboard: 'TestData Leakage is a binary indicator specifying whether any test data was present in the training set.' (README update log 2025-08-05); submitters must answer Yes 'If your training data includes any dataset that is part of our test corpus' but 'Models trained solely on our provided training split' do not count as leaking. Model types: 'pretrained' = 'The pretraining corpus may include GIFT-Eval train splits.'; 'zero-shot' = pretraining data 'has no common datasets with GiftEval data pool (both train and test)'; 'fine-tuned' = further training on GIFT-Eval train/validation data. As of repo commit 577a65cf (2026-10-09), 20 of 143 result folders self-report testdata_leakage = Yes (see open_questions; self-reported, not facts about the models): Chronos_base, Chronos_large, Chronos_small, chronos_bolt_base, chronos_bolt_small, TimesFM, timesfm_2_0_500m, Lag-Llama, TTM-R1-Pretrained, TTM-R2-Pretrained, TTM-R2-Finetuned, MoiraiAgent-leaking, Super-Linear, Tabby-Pretrain, Tabby, TEMPO_ENSEMBLE, TSOrchestra-test, TimeTron-v2-33M, Fracast-0, RAES-Conductance-Ensemble. GIFT-Eval Table 23 lists the GIFT-Eval test datasets shared with LOTSA: hierarchical_sales, loop_seattle, m_dense, restaurant, sz_taxi. Salesforce/GiftEvalParquet (2026) re-publishes the test windows (history + future) per config; Salesforce/GiftEvalCTX (2026) is a derived contextual benchmark on 20 GIFT-Eval datasets. Version used: arXiv 2410.10393v2 (2024-11-11); HF dataset Salesforce/GiftEval revision 30841734ac5c (last modified 2025-01-21)
Sources
- GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation (arXiv v2) · paper · retrieved 2026-10-15
- Salesforce/GiftEval dataset card · dataset card · retrieved 2026-10-15
- GIFT-Eval GitHub README · readme · retrieved 2026-10-15
- gift_eval/data.py · code · retrieved 2026-10-15
- GIFT-Eval dataset_configurations.csv (97 dataset/frequency/term configurations) · code · retrieved 2026-10-15
- HF API metadata for Salesforce/GiftEval · website · retrieved 2026-10-15
- Salesforce/GiftEvalParquet dataset card · dataset card · retrieved 2026-10-15
- GIFT-Eval leaderboard · leaderboard · retrieved 2026-10-15