Time series / Benchmark
Time-series anomaly detection
Time-series anomaly detection datasets and benchmark suites (TSB-AD-U/M, UCR anomaly archive, SMD, MSL, SMAP, PSM, SWaT, Yahoo S5, NAB); grouping used on this site, not an official suite.
Test split: Protocols differ by dataset. SMD, MSL, SMAP and PSM come with an official unlabelled train part (mostly normal) and a labelled test part: SMD splits each machine's series into two equal halves (OmniAnomaly README); MSL/SMAP give pre-split train/test per channel with labels on the test part (Telemanom README); PSM ships train.csv, test.csv and test_label.csv (RANSynCoders repo). SWaT.A1 has 7 days of normal operation and 4 days with attacks; the iTrust page defines no other split. UCR anomaly archive: each series has a training prefix encoded in its file name and one anomaly after it (Wu & Keogh). NAB: no split; the first 15% of each file is an unscored probationary period (Lavin & Ahmad). Yahoo S5: no split. TSB-AD: series-level Eval/Tuning partition (15% of each dataset's series for hyperparameter tuning; Eval of TSB-AD-U is re-sampled to balance UCR, YAHOO and WSD). Within each TSB-AD series the file name gives a training prefix ('tr_N'): semi-supervised detectors are trained on data[:N] and scored with labels over the whole series (Run_Detector_U.py). For contamination, the whole series (including test anomalies) of all these datasets is public.
Contamination matrix
- TestTest data seen3
- SeenSeen, split unclear0
- TrainTraining data seen10
- Related data seen10
- NoNot seen210
- ?Unknown217
Hover a cell for the reason; click it for the evidence.
Notes
Grouping used on this site. Wu & Keogh (2021) argue the Yahoo, Numenta, NASA (MSL/SMAP) and OMNI (SMD) datasets are flawed and should be abandoned; TSB-AD (2024) curates series from them.
Sources
- The Elephant in the Room: Towards A Reliable Time-Series Anomaly Detection Benchmark (Liu & Paparrizos, NeurIPS 2024 Datasets and Benchmarks Track) · paper · retrieved 2026-10-15
- TSB-AD benchmark runner (Run_Detector_U.py) · code · retrieved 2026-10-15
- OmniAnomaly README (SMD split) · readme · retrieved 2026-10-15
- Telemanom README (MSL/SMAP split) · readme · retrieved 2026-10-15
- PSM data folder listing · code · retrieved 2026-10-15
- iTrust SWaT page · website · retrieved 2026-10-15
- Wu & Keogh, Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress (arXiv v4; IEEE TKDE) · paper · retrieved 2026-10-15
- NAB paper (probationary period) · paper · retrieved 2026-10-15