Time series / Benchmark

Time-series anomaly detection

Time-series anomaly detection datasets and benchmark suites (TSB-AD-U/M, UCR anomaly archive, SMD, MSL, SMAP, PSM, SWaT, Yahoo S5, NAB); grouping used on this site, not an official suite.

Test split: Protocols differ by dataset. SMD, MSL, SMAP and PSM come with an official unlabelled train part (mostly normal) and a labelled test part: SMD splits each machine's series into two equal halves (OmniAnomaly README); MSL/SMAP give pre-split train/test per channel with labels on the test part (Telemanom README); PSM ships train.csv, test.csv and test_label.csv (RANSynCoders repo). SWaT.A1 has 7 days of normal operation and 4 days with attacks; the iTrust page defines no other split. UCR anomaly archive: each series has a training prefix encoded in its file name and one anomaly after it (Wu & Keogh). NAB: no split; the first 15% of each file is an unscored probationary period (Lavin & Ahmad). Yahoo S5: no split. TSB-AD: series-level Eval/Tuning partition (15% of each dataset's series for hyperparameter tuning; Eval of TSB-AD-U is re-sampled to balance UCR, YAHOO and WSD). Within each TSB-AD series the file name gives a training prefix ('tr_N'): semi-supervised detectors are trained on data[:N] and scored with labels over the whole series (Run_Detector_U.py). For contamination, the whole series (including test anomalies) of all these datasets is public.

Contamination matrix

Dataset Chronos (T5)Chronos-2Chronos-2-SynthChronos-BoltDADA (General Time Series Anomaly Detector with Adaptive Bottlenecks and Dual Adversarial Decoders)Granite-TimeSeries-TTM-R1Granite-TimeSeries-TTM-R2Granite-TimeSeries-TTM-R2.1Granite-TTM-R3Lag-LlamaMantis (Mantis-8M)Mantis+ (MantisPlus)MantisV2Moirai-1.0-RMoirai-1.1-RMoirai-2.0-RMoirai-MoE-1.0-RMOMENT-1NuTimeSundial (Timer 3.0)Time-MoETime-RCD (TimeRCD)TimerTimer-S1Timer-XL (timer-base-84m)TimeRadarTimesFM 1.0TimesFM 2.0TimesFM 2.5TimesFM 3.0Tiny Time Mixer (TTM) Research-Use R2TiRexTiRex 1.1 (gifteval)TiRex-2TiRex-2-fevbench (TiRex-2-f)TiRex-2-gifteval-pretrain (TiRex-2-gp)TiRex-2-gifteval-zs (TiRex-2-g)Toto (2024 technical report)Toto 2.0Toto-2.0-2.5B-FTToto-Open-Base-1.0TSPulse r1TTM-R3UniTSUniTS few-shot anomaly detection checkpoints
TSB-AD-U (TSB-AD univariate) No No No ? Test No No No ? No ? No No No ? ? No No ? ? No No ? ? ? ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No
TSB-AD-M (TSB-AD multivariate) No No No ? ? No No No ? No ? No No No ? ? No No ? ? No No ? ? ? ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No
UCR Time Series Anomaly Archive (Hexagon ML/UCR, KDD Cup 2021) No No No ? No No No ? No ? No No No ? ? No Train No ? ? No No ? ? No ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No No
Server Machine Dataset (SMD) No No No ? No No No ? No ? No No No ? ? No Test No ? ? No No ? ? No ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No Train
Mars Science Laboratory (MSL) telemetry anomaly dataset No No No ? No No No ? No ? No No No ? ? No Train No ? ? No No ? ? No ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No Train
Soil Moisture Active Passive (SMAP) telemetry anomaly dataset No No No ? No No No ? No ? No No No ? ? No Train No ? ? No No ? ? No ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No Train
Pooled Server Metrics (PSM) No No No ? No No No No ? No ? No No No ? ? No ? No ? ? No No ? ? No ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No Train
Secure Water Treatment (SWaT) No No No ? No No No ? No ? No No No ? ? No ? No ? ? No No ? ? No ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No Train
Yahoo S5 Labeled Anomaly Detection Dataset (Yahoo Webscope S5) No No No ? Test No No No ? No ? No No No ? ? No Train No ? ? No No ? ? ? ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No No
Numenta Anomaly Benchmark (NAB) No No No ? No No No ? No ? No No No ? ? No Train No ? ? No No ? ? ? ? ? ? ? No No ? ? No ? No ? ? ? ? No ? No No

Hover a cell for the reason; click it for the evidence.

Notes

Grouping used on this site. Wu & Keogh (2021) argue the Yahoo, Numenta, NASA (MSL/SMAP) and OMNI (SMD) datasets are flawed and should be abandoned; TSB-AD (2024) curates series from them.

Sources