Contamination Map Methodology

Vision / Benchmark

Industrial anomaly detection

Industrial anomaly detection datasets; grouping used on this site, not an official suite.

Test split: Standard (unsupervised / 1-class) protocol: models are trained on the defect-free train split of each category and evaluated on the test split, which contains normal and anomalous images with pixel masks (MVTec AD; VisA 1-class: 90% of normal images train, 10% normal + all anomalous test; Real-IAD UIAD). MVTec AD 2 test ground truth is partly private (evaluation server). Zero-shot protocol: no training samples of the target dataset are used (WinCLIP uses CLIP 'without further tuning'); methods that learn prompts/adapters use auxiliary data, and the common cross-dataset protocol trains on the TEST split of another dataset - e.g. AnomalyCLIP fine-tunes on MVTec AD test data to evaluate on all other datasets, and on VisA test data to evaluate on MVTec AD. KolektorSDD2's official train split also contains defective images.

Contamination matrix

Dataset AA-CLIP, trained on MVTec AD (for VisA results)AA-CLIP, trained on VisAAdaCLIP, trained on all 14 datasets (demo model)AdaCLIP, trained on MVTec AD + ColonDBAdaCLIP, trained on VisA + ClinicDBAdaptCLIP, trained on MVTec ADAdaptCLIP, trained on VisAAnomaly-OneVision (Anomaly-OV)AnomalyCLIP, trained on MVTec ADAnomalyCLIP, trained on VisAAnomalyDINOAnomalyGPT, supervised (demo model)AnomalyGPT, unsupervised on MVTec ADAnomalyGPT, unsupervised on VisAAPRIL-GAN (VAND 2023), trained on MVTec ADAPRIL-GAN (VAND 2023), trained on VisADINOv2DINOv3 (web / LVD-1689M models)FiLo, trained on MVTec ADFiLo, trained on VisAInCTRL, trained on MVTec ADInCTRL, trained on VisAMetaUASMuScOpenAI CLIPOpenCLIP (LAION-2B checkpoints)OpenCLIP (LAION-400M checkpoints)SAM 2 / SAM 2.1Segment Any Anomaly (SAA / SAA+)Segment Anything Model (SAM)SigLIPSigLIP 2UniVADVCP-CLIP, trained on MVTec-ADVCP-CLIP, trained on VisAWideResNet-50-2 (supervised ImageNet-1k, torchvision)WinCLIP / WinCLIP+
MVTec AD Seen ? Test Test ? Test ? Seen Test ? ? Seen Train ? Test ? ? ? Test ? Test ? ? ? ? ? ? ? ? ? ? ? ? Test ? No ?
MVTec AD 2 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? No No ? ? ? No No ? ? No No No ? ? ? No No
MVTec LOCO AD ? ? ? ? ? ? ? ? ? ? ? Seen ? ? ? ? ? ? ? ? No No ? ? ? No No ? ? ? ? ? ? ? ? No No
VisA ? Seen Test ? Test ? Test Seen ? Test ? Seen ? Train ? Test ? ? ? Test No Test ? ? ? No No ? ? ? ? ? ? ? Test No No
Real-IAD ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? No No ? ? ? No No ? ? No No No ? ? ? No No
MPDD ? ? Test ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? No ?
BTAD ? ? Test ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? No ?
KolektorSDD2 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? No ?

Hover a cell for the reason; click it for the evidence.

Notes

Grouping used on this site, not an official suite. MVTec AD 2 vs MVTec AD: separate datasets from the same group; no image re-use stated.

Sources