Multiverse-core leaderboard

23 estimators on 52 datasets · 7 metrics · ordered by average accuracy rank · built 2026-08-31

#	Estimator	Accuracy		Balanced accuracy		AUROC		F1		Log loss ↓		Sensitivity		Specificity
#	Estimator	Score	Rank	Score	Rank	Score	Rank	Score	Rank	Score	Rank	Score	Rank	Score	Rank
1	HC2	0.7909	7.64	0.7518	8.28	0.8990	5.94	0.7273	8.08	0.5383	6.08	0.7459	8.49	0.7943	7.23
2	MRHydra	0.7837	8.14	0.7564	7.85	0.8105	16.24	0.7316	7.70	7.7974	18.89	0.7642	8.04	0.7757	8.92
3	RDST	0.7734	9.02	0.7333	9.54	0.7912	16.81	0.6991	9.88	8.1667	19.14	0.7109	10.50	0.7874	8.48
4	RIST	0.7720	9.60	0.7397	10.40	0.8748	7.95	0.7147	10.15	0.6218	8.29	0.7408	10.51	0.7655	10.52
5	DrCIF	0.7747	9.86	0.7429	10.26	0.8813	8.31	0.7173	10.60	0.6484	9.15	0.7397	11.12	0.7708	10.58
6	FreshPRINCE	0.7743	9.87	0.7487	10.26	0.8745	7.88	0.7211	10.16	0.6007	6.15	0.7414	10.60	0.7770	10.70
7	CIF	0.7781	9.94	0.7471	10.17	0.8908	8.14	0.7212	10.21	0.6430	9.13	0.7441	11.03	0.7753	9.99
8	QUANT	0.7720	10.29	0.7462	10.24	0.8831	7.49	0.7189	10.31	0.6175	7.33	0.7521	10.40	0.7581	11.19
9	Arsenal	0.7680	10.41	0.7321	10.29	0.8457	12.77	0.7024	10.41	3.8631	16.03	0.7257	10.52	0.7732	10.27
10	ROCKET	0.7690	10.58	0.7326	10.35	0.7925	17.52	0.7019	10.83	8.3249	19.87	0.7200	11.40	0.7764	10.73
11	LITETime-MV	0.7506	10.92	0.7299	9.60	0.8513	9.82	0.6820	9.70	1.3206	11.65	0.7132	9.58	0.7637	10.88
12	STSF	0.7724	11.29	0.7477	10.94	0.8804	9.71	0.7080	11.52	0.6432	8.23	0.7345	11.82	0.7826	11.88
13	H-InceptionTime	0.7408	11.39	0.7190	10.60	0.8496	10.24	0.6838	10.29	1.3227	12.42	0.7223	10.16	0.7378	12.14
14	LiteTIME	0.7341	12.08	0.7104	11.26	0.8394	11.33	0.6680	11.55	1.4776	12.60	0.7113	10.39	0.7336	11.97
15	PatchMTSC	0.7428	12.77	0.6897	13.54	0.8261	12.45	0.6533	13.18	0.7655	9.19	0.6852	12.85	0.7352	12.61
16	ConvTran	0.7462	12.89	0.7102	13.06	0.8592	10.86	0.6767	12.61	0.8190	9.29	0.7159	12.49	0.7345	13.23
17	Catch22	0.7475	12.93	0.7181	13.38	0.8697	10.59	0.6922	13.42	0.7147	10.65	0.7240	13.27	0.7374	13.28
18	STC	0.7545	13.63	0.7172	13.82	0.8744	11.08	0.6940	13.66	0.6391	9.75	0.7185	13.85	0.7537	13.56
19	TSF	0.7515	13.63	0.7236	13.20	0.8740	11.42	0.6883	13.58	0.7252	10.37	0.7093	14.21	0.7606	13.44
20	TDE	0.7262	14.21	0.6813	14.38	0.8374	12.07	0.6382	13.88	0.8869	11.15	0.6714	13.46	0.7344	12.65
21	Summary	0.6858	16.12	0.6574	15.89	0.8268	14.88	0.6230	15.89	0.9123	12.96	0.6574	15.81	0.6844	16.29
22	1NN-DTW	0.6712	17.82	0.6454	17.06	0.7197	20.45	0.6136	17.07	11.8506	21.99	0.6521	15.80	0.6636	17.88
23	Dummy	0.3645	20.95	0.3029	21.64	0.5000	22.05	0.1507	21.32	1.4067	15.67	0.2855	19.70	0.3816	17.59

Average score and average rank over the 52 datasets with results for every estimator on every metric. Best in each column is highlighted. Metrics marked ↓ are better when lower.

Missing results

HC2 — AustraliaRainfall_disc (Time limit); STEW, Tiselac, USCActivity (cancelled before completion)
MRHydra — AustraliaRainfall_disc (OOM at 128GB); PenDigits (ValueError: n_timepoints must be >= 9, but found 8); Tiselac (LAPACK integer overflow in the RidgeClassifierCV SVD (aeon issue 3738))
RDST — AustraliaRainfall_disc, Tiselac (LAPACK integer overflow in the RidgeClassifierCV SVD (aeon issue 3738)); USCActivity (OOM at 64GB)
FreshPRINCE — FaceDetection, FordChallenge, Skoda, Tiselac (OOM at 128GB)
ROCKET — AustraliaRainfall_disc (LAPACK integer overflow in the RidgeClassifierCV SVD (aeon issue 3738))
STSF — AustraliaRainfall_disc, PenDigits (not recorded)
LiteTIME — BIDMC32HR_disc, BIDMC32SpO2_disc, USCActivity (not recorded)
PatchMTSC — EmoPain (ValueError: input collection has too little variation (std <= 1e-07)); PenDigits (ValueError: patch_len exceeds the number of timepoints)
ConvTran — Alzheimers, EigenWorms, PhotoStimulation (CUDA out of memory); EmoPain (ValueError: input collection has too little variation (std <= 1e-07))
TSF — AustraliaRainfall_disc (not recorded)
TDE — AustraliaRainfall_disc, Tiselac, USCActivity (Time limit); STEW (cancelled before completion)
Summary — AustraliaRainfall_disc (not recorded)
1NN-DTW — BIDMC32HR_disc (Time limit); BIDMC32SpO2_disc (not recorded)

Scoring uses the 52 datasets every estimator completed, so a dataset any one of them is missing is left out for all. Reasons are from the job logs of these runs.

Reproducing this page

from aeon.datasets.tsc_datasets import multiverse_core
from multiverse.experiments.tables import leaderboard

leaderboard(
    datasets=sorted(multiverse_core),
    estimators=["HC2", "MRHydra", "RDST", "RIST", "DrCIF", "FreshPRINCE", "CIF", "QUANT", "Arsenal", "ROCKET", "LITETime-MV", "STSF", "H-InceptionTime", "LiteTIME", "PatchMTSC", "ConvTran", "Catch22", "STC", "TSF", "TDE", "Summary", "1NN-DTW", "Dummy"],
    metrics=["accuracy", "balacc", "auroc", "f1", "logloss", "sensitivity", "specificity"],
    sort_by="accuracy",
)

Or python -m multiverse.experiments.tables to rebuild it with the defaults.