Hierarchy Generation and Exploitation for Enhanced Multi-Class Classification

Abstract

Background: Hierarchical Classification (HC) has long been recognized for improving predictive performance by exploiting relationships between classes. However, most tabular multi-class datasets lack predefined class hierarchies, limiting the broader applicability of hierarchy-aware learning methods.

Objectives: This study introduces HiGEC (Hierarchy Generation and Exploitation for Classification), a unified framework that enables hierarchy-aware learning for standard flat-label tabular classification problems through automatically generated class hierarchies. The objectives are to determine whether data-driven hierarchies can consistently improve performance over Flat Classification (FC), whether hierarchy-aware evaluation metrics improve under enhanced exploitation strategies, and whether probabilistic aggregation mitigates hierarchical error propagation.

Methods: HiGEC systematically integrates Hierarchy Generation (HG) and Hierarchy Exploitation (HE) within a unified framework. Two enhanced HE schemes are proposed: HE+, which mitigates error propagation through probabilistic path aggregation, and HE+F, which combines hierarchical and flat predictions through calibrated convex fusion. A large-scale benchmark involving 100 tabular datasets and ten classifiers—including gradient boosting, ensemble, generative, instancebased, and transformer-based methods—was conducted. Evaluation included both conventional classification metrics and hierarchy-aware metrics, together with statistical significance testing and runtime analysis.

Results: HiGEC consistently improved predictive performance over FC baselines, particularly in F1-score, with statistically significant gains across diverse classifiers. Hierarchy-aware evaluation further demonstrated improvements in structural consistency, with enhanced HE schemes achieving higher hierarchical F-measure (hF) and lower path-based loss than baseline hierarchical methods. Runtime analysis showed that optimized HE+F configurations typically incurred moderate computational overhead (approximately 2–3× slower than FC), while certain lightweight HE+ configurations achieved both higher predictive performance and lower runtime than their FC counterparts.

Conclusions: The empirical findings support the three formal hypotheses of this study: (1) HiGEC improves flat predictive performance over FC, (2) enhanced HE strategies improve hierarchy-aware evaluation metrics, and (3) probabilistic aggregation mitigates hierarchical error propagation. Factor-wise ranking analysis further demonstrated that HE strategies contribute more strongly to performance variation than HG methods under aggregated evaluation. Overall, HiGEC provides a unified and reproducible framework for hierarchy-aware learning on tabular multi-class benchmarks.