Robustness Evaluation of Machine Learning and Deep Learning Models for Patient Adherence Classification under Feature-Homogeneous Clinical Data

Abstract

Model performance in clinical classification is strongly influenced by dataset characteristics, particularly feature homogeneity and interaction complexity. This study evaluates Naive Bayes, Support Vector Machine, Random Forest, XGBoost, CatBoost, LightGBM, Extreme Learning Machine, and a tree-based ensemble on patient adherence and health-risk claim datasets. Experimental results show that on the patient adherence dataset, all models exhibit convergent performance, with accuracies ranging from 0.5816 to 0.6232, indicating limited separability and weak nonlinear structure. In contrast, the health-risk claim dataset presents a more complex feature space, where boosting-based models achieve superior performance: CatBoost 0.8333, LightGBM 0.8305, and XGBoost 0.8253, while Naive Bayes significantly degrades to 0.3356. These findings highlight the importance of feature interaction complexity in determining model effectiveness. Robustness is evaluated by measuring relative performance degradation across datasets under distributional shift, comparing performance on the simpler dataset with that on the more complex one. Boosting-based models demonstrate stable robustness between 30.28% and 31.98%, whereas Naive Bayes exhibits severe negative robustness 67.33%. The ensemble model achieves a lower robustness 24.73%, indicating limited benefit from combining highly correlated learners. These results demonstrate that boosting-based models are more resilient to dataset shifts because they can capture nonlinear feature interactions, whereas probabilistic models are highly sensitive to violations of independence assumptions. The main contribution of this study is a cross-dataset robustness evaluation that quantifies model sensitivity under structural variation. The findings further show that feature homogeneity and interaction complexity are more influential than algorithm selection in determining model performance and robustness.