A hybrid feature selection and ensemble learning model for heart disease diagnosis
Heart disease remains one of the leading causes of mortality worldwide, making early and accurate diagnosis essential for improving patient outcomes. This study proposes a hybrid feature selection and ensemble learning framework for heart disease diagnosis using a publicly available dataset containing 303 records and 13 clinical attributes. The preprocessing pipeline included missing-value imputation, one-hot encoding of categorical variables, and minimum–maximum normalization. Support vector machine–recursive feature elimination (SVM–RFE) was used to identify the most informative clinical features and remove redundant attributes. Multiple machine learning classifiers, including logistic regression, k-nearest neighbors, SVMs, decision trees, random forests, and extreme gradient boosting (XGBoost), were evaluated. Bayesian optimization was applied to tune model hyperparameters, and performance was assessed using an 80:20 train–test split together with stratified five-fold cross-validation. The proposed ensemble model achieved 93.62% accuracy, 94.08% precision, 93.11% recall, 93.58% F1-score, and 0.96 receiver operating characteristic area under the curve. The selected features included chest pain type, ST-slope, exercise-induced angina, maximum heart rate, and age, all of which are clinically meaningful predictors of cardiovascular risk. The results show that combining SVM-RFE, hyperparameter optimization, and ensemble learning improved predictive performance, generalization, and interpretability. The proposed framework can support early heart disease screening and clinical decision-making.

- Dwivedi G, Vyas R, Tripathi A, et al. MRI-Driven Deep Learning in Parkinson’s Disease Diagnosis: Innovations, Challenges, and Future Directions. Proc Natl Acad Sci India Sect A Phys Sci. 2026. doi: 10.1007/s40010-026-01003-7
- Manjulatha B, Pabboju S. An ensemble model for predicting chronic diseases using machine learning algorithms. In: Satapathy SC, Bhateja V, Favorskaya MN, Adilakshmi T, eds. Smart Computing Techniques and Applications. Smart Innovation, Systems and Technologies. Singapore: Springer; 2021: vol 224. doi: 10.1007/978-981-16-1502-3_34
- Al-Jamimi HA. Synergistic feature engineering and ensemble learning for early chronic disease prediction. IEEE Access. 2024;12:62215-62233. doi: 10.1109/access.2024.3395512
- Burse K, Kirar VPS, Burse A, Burse R. Various preprocessing methods for neural network-based heart disease prediction. In: Tiwari S, Trivedi M, Mishra K, Misra A, Kumar K, eds. Smart Innovations in Communication and Computational Sciences. Advances in Intelligent Systems and Computing. Singapore: Springer; 2019: vol 851. doi: 10.1007/978-981-13-2414-7_6
- Zhou H, Zhang PY, Zou X, Liu J, Wang WJ. Chronic disease diagnosis model based on convolutional neural network and ensemble learning method. Digital Health. 2023;9:20552076231198643. doi: 10.1177/20552076231198643
- Listiana E, Muzayanah R, Muslim MA, Sugiharti E. Optimization of support vector machine using information gain and AdaBoost to improve accuracy of chronic kidney disease diagnosis. J Soft Comput Explor. 2023;4(3):152–158. doi: 10.52465/joscex.v4i3.218
- Al-Jamimi HA, BinMakhashen GM, Saleh TA. Artificial intelligence approach for modeling petroleum refinery catalytic desulfurization process. Neural Comput Appl. 2022;34(20):17809-17820. doi: 10.1007/s00521-022-07423-x
- Chen H. Enterprise marketing strategy using big data mining technology combined with XGBoost model in the new economic era. PLoS ONE. 2023;18(6):e0285506. doi: 10.1371/journal.pone.0285506
- Hosmer DW, Lemeshow S, Sturdivant RX. Applied Logistic Regression. Wiley Series in Probability and Statistics. New York, NY, USA: John Wiley & Sons, Inc; 2013. doi: 10.1002/9781118548387
- Bergstra J, Bengio Y. Random search for hyper-parameter optimization. J Mach Learn Res. 2012;13(1):281–305.
- Wu J, Chen XY, Zhang H, Xiong LD, Lei H, Deng S. Hyperparameter optimization for machine learning models based on Bayesian optimization. J Electron Sci Technol. 2019;17(1):26-40. doi: 10.11989/JEST.1674-862X.80904120
- Jongbo OA, Adetunmbi AO, Ogunrinde RB, Badeji-Ajisafe B. Development of an ensemble approach to chronic kidney disease diagnosis. Sci Afr. 2020;8:e00456. doi: 10.1016/j.sciaf.2020.e00456
- Hannan SA, Pal P. Detection and classification of kidney disease using convolutional neural networks. J Neurol Neurorehab Res. 2023;8(2):136. doi: 10.35841/aajnnr-8.2.136
- Chicco D, Lovejoy CA, Oneto L. A machine learning analysis of health records of patients with chronic kidney disease at risk of cardiovascular disease. IEEE Access. 2021;9:165132- 165144. doi: 10.1109/access.2021.3133700
- Moreno-Sánchez PA. Data-Driven Early Diagnosis of Chronic Kidney Disease: Development and evaluation of an explainable AI model. IEEE Access. 2023;11:38359-38369. doi: 10.1109/access.2023.3264270
- Chen T, Guestrin C. XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM. 2016:785-794. doi: 10.1145/2939672.2939785
- Betrò B. Bayesian Methods in Global Optimization. In: Gritzmann P, Hettich R, Horst R, Sachs E, eds. Operations Research ’91. Heidelberg, Germany: Physica-Verlag HD; 1992:16-18. doi: 10.1007/978-3-642-48417-9_6
- Jain V, Agrawal M. Heart Failure Prediction Using XGB Classifier, Logistic Regression and Support Vector Classifier. In: Proceedings of the 2023 International Conference on Advancement in Computation & Computer Technologies (InCACCT). New York, NY, USA: IEEE; 2023:1-5. doi: 10.1109/incacct57535.2023.10141752
- Kong Y, Wang Y, Sun S, Wang J. XGB and SHAP credit scoring model based on Bayesian optimization. J Comput Electron Inf Manag. 2023;10(1):46-53. doi: 10.54097/jceim.v10i1.5368
- Basha N, Ibrahim G, Choudhury HA, et al. Bayesian-optimized Neural Networks and their application to model gas-to-liquid plants. Gas Sci Eng. 2023;113:204964. doi: 10.1016/j.jgsce.2023.204964
- Sharmila SL, Dharuman C, Venkatesan P. Disease classification using machine learning algorithms-a comparative study. Int J Pure Appl Math. 2017;114(6):1–10.
- Ruiz VM, Goldsmith MP, Shi L, et al. Early prediction of clinical deterioration using data-driven machine-learning modeling of electronic health records. J Thorac Cardiovasc Surg. 2022;164(1):211-222.e3. doi: 10.1016/j.jtcvs.2021.10.060
- Liu T, Krentz A, Lu L, Wang Y, Curcin V. Benchmarking survival machine learning models for 10-year cardiovascular disease risk prediction using large-scale electronic health records. Digital Health. 2026;12. doi: 10.1177/20552076251408534
- Zaman MAU, Adepoju OG. A Solution Approach with Ensemble-Based Learning Technology for Predicting Early Readmission Among Patients with Heart Failure (HF) Diagnosis Using Electronic Health Records (EHR). Medinformatics. 2025. doi: 10.47852/bonviewmedin52025119
- S NV, T M, Devi SP. Integrating Machine Learning and Deep Learning Approaches for Accurate Cardiovascular Disease Prediction from Electronic Health Records. In:Proceedings of the 2025 International Conference on Multi-Agent Systems for Collaborative Intelligence (ICMSCI). India; Erode; 2025:1090-1096. doi: 10.1109/icmsci62561.2025.10894568
- Sowmiya R, Devi BR, Vasuki R, et al. Combining Deep Learning and Machine Learning Techniques to Predict Cardiovascular Disease Accurately from Electronic Health Records. In: Proceedings of the 2025 3rd International Conference on Sustainable Computing and Data Communication Systems (ICSCDS). New York, NY, USA: IEEE; 2025:466-470. doi: 10.1109/icscds65426.2025.11166930
- Pathan MS, Nag A, Pathan MM, Dev S. Analyzing the impact of feature selection on the accuracy of heart disease prediction. Healthc Anal. 2022;2:100060. doi: 10.1016/j.health.2022.100060
- Sarra RR, Dinar AM, Mohammed MA, Abdulkareem KH. Enhanced Heart Disease Prediction Based on Machine Learning and χ2 Statistical Optimal Feature Selection Model. Designs. 2022;6(5):87. doi: 10.3390/designs6050087
- Noroozi Z, Orooji A, Erfannia L. Analyzing the impact of feature selection methods on machine learning algorithms for heart disease prediction. Sci Rep. 2023;13(1). doi: 10.1038/s41598-023-49962-w
- Dissanayake K, Md Johar MG. Comparative Study on Heart Disease Prediction Using Feature Selection Techniques on Classification Algorithms. Appl Comput Intell Soft Comput. 2021;2021:1-17. doi: 10.1155/2021/5581806
- Manikandan G, Pragadeesh B, Manojkumar V, Karthikeyan AL, Manikandan R, Gandomi AH. Classification models combined with Boruta feature selection for heart disease prediction. Inform Med Unlocked 2024;44:101442. doi: 10.1016/j.imu.2023.101442
- Ara J, Bhuiyan H, Roza II, Nahin ASM. Importance of balanced datasets with feature selection and ensemble methods on heart disease classification using distinctive machine learning techniques: a comparative analysis. Sci Rep. 2026;16(1):11706. doi: 10.1038/s41598-026-47691-4
- Luo L, Wang Y, Mo DY. Identifying heart disease risk factors from electronic health records using an ensemble of deep learning method. IISE Trans Healthc Syst Eng. 2023;13(3):237-247. doi: 10.1080/24725579.2023.2205665
- Dwivedi G, Narang T. A Comprehensive Review of Graph- Based Anomaly Detection Algorithms for Complex Data Environments. RACSC. 2025;19. doi: 10.2174/0126662558383302250922190426
- Bebortta S, Tripathy SS, Basheer S, Chowdhary CL. FedEHR: A Federated Learning Approach towards the Prediction of Heart Diseases in IoT-Based Electronic Health Records. Diagnostics. 2023;13(20):3166. doi: 10.3390/diagnostics13203166
- Malwade S, Sanap AR, Gite YS, Pandit P, Gaikwad YJ, Mhatre A. Predicting Heart Diseases in IoT-Based Electronic Health Records: A Federated Learning Approach. J Electr Syst. 2024;20(2):472-484. doi: 10.52783/jes.1465
- Gupta PK, Dalei V. Heart Disease Prediction Through Health Record Analysis Using Ensemble Learning. In: Proceedings of the 2025 OITS International Conference on Information Technology (OCIT). India: Bhubaneswar; 2025:1111-1116. doi: 10.1109/ocit66168.2025.11400391
- Jrab D, Eleyan D, Eleyan A, Bejaoui T. Heart Disease Prediction Using Machine Learning Algorithms. In: Proceedings of the 2024 International Conference on Smart Applications, Communications and Networking (SmartNets). VA, USA: Harrisonburg; 2024:1-8. doi: 10.1109/smartnets61466.2024.10577725
- Sondos Jameel Mukhyber. Classification Of Heart Disease Using Feature Selection and Machine Learning Techniques. Phys Sci Life Sci Eng. 2025;2(3):9. doi: 10.47134/pslse.v2i3.386
- El-Sofany HF. Predicting Heart Diseases Using Machine Learning and Different Data Classification Techniques. IEEE Access. 2024;12:106146-106160. doi: 10.1109/access.2024.3437181
- Stevens C, Mahani A, Ray K, Vallejo-Vaz A, Sharabiani M. Ensemble machine learning for screening cardiovascular diseases in electronic health records. Atherosclerosis. 2023;379:S194. doi: 10.1016/j.atherosclerosis.2023.06.644
- Viswateja K, Supriya D, Jahnavi KB, Aruna VBKL. Enhanced Feature Selection Technique Using Machine Learning Algorithm for Accurate Heart Disease Classification. In: Proceedings of the 2025 International Conference on Computing Technologies (ICOCT). India: Bengaluru; 2025:1-6. doi: 10.1109/icoct64433.2025.11118525
- Aşkın L, Polat E, Hoşoğlu Y, Tanrıverdi O. The Application of Artificial Intelligence in the Field of Cardiovascular Diseases Focuses on Both Diagnostic and Therapeutic Aspects. Exp Appl Med Sci. 2024;5(2):22-35. doi: 10.46871/eams.1438927
