Cardiovascular disease (CVD) remains the leading global cause of mortality, underscoring the need for accurate and interpretable prediction models to facilitate early diagnosis. Existing machine learning (ML) approaches often face challenges balancing predictive performance with clinical interpretability, limiting their adoption. This study introduces a structured evaluation framework combining A/B testing with statistical hypothesis validation to rigorously compare ML models for CVD risk prediction. Utilizing a dataset of 1,001 patient records, models including logistic regression, random forest (RF), artificial neural networks, and extreme gradient boosting (XGBoost) were trained and evaluated. Synthetic Minority Oversampling Technique was applied to address class imbalance, while Shapley Additive Explanations (SHAP) provided insights into feature contributions and guided the development of reduced-feature models. Results indicate that RF achieved the highest accuracy (98.5%) and area under the receiver operating characteristic curve (0.9991), whereas XGBoost coupled with SHAP enabled effective feature selection with minimal loss in predictive power. A/B testing demonstrated the trade-offs between model complexity and interpretability, while statistical testing confirmed the significance of performance differences. These findings suggest that interpretable, reduced-feature models may be viable for deployment in resource-limited clinical settings, advancing the integration of artificial intelligence in cardiovascular healthcare.