Evaluating machine learning models for neonatal mortality prediction in data-scarce settings: a case study of Ghana

Abstract Neonatal mortality is a persistent challenge in low-resource countries, including Ghana, where achieving the Sustainable Development Goal (SDG) 3.2.2 target by 2030 requires effective forecasting strategies. This study used five machine learning models, namely, Facebook Prophet, XGBoost, random forest, LightGBM, and support vector regression (SVR), to predict neonatal mortality via World Bank data from 1965 to 2022. Owing to the small sample size ( n  = 58), a simulation-based approach was employed to assess model performance across different data complexities and sample sizes. Facebook Prophet consistently outperforms other models in low-to-medium complexity scenarios, whereas XGBoost performs excellently in high-complexity environments. The best performance was achieved via a 70:20:10 data partitioning strategy. When the machine learning models were applied to historical data, Facebook Prophet (MAE = 0.2039, RMSE = 0.2290, AIC = 9.3382, AICc = 69.3382, BIC = 8.2970, and Theil’s U = 0.0101) achieved the lowest forecast errors and residual biases, with strong agreement between the predicted and actual values. Forecasts from 2023 to 2030 indicate a gradual decrease in neonatal mortality from 19.97 to 14.76 per 1000 live births. Although encouraging, this decline is not fast enough to meet the SDG target by 2030 without intensified health interventions. The findings highlight the usefulness of the Facebook Prophet in data-scarce settings and demonstrate the importance of simulation-based validation for reliable mortality forecasts in resource-limited environments. Health policymakers should adopt advanced machine learning forecasting models in neonatal mortality surveillance to improve prediction and resource allocation. This analysis relied on annual national-level World Bank estimates, which may mask subnational and facility-level variations and are subject to reporting and measurement uncertainty. In addition, the forecasting models used only historical mortality trends and did not explicitly incorporate causal drivers (e.g., health system shocks, policy changes, or socioeconomic covariates), so projections should be interpreted as trend-based rather than causal predictions. Graphical Abstract