External validation of machine learning models for estimation of mortality 1, 3, 6, and 12 months after hip fracture on 5,055 consecutive patients
DOI:
https://doi.org/10.2340/17453674.2026.45871Keywords:
Hip, OsteoporosisAbstract
Background and purpose: Many models for prediction of mortality after hip fracture have been published but few have undergone external validation, usually considered a prerequisite for assessing the actual precision before being put to clinical use. The aim of our study was to externally validate our previously published models in an independent Swedish cohort.
Methods: We retrospectively analyzed 5,055 consecutive patients with hip fractures from 2 hospitals in Stockholm, Sweden, between 2010 and 2020. Previously developed Random Forest (RF), eXtreme Gradient Boosting (XGB), and Generalized Linear Models (GLM) (Mosfeldt et al. 2024) were deployed to estimate mortality at 1, 3, 6, and 12 months. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), calibration metrics, and decision curve analysis (DCA). Bootstrapped isotonic regression was used for recalibration.
Results: All models showed acceptable performance, with XGB performing best (AUC 0.72, 0.74, 0.75, and 0.77 for 1-, 3-, 6-, and 12-month mortality). Mortality was lower in the validation cohort, so the models were recalibrated to adjust for this difference.
Conclusion: External validation of the previously published models confirmed the original findings, with the XGB models again demonstrating the best overall performance. Recalibration addressed cohort differences in mortality rates and resulted in well-aligned predictions. The updated models for 3- and 12-month mortality are available online (https://hipfx.shinyapps.io/hipfxswe/), allowing clinicians to input patient data and receive individualized mortality predictions to support clinical decision-making.
Downloads
References
Lei M, Han Z, Wang S, Han T, Fang S, Lin F, et al. A machine learning-based prediction model for in-hospital mortality among critically ill patients with hip fracture: an internal and external validated study. Injury 2023; 54(2): 636-44. doi: 10.1016/j.injury.2022.11.031. DOI: https://doi.org/10.1016/j.injury.2022.11.031
Lex J R, Di Michele J, Koucheki R, Pincus D, Whyne C, Ravi B. Artificial intelligence for hip fracture detection and outcome prediction: a systematic review and meta-analysis. JAMA Network Open 2023; 6(3): e233391. doi: 10.1001/jamanetworkopen.2023.3391. DOI: https://doi.org/10.1001/jamanetworkopen.2023.3391
Liu F, Liu C, Tang X, Gong D, Zhu J, Zhang X. Predictive value of machine learning models in postoperative mortality of older adults patients with hip fracture: a systematic review and meta-analysis. Arch Gerontol Geriatr 2023; 115: 105120. doi: 10.1016/j.archger.2023.105120. DOI: https://doi.org/10.1016/j.archger.2023.105120
Mosfeldt M, Jørgensen H L, Lauritzen J B, Jansson K. Development and internal validation of a multivariable prediction model for mortality after hip fracture with machine learning techniques. Calcif Tissue Int 2024; 114(6): 568-82. doi: 10.1007/s00223-024-01208-1. DOI: https://doi.org/10.1007/s00223-024-01208-1
Collins G S, Reitsma J B, Altman D G, Moons K G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ 2015; 350: g7594. doi: 10.1136/bmj.g7594. DOI: https://doi.org/10.1136/bmj.g7594
Palm H, Krasheninnikoff M, Holck K, Lemser T, Foss N B, Jacobsen S, et al. A new algorithm for hip fracture surgery: reoperation rate reduced from 18% to 12% in 2,000 consecutive patients followed for 1 year. Acta Orthop 2012; 83(1): 26-30. doi: 10.3109/17453674.2011.652887. DOI: https://doi.org/10.3109/17453674.2011.652887
Parker M J, Palmer C R. A new mobility score for predicting mortality after hip fracture. J Bone Joint Surg Br 1993; 75(5): 797-8. doi: 10.1302/0301-620x.75b5.8376443. DOI: https://doi.org/10.1302/0301-620X.75B5.8376443
Stekhoven D J, Bühlmann P. MissForest: non-parametric missing value imputation for mixed-type data. Bioinformatics 2012; 28(1): 112-18. doi: 10.1093/bioinformatics/btr597. DOI: https://doi.org/10.1093/bioinformatics/btr597
Steyerberg E W, Vergouwe Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur Heart J 2014; 35(29): 1925-31. doi: 10.1093/eurheartj/ehu207. DOI: https://doi.org/10.1093/eurheartj/ehu207
Vickers A J, van Calster B, Steyerberg E W. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res 2019; 3: 18. doi: 10.1186/s41512-019-0064-7. DOI: https://doi.org/10.1186/s41512-019-0064-7
Van Calster B, McLernon D J, van Smeden M, Wynants L, Steyerberg E W, Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative. Calibration: the Achilles heel of predictive analytics. BMC Med 2019; 17(1): 230. doi: 10.1186/s12916-019-1466-7. DOI: https://doi.org/10.1186/s12916-019-1466-7
Steyerberg E W. Clinical prediction models: a practical approach to development, validation, and updating. 2nd ed. Cham, Switzerland: Springer; 2019. doi: 10.1007/978-3-030-16399-0. DOI: https://doi.org/10.1007/978-3-030-16399-0
Collett D. Modelling survival data in medical research. Boca Raton: CRC Press; 2015. doi: https://doi.org/10.1201/b18041. DOI: https://doi.org/10.1201/b18041
Haentjens P, Magaziner J, Colón-Emeric C S, Vanderschueren D, Milisen K, Velkeniers B, et al. Meta-analysis: excess mortality after hip fracture among older women and men. Ann Intern Med 2010; 152(6): 380-90. doi: 10.7326/0003-4819-152-6-201003160-00008. DOI: https://doi.org/10.7326/0003-4819-152-6-201003160-00008
Jonsson M H, Bentzer P, Turkiewicz A, Hommel A. Accuracy of the Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity score and the Nottingham risk score in hip fracture patients in Sweden: a prospective observational study. Acta Anaesthesiol Scand 2018; 62(8): 1057-63. doi: 10.1111/aas.13131. DOI: https://doi.org/10.1111/aas.13131
Schuijt H J, Smeeing D P J, Groenwold R H H, van der Velde D, Weaver M J. External validation of the U-HIP prediction model for in-hospital mortality in geriatric hip fracture patients. Injury 2022; 53(3): 1144-8. doi: 10.1016/j.injury.2021.12.028. DOI: https://doi.org/10.1016/j.injury.2021.12.028
Additional Files
Published
How to Cite
License
Copyright (c) 2026 Mathias Mosfeldt, Henrik L Jørgensen, Jes B Lauritzen, Karl-Åke Jansson
This work is licensed under a Creative Commons Attribution 4.0 International License.
PlumX (by Elsevier) is an altmetrics platform that tracks and visualizes the online attention, usage, captures, citations, and social media engagement.
