Can a Bayesian belief network for survival prediction in patients with extremity metastases (PATHFx) be externally validated in an Asian cohort of 356 surgically treated patients?

Hsiang-Chieh Hsieh; Yi-Hsiang Lai; Chia-Che Lee; Hung-Kuan Yen; Ting-En Tseng; Jiun-Jen Yang; Shin-Yiing Ling; Ming-Hsiao Hu; Chun-Han Hou; Rong-Sen Yang; Rikard Wedin; Jonathan A Forsberg; Wei-Hsin Lin

doi:10.2340/17453674.2022.4545

Authors

Hsiang-Chieh Hsieh Department of Orthopaedic Surgery, National Taiwan University Hospital, Hsin-Chu branch, Hsin-Chu City, Taiwan https://orcid.org/0000-0003-2525-9417
Yi-Hsiang Lai Department of Medical Education, National Taiwan University Hospital, Taipei City, Taiwan https://orcid.org/0000-0003-0751-4648
Chia-Che Lee Department of Orthopaedic Surgery, National Taiwan University Hospital, Taipei City, Taiwan
Hung-Kuan Yen Department of Orthopaedic Surgery, National Taiwan University Hospital, Hsin-Chu branch, Hsin-Chu City, Taiwan; Department of Medical Education, National Taiwan University Hospital, Hsin-Chu branch, Hsin-Chu City, Taiwan https://orcid.org/0000-0003-4381-2220
Ting-En Tseng Department of Medical Education, National Taiwan University Hospital, Taipei City, Taiwan; Department of Orthopaedic Surgery, National Taiwan University Hospital, Taipei City, Taiwan
Jiun-Jen Yang Department of Medical Education, National Taiwan University Hospital, Taipei City, Taiwan
Shin-Yiing Ling Department of Orthopaedic Surgery, National Taiwan University Hospital, Taipei City, Taiwan
Ming-Hsiao Hu Department of Orthopaedic Surgery, National Taiwan University Hospital, Taipei City, Taiwan https://orcid.org/0000-0001-8920-2052
Chun-Han Hou Department of Orthopaedic Surgery, National Taiwan University Hospital, Taipei City, Taiwan
Rong-Sen Yang Department of Orthopaedic Surgery, National Taiwan University Hospital, Taipei City, Taiwan
Rikard Wedin Department of Trauma and Reparative Medicine, Karolinska University Hospital, and Department of Molecular Medicine and Surgery, Karolinska Institutet, Stockholm, Sweden https://orcid.org/0000-0002-2440-1218
Jonathan A Forsberg Department of Orthopaedic Surgery, Johns Hopkins University, Baltimore, MD, USA
Wei-Hsin Lin Department of Orthopaedic Surgery, National Taiwan University Hospital, Taipei City, Taiwan

DOI:

https://doi.org/10.2340/17453674.2022.4545

Keywords:

Asian, External validation, Machine learning model, Oncology, PATHFx

Abstract

Background and purpose: Predicted survival may influence the treatment decision for patients with skeletal extremity metastasis, and PATHFx was designed to predict the likelihood of a patient dying in the next 24 months. However, the performance of prediction models could have ethnogeographical variations. We asked if PATHFx generalized well to our Taiwanese cohort consisting of 356 surgically treated patients with extremity metastasis.
Patients and methods: We included 356 patients who underwent surgery for skeletal extremity metastasis in a tertiary center in Taiwan between 2014 and 2019 to validate PATHFx’s survival predictions at 6 different time points. Model performance was assessed by concordance index (c-index), calibration analysis, decision curve analysis (DCA), Brier score, and model consistency (MC).
Results: The c-indexes for the 1-, 3-, 6-, 12-, 18-, and 24-month survival estimations were 0.71, 0.66, 0.65, 0.69, 0.68, and 0.67, respectively. The calibration analysis demonstrated positive calibration intercepts for survival predictions at all 6 timepoints, indicating PATHFx tended to underestimate the actual survival. The Brier scores for the 6 models were all less than their respective null model’s. DCA demonstrated that only the 6-, 12-, 18-, and 24-month predictions appeared useful for clinical decision-making across a wide range of threshold probabilities. The MC was < 0.9 when the 6- and 12-month models were compared with the 12-month and 18-month models, respectively.
Interpretation: In this Asian cohort, PATHFx’s performance was not as encouraging as those of prior validation studies. Clinicians should be cognizant of the potential decline in validity of any tools designed using data outside their particular patient population. Developers of survival prediction tools such as PATHFx might refine their algorithms using data from diverse, contemporary patients that is more reflective of the world’s population.