Intra- and inter-observer agreement in distal radius fracture dislocation measurement of casting position
Katriina PAASIKALLIO 1, Reijo SUND 2, Simo MIETTINEN 1,2, Sonja KAURANEN 2, Hannu SORSA 2, Heikki KRÖGER 1,2, and Joonas SIROLA 1,2
1 Department of Orthopaedics, Traumatology, and Hand Surgery, Kuopio University Hospital, Kuopio; 2 University of Eastern Finland, Institute of Clinical Medicine, Faculty of Health Sciences, Kuopio, Finland
Abstract
Background and purpose — Most displaced distal radius fractures (DRF) are treated nonoperatively, with reduction and immobilization in a cast. Studies assessing intra- or inter-observer agreement on radiologic measurements of casting position have not been published, which was the aim of our study.
Patients and methods — Our study is based on the Kuopio Osteoporosis Risk Factor and Prevention (OSTPRE) study. All detected DRFs during the OSTPRE follow-up were retrieved and based on sample size calculations 50 fractures were randomly selected for the study. 5 independent reviewers measured dislocation parameters and wrist position in a cast from the radiographs. A linear mixed model was used to estimate the concordance correlation coefficient (CCC) and total deviance index (TDI) that were used to evaluate intra- and inter-observer agreement. We used Kappa values to determine intra- and inter-rater agreement on radiographically acceptable reduction of the DRF. Limits of acceptable position were those defined by Finnish Current Care Guidelines.
Results — For radial inclination, radial shortening, and dorsal/volar tilt, intra- and inter-observer correlations were high (CCC > 0.76). In addition, measurements of wrist angle in a splint had high correlations (CCC > 0.78), whereas measurement of intra-articular gap and step had poor correlations (CCC < 0.52). The Kappa value for overall agreement on the radiographically acceptable position of the DRF was modest (0.59).
Conclusion — Intra- and inter-observer repeatability of casting position of radial inclination, radial shortening, and dorsal/volar tilt were high whereas intra-articular gap and step had poor correlations.
Citation: Acta Orthopaedica 2023; 94: 379–386. DOI: https://doi.org/10.2340/17453674.2023.13707.
Copyright: © 2023 The Author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/), allowing third parties to copy and redistribute the material in any medium or format and to remix, transform, and build upon the material for non-commercial purposes, provided proper attribution to the original work.
Submitted: 2022-11-18. Accepted: 2023-05-01. Published: 2023-07-21.
Correspondence: katriina.paasikallio@kuh.fi
KP participated in the collection of patient data and interpretation of results and analyzed the data and drafted the manuscript. SM and JS participated in the planning and design of the study, in the collection of patient data and data interpretation, and critically revised the manuscript. RS participated in the planning and design of the study, analyzed the data, participated in the data interpretation, and critically revised the manuscript. HS and SK participated in the collection of patient data. HK critically revised the manuscript. All the authors have read and approved the final manuscript.
Handling co-editor: Taco Gosens
Acta thanks Hanne Dalsgaard and Gerald Kraan for help with peer review of this manuscript.
Distal radius fracture (DRF) is one of the most common fractures and in people aged over 65 years, it accounts for 18% of all fractures [1,2].
Based on Finnish Current Care Guidelines (FCCG), measurements used to assess DRF dislocation are radial inclination, radial shortening, dorsal/volar tilt, intra-articular step and gap [3].
It has been estimated that at least 80% of DRFs can be reduced nearly anatomically [4,5]. However, the association between radiographic measurements and functional outcome remains unclear [6]. There is some evidence that the operative treatment of fracture with post-reduction radial shortening > 3 mm, dorsal tilt > 10 degrees, or intra-articular gap or step-off > 2 mm leads to a better radiographic and functional outcome [7]. The ulnar variance has been found to be the most important radiographic parameter to be restored to obtain good functional outcome [8]. If the reduction is within these previously mentioned limits for each parameter, it is recommended to treat the fracture nonoperatively with cast immobilization for 5 weeks [3]. However, there is a lack of evidence for the optimal immobilization method (casting material, casting technique, wrist position, and duration of immobilization) [9]. To estimate the casting angle of the wrist in the splint, flexion/extension and deviation of the wrist can be measured from the PA and ML radiographs. Previous studies assessing intra- or inter-observer agreement on radiologic measurements of casting position have not been published [10-13].
The purpose of this study was to investigate the inter- and intra-observer agreement of the radiographic measurements of dislocation and casting position of DRF and investigate the reliability of these measurements in a clinical setting.
Patients and methods
The study was based on the Kuopio Osteoporosis Risk Factor and Prevention (OSTPRE) study ( https://www3.uef.fi/en/web/kmru/ostpre), which investigates the significance of lifestyle and health disorders for bone mineral density and the susceptibility to fall and fractures of peri- to post-menopausal women [14].
For the present study, all self- or register-reported wrist fractures between 2010 and 2019 (n = 737) were retrieved and validated from the medical records by study group physicians. The radiographs were searched with the personal identity code from the PACS system (Picture Archiving and Communication System, SECTRA Ab, Sweden) used in the hospital district of Northern Savo, Finland. Simple random sampling was used to select a total of 50 fractures from the study population. All randomly selected images were measured regardless of the image field quality or size of the image field.
The radiographs taken after first reduction and casting were selected to validate the fracture dislocation parameters and the casting angle. The chosen parameters for evaluating the dislocation of the fracture were based on the FCCG. 5 independent reviewers (KP, SM, SK, HS, and JS) measured the following dislocation parameters from the radiographs (Figure 1): (1) radial inclination; (2) radial shortening; (3) sagittal shift; (4) dorsal/volar tilt; (5) intra-articular gap; and (6) intra-articular step. In addition, the casting angle of the wrist in the splint was measured. All these parameters were measured in the PACS system. The accuracy of the PACS system is 0.1 mm. The casting angle of the wrist was defined as the angle between the radial axis and the axis of the 3rd metacarpal bone in the lateral (wrist flexion/extension) and PA view (ulnar/radial deviation) (Figure 1). The measurements were carried out twice by all 5 reviewers with at least a week between the measurements, and data for the first measurements were unavailable. 2 of the observers (SK and HS) were third-year medical students and prior to the current study were instructed to measure parameters of dislocation and casting angle by KP with randomly selected cases, not included in the present study sample or the OSTPRE cohort. 1 of the observers was a resident in orthopedics and traumatology (KP) and 2 were consultant orthopedic surgeons (SM and JS).
Figure 1. Dislocation and casting angle of the wrist in the splint measurements in PA and M/L radiographs.
A. Measurements of radial inclination (a) and radial shortening (b). The long axis of radius (1). A line perpendicular to the long axis of the radius is drawn at the level of the most distal aspect of the radial articular surface (2). The radial and ulnar margins of the distal radial articular surface are connected (3). A line perpendicular to the long axis of the radius is drawn at the level of the most distal aspect of the ulnar articular surface (4). Radial inclination (a) is the angle between lines 2 and 3. Radial shortening is the distance between lines 2 and 4.
B. Measurement of dorsal/volar tilt (a). Long axis of the radius (1). The anterior and posterior margins of the most distal aspect of the radial articular face are connected (2). A line perpendicular to the long axis of the radius is drawn at a convenient level (3). The angle between lines 2 and 3 is dorsal/volar tilt (negative tilt is dorsal tilt).
C. Measurement of the gap.
D. Measurement of the step.
E. Measurement of the sagittal shift.
F. and G. Measurements of the casting angle in PA (F) and M/L (G) radiographs. The long axis of the radius (1). The long axis of the third metacarpal bone (2). Wrist deviation (a) is the angle between lines 1 and 2 in the PA radiograph. Wrist flexion/extension (b) is the angle between lines 1 and 2 in the M/L radiograph.
Statistics
As measurement agreement is not about the testing of hypotheses but about estimation of parameters, adequate sample size was assessed in terms of expected standard errors by using the variances of parameters obtained from the supplemental data published in Watson et al. [11] with the simulation technique for sample size determination presented in Choudhary and Nagaraja [15]. Based on the simulations, a set of 50 images was considered to be a suitable compromise between the uncertainty and pragmatic possibilities. This is effectively more than the sample sizes used in previous studies (n = 30 to 37) for intra-observer analyses (10,11). Our final data consisted of 5 raters with 2 repeated measurements for 50 images. We used trellis plots to visualize the intra- and inter-observer agreement. We used a linear mixed model to take different dependencies adequately into account in order to the calculate concordance correlation coefficient (CCC) and total deviance index (TDI) for measures of agreement and repeatability (15). The CCC estimate and its 95% confidence interval (CI) lower bound indicate the degree of agreement. The CCC representing intraclass correlation and reliability may be viewed as a chance-corrected measure, reflecting the extent of agreement in excess of what is expected by chance alone. To assess the extent of agreement, we also estimated 90% TDI values. The upper bound of the CI for TDI determines the range on the scale of measured variable that captures 90% of differences in measurements with 95% confidence.
In addition to single continuous parameters, we were also interested in the combined agreement of parameters reflecting radiographically acceptable reduction of the DRF. Chosen limits for acceptable reduction and conservative treatment were those defined by FCCG (dorsal tilt ≤ 15° in relation to perpendicular position, radial shortening ≤ 3 mm with the assumption of neutral ulnar variance, intra-articular step ≤ 1 mm, intra-articular gap ≤ 1 mm, radial inclination ≥ 15°), fundin and a binary treatment choice variable, being 1 if all conditions are fulfilled and zero otherwise, was constructed. We used Cohen’s Kappa to estimate the chance-corrected intra- and inter-observer agreement. We considered agreement to be weak if Kappa was < 0.40, good for 0.40–0.75 and excellent for Kappa > 0.75.
Ethics, funding, and disclosures
The study was approved by the ethical review board of the Kuopio University Hospital on March 17, 2020 (permission 578/2020). All authors have no conflict of interest to report. This research received no specific grant from any funding agency in the public, commercial, or non-profit sectors. Completed disclosure forms for this article following the ICMJE template are available on the article page, DOI: 10.2340/17453674.2023.13707
Results
Characteristics of the study population and fractures are given in Table 1. The study population included only women. The mean age at the time of fracture was 76 (SD 3.8, range 68–86) years. The mean number of self-reported chronic diseases was 5.5 (SD 3.1, range 0–18). Numbers of diseases were collected from the questionnaire nearest the time of fracture. The most common diseases were hypertension and knee/hip osteoarthritis. The average value for radial shortening was 2.2 (SD 2.1, range 0.0–9.9) mm and for tilt 4.9 (SD 9.1, range –29.6 to 29.7) degrees. The mean values of casting angle were wrist flexion/extension 14.7 (SD 14.0, range –22.2 to 84.4) and wrist deviation 8.1 (SD 8.3, range –15.7 to 32.2) degrees.
Intra- and inter-observer CCCs for every measured dislocation parameter are shown in Table 2 (see Appendix). The average and the range of all 5 reviewers’ CCCs and TDIs are given in Table 3. Inter-observer correlation was high for radial inclination (range 0.74–0.91), radial shortening (0.78–0.87), and wrist flexion/extension (0.80–0.98). Sagittal shift (0.26– 0.48), intra-articular gap (0.10–0.35), and step (0.02–0.09) had poor agreement. In each parameter, intra-observer CCC values were higher than inter-observer CCCs. Intra-observer agreement of radial inclination (0.74–0.96), radial shortening (0.88–0.96), dorsal/volar tilt (0.78–0.96), wrist flexion/extension (0.84–1.00), and ulnar/radial deviation (0.62–0.99) was excellent. Intra-articular gap (0.34–0.72) and step (0.07–0.90) intra-observer CCCs had a wide range.
Inter- and intra-observer TDIs are given in Table 2. Interobserver TDIs had a wider range compared with intra-observer values.
The trellis plots to illustrate repeatability and agreement are shown in Figure 2 (see Appendix). The casting angle (wrist flexion/extension and radial/ulnar deviation) had the greatest repeatability and agreement in an intra- and inter-observer setting. Additionally, radial inclination, radial shortening, and dorsal/volar tilt had good repeatability and agreement. Repeatability and agreement of sagittal shift and intra-articular gap and step were poor.
Kappa values for intra- and inter-observer agreement on radiographically acceptable reduction of the DRF are given in Table 4. Limits of acceptable reduction were those defined by the FCCG. Overall agreement concerning acceptable reduction between 5 raters was 0.59. The highest agreement between 2 raters was 0.71. The highest intra-rater agreement was 0.84. The mean overall intra- and inter-observer kappa for each measurement parameter was for radial inclination 0.77 (CI 0.67–0.86), for radial shortening 0.69 (0.59–0.78), for intra-articular gap 0.23 (0.13–0.33), for intra-articular step 0.22 (0.12–0.32), and for dorsal/volar tilt 0.60 (0.50–0.69).
The overall intra- and inter-observer disagreement between operative vs. nonoperative (i.e., acceptable vs. non-acceptable) treatment criteria was 12.8% (intra) and 20% (inter). The intra-observer disagreement ranged from 3.6–12.8% and inter-observer disagreement 6–20% for individual measurement criteria (i.e., radial shortening, radial inclination, intraarticular gap, intra-articular step,8ui and dorsal/volar tilt).
Discussion
The aim of our study was to investigate the inter- and intraobserver agreement of the radiographic measurements of dislocation and casting position of DRF. We showed that intra- and inter-observer correlations were high for radial inclination, dorsal/volar tilt, radial shortening, and casting angle of the wrist in the splint. Even though the correlations for these measurements were high, TDIs had quite a wide range. Intraarticular gap and step showed poor correlations. Inter-rater agreement was moderate when all 5 dislocation measurements defined by FCCG were taken into account.
Most distal radius fractures can be treated nonoperatively but the measurements of dislocation should be reliable and repeatable in order to allow for analogical treatment decisions between clinicians [16]. The choice of treatment is based on the patient’s age, functional status, and guidelines for acceptable radiographic parameters of the distal radius fracture after reduction. In 2009, the American Academy of Orthopedic Surgeons recommended surgical treatment if there is a gap or step > 2 mm, volar tilt > 10°, or radial shortening > 3 mm after reduction [7], which are widely used internationally [17]. In Finland, however, national criteria for acceptable DRF reduction have been adopted, and we decided to use these national criteria: dorsal tilt ≤ 15°, radial shortening ≤ 3 mm, intra-articular step ≤ 1 mm, intra-articular gap ≤ 1 mm, radial inclination ≥ 15°) [3]. Accordingly, our results must be interpreted in the light of FCCG criteria.
Kreder et al. [12] defined 8 standardized measurements of distal radius fracture. To determine inter-observer agreement, a total of 16 individuals measured these parameters on 6 radiographs of healed distal radius fractures. Thomason and Smith [13] replicated the study by Kreder et al. Stirling et al. [10] had 2 observers and a large sample of 367 radiographs of acute distal radius fractures for inter-observer analysis, but they had a sample of only 37 radiographs for intra-observer analysis. Watson et al. [11] had 5 observers and a sample of 30 radiographs. We used 50 radiographs taken after reduction and casting to evaluate reliability of measurements in a clinical setting. Accordingly our study has the largest sample size studying the intra-observer repeatability of DRF radiological measurements, increasing the clinical value of the results. We also studied the repeatability of casting position measurement for the first time. The casting position may be used in further studies investigating the optimal casting angle in conservative treatment of DRFs.
We found strong correlation in intra- and inter-observer measurements of radial inclination, dorsal/volar tilt, and radial shortening. The results are similar to those reported in previous studies [10-13]. These studies analyzed ulnar variance instead of radial shortening. Intra- and inter-observer correlations of ulnar variance were lower compared with the radial shortening correlations of our study. The results of our study suggest that radial inclination, dorsal/volar tilt, and radial shortening are reliable and repeatable enough to be used as displacement thresholds for surgical intervention. However, quite widespread error margins should be taken into account when making decisions based on thresholds. Experienced observers did not have higher correlations compared with unexperienced medical students. This suggests that trained clinicians with limited experience can use these measurements reliably when making treatment decisions.
Intra-articular gap and step had poor intra- and interobserver correlations in our study, which has also been shown previously [10-13]. This inconsistency may be due to the low number of intra-articular fractures in the study population and the narrow range of parameters (step 0–2.4 mm, gap 0–5.4 mm). There was also disagreement between raters concerning whether there was any intra-articular component or not. These results suggest that although intra-articular gap and step have been shown to be associated with poorer functional and radiological outcome, measurements of intra-articular gap and step from radiographs are not reliable enough to be used as thresholds for guiding treatment choices. Sagittal shift had slightly more reliable correlations but the clinical impact of this parameter on functional outcome is unclear.
Based on the Kappa values our research shows that when all 5 measurements used in FCCG were taken into account, intrarater agreement was excellent and inter-rater agreement for all 5 reviewers was good. This confirms that treatment decisions based on FCCG measurements are reliably reproducible.
If reduction of the distal radius fracture is acceptable, the standard treatment is immobilization in a cast. The optimal immobilization position is still controversial [9]. In reviewing the literature, studies assessing reliability or agreement of casting angle measurements in radiography were not found. In our study, intra- and inter-observer correlations of casting angle parameters were excellent.
There are some limitations to this study. The measurements were performed in a set-up similar to clinical work. Thus, human error or individual measurement accuracy differences are possible, but on the other hand the results are comparable in clinical practice. In Finland there is a standard protocol for radiography of the wrist. In this study, radiographs were taken in different hospitals, and hence there might have been some variation in the viewing angles. The average age of the study population was 76 years, and therefore arthrosis and earlier wrist injuries might have had an influence on the ability to recognize and measure intra-articular step or gap. There were females only in the study population, as the cohort was originally aimed to study postmenopausal osteoporosis and thus included only postmenopausal females. This might have had an influence on fracture type and quality of the bone. Nevertheless, sex impact on the repeatability of measurement, which was the aim of our study, may be considered low. In FCCG, surgical treatment is not recommended for patients over 65 years age, therefore our study population differs from the population in which FCCG recommendations for surgical treatment are used.
According to our study, gap and step had poor intra- and inter-observer correlations and therefore a computer tomography (CT) scan may be necessary in evaluating intra-articular incongruence. In Finland the majority of DRFs (over 80%) are treated in general healthcare, where they do not have the possibility to perform CT scans unless the patient is referred to special healthcare. Therefore, to achieve results that are comparable in clinical practice, our study is based on radiographs.
Conclusion
Intra- and inter-observer repeatability of radial inclination, radial shortening, and dorsal/volar tilt of casting position were high. Furthermore, measurements of casting angle had strong intra- and inter-observer correlations. Reliable measurements of casting angle will be useful in future research into the influence of casting angle on successful nonoperative treatment.
- Nellans K W, Kowalski E, Chung K C. The epidemiology of distal radius fractures. Hand Clin 2012; 28: 113–25. doi: 10.1016/j.hcl.2012.02.001.
- Solvang H W, Nordheggen R A, Clementsen S, Hammer OL, Randsborg P H. Epidemiology of distal radius fracture in Akershus, Norway, in 2010–2011. J Orthop Surg Res 2018; 13(1): 199. doi: 10.1186/s13018018-0904-0.
- Working group set up by the Finnish Medical Society Duodecim and the Finnish Cardiac Society. Distal radius fracture: current care guidelines. Helsinki: Finnish Medical Society Duodecim; 2016.
- Wichlas F, Haas N P, Lindner T, Tsitsilonis S. Closed reduction of distal radius fractures: does instability mean irreducibility? Arch Orthop Trauma Surg 2013; 133(8): 1073-8. doi: 10.1007/s00402-013-1758-x.
- Bentohami A, Bijlsma T S, Goslings J C, de Reuver P, Kaufmann L, Schep N W L. Radiological criteria for acceptable reduction of extra-articular distal radial fractures are not predictive for patient-reported functional outcome. J Hand Surg Eur 2013; 38(5): 524-9. doi: 10.1177/1753193412468266.
- Plant C E, Parsons N R, Costa M L. Do radiological and functional outcomes correlate for fractures of the distal radius? Bone Joint J 2017; 99B (3): 376-82. doi: 10.1302/0301-620X.99B3.35819.
- Lichtman D M, Bindra R R, Boyer M I, Putnam M D, Ring D, Slutsky D J, et al. Treatment of distal radius fractures. J Am Acad Orthop Surg 2010; 18: 180-9. doi: 10.5435/00124635-201003000-00007.
- Perugia D, Guzzini M, Civitenga C, Guidi M, Dominedò C, Fontana D, et al. Is it really necessary to restore radial anatomic parameters after distal radius fractures? Injury 2014; 45: S21–6. doi: 10.1016/j.injury.2014.10.018.
- Handoll H H, Madhok R. Conservative interventions for treating distal radial fractures in adults. Cochrane Database Syst Rev 2003; (2): CD000314. doi: 10.1002/14651858.CD000314.
- Stirling E, Jeffery J, Johnson N, Dias J. Are radiographic measurements of the displacement of a distal radial fracture reliable and reproducible? Bone Joint J 2016; 98-B(8): 1069-73. doi: 10.1302/0301620X.98B8.37469.
- Watson N J, Asadollahi S, Parrish F, Ridgway J, Tran P, Keating J L. Reliability of radiographic measurements for acute distal radius fractures. BMC Med Imaging 2016; 16(1): 44. doi: 10.1186/s12880-016-0147-7.
- Kreder H J, Hanel D P, McKee M, Jupiter J, McGillivary G, Swiontkowski M F. X-ray film measurements for healed distal radius fractures. J Hand Surg 1996; 21(1): 31-9. doi: 10.1016/S0363-5023(96)80151-1.
- Thomason K, Smith K L. The reliability of measurements taken from computer-stored digitalised X-rays of acute distal radius fractures. J Hand Surg Eur 2008; 33(3): 369-72. doi: 10.1177/1753193407087509.
- Tuppurainen M, Honkanen R, Kröger H, Saarikoski S, Alhava E. Osteoporosis risk factors, gynaecological history and fractures in perimenopausal women: the results of the baseline postal enquiry of the Kuopio Osteoporosis Risk Factor and Prevention Study. Maturitas 1993; 17(2): 89-100. doi: 10.1016/0378-5122(93)90004-2.
- Choudhary P K, Nagaraja H N. Measuring agreement: models, methods, and applications. Hoboken, NJ: Wiley; 2018.
- Mauck B M, Swigler C W. Evidence-based review of distal radius fractures. Orthop Clin North Am 2018; 49: 211-22. doi: 10.1016/j.ocl.2017.12.001.
- Xie L, Chen C, Yu X, Hu W, Chen H, Sun L. Management of distal radius fracture: a comparison of actual and theoretical treatments. Int J Surg 2018; 60: 137-40. doi: 10.1016/j.ijsu.2018.11.005.
Appendix
Figure 2. Both measurements of all 5 reviewers presented as trellis plots for (a) radial inclination, (b) radial shortening, (c) intra-articular gap, (d) intra-articular step, (e) dorsal/volar tilt, (f) sagittal shift, (g) casting angle: wrist flexion/extension, and (h) casting angle: wrist radial/ulnar deviation.