TY - JOUR

T1 - Clinicians' heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading

T2 - An exploratory study of knee radiographs using paired comparisons

AU - Pedersen, Mads Møller

AU - Mongelard, Kristian Breds Geoffroy

AU - Mørup-Petersen, Anne

AU - Christensen, Karl Bang

AU - Odgaard, Anders

PY - 2021

Y1 - 2021

N2 - Objectives Ordinal scales provide means for communicating the severity of a condition, but they are affected by cognitive biases, they introduce statistical problems and they sacrifice resolution. Clinicians discern more details than contained in scales, for example, when assessing radiographs, but clinicians' distinctions are often based on experience-based rules of thumb, that is, heuristics. The objectives of this study are to compare clinicians' heuristic assessments to ordinal grading, to identify case elements that influence clinicians' judgements and to present a method for quantifying heuristic assessments. Design Clinicians were presented with 17 207 random pairs from a set of 1087 knee radiographs. For each pair, the radiograph with more severe osteoarthritis was selected. The Bradley-Terry model was used to calculate an osteoarthritis strength parameter for each radiograph. Similarly, strength parameters were determined for 12 morphological features with five additional features being considered either present or absent. All radiographs were also graded according to conventional ordinal systems (Kellgren-Lawrence and Ahlbäck). Relations between clinicians' judgements and (1) the heuristics-based osteoarthritis strength, (2) conventional ordinal systems and (3) morphological features were investigated. Results Receiver operating characteristic analysis showed that the Bradley-Terry model provided a good description of clinicians' assessments (area under the curve (AUC)=0.97, 95% CI 0.968 to 0.972). Morphological features (AUC=0.90, 95% CI 0.900 to 0.908) provided a superior description of clinicians' choices compared with conventional ordinal systems (AUC=0.88, 95% CI 0.878 to 0.887 and AUC=0.80, 95% CI 0.796 to 0.809) for Ahlbäck and Kellgren-Lawrence, respectively). The features most strongly associated with osteoarthritis strength were medial joint space width, flattening of the medial femoral and tibial condyles, medial osteophytes and alignment. Conclusions Heuristics-based assessments give a better distinction than conventional grading systems of knee osteoarthritis. The example presents a general approach to evaluate which features are part of experts' heuristics. The data suggest that experts discern more details than included in conventional ordinal grading systems. Quantitative heuristic assessments may replace ordinal scales.

AB - Objectives Ordinal scales provide means for communicating the severity of a condition, but they are affected by cognitive biases, they introduce statistical problems and they sacrifice resolution. Clinicians discern more details than contained in scales, for example, when assessing radiographs, but clinicians' distinctions are often based on experience-based rules of thumb, that is, heuristics. The objectives of this study are to compare clinicians' heuristic assessments to ordinal grading, to identify case elements that influence clinicians' judgements and to present a method for quantifying heuristic assessments. Design Clinicians were presented with 17 207 random pairs from a set of 1087 knee radiographs. For each pair, the radiograph with more severe osteoarthritis was selected. The Bradley-Terry model was used to calculate an osteoarthritis strength parameter for each radiograph. Similarly, strength parameters were determined for 12 morphological features with five additional features being considered either present or absent. All radiographs were also graded according to conventional ordinal systems (Kellgren-Lawrence and Ahlbäck). Relations between clinicians' judgements and (1) the heuristics-based osteoarthritis strength, (2) conventional ordinal systems and (3) morphological features were investigated. Results Receiver operating characteristic analysis showed that the Bradley-Terry model provided a good description of clinicians' assessments (area under the curve (AUC)=0.97, 95% CI 0.968 to 0.972). Morphological features (AUC=0.90, 95% CI 0.900 to 0.908) provided a superior description of clinicians' choices compared with conventional ordinal systems (AUC=0.88, 95% CI 0.878 to 0.887 and AUC=0.80, 95% CI 0.796 to 0.809) for Ahlbäck and Kellgren-Lawrence, respectively). The features most strongly associated with osteoarthritis strength were medial joint space width, flattening of the medial femoral and tibial condyles, medial osteophytes and alignment. Conclusions Heuristics-based assessments give a better distinction than conventional grading systems of knee osteoarthritis. The example presents a general approach to evaluate which features are part of experts' heuristics. The data suggest that experts discern more details than included in conventional ordinal grading systems. Quantitative heuristic assessments may replace ordinal scales.

KW - diagnostic radiology

KW - knee

KW - musculoskeletal disorders

KW - radiology & imaging

KW - statistics & research methods

U2 - 10.1136/bmjopen-2020-041793

DO - 10.1136/bmjopen-2020-041793

M3 - Journal article

C2 - 34006025

AN - SCOPUS:85102257381

VL - 11

JO - BMJ Open

JF - BMJ Open

SN - 2044-6055

IS - 3

M1 - e041793

ER -