Rationale and objectives: The ability of a test to influence diagnostic confidence is used as a measure of its efficacy. Our aim was to compare analytic methods that evaluate changes in confidence.
Materials and methods: The approaches compared were "basic," "retained diagnoses," "Omary," "Tsushima," and "score-based" methods. For illustration, data from a clinical study assessing changes in diagnostic confidence (0%-100%) before and after abdominopelvic computed tomography (CT) in patients with acute abdominal pain were used.
Results: The basic, retained diagnoses and Omary methods all ignore whether the test yields a correct diagnosis (confident, but incorrect, diagnoses are regarded positively). Although the Tsushima method takes some account of diagnostic accuracy, all misdiagnoses are considered equal. The score-based method addresses some of the fundamental limitations in the other analytical methods, such as diagnostic accuracy and the varying nature of different misdiagnoses. In the case study, mean (SD) diagnostic confidence for the cohort as a whole (n = 62) increased following CT: 50.7% (20.8%) to 73.2% (20.9%). Pretest diagnoses were changed following CT in 43% (27 of 62) of patients. Pretest diagnoses proved to be incorrect in 52% (32 of 62), and post-test diagnoses incorrect in as many as 19% (12 of 62) of patients. All five analytic methods indicated a positive contribution for CT (all P < or = .003).
Conclusion: Although our illustrative case study revealed no consequential differences across the five methods, there remain substantial differences in the fundamental principles underlying them that should affect choice of analytic method when assessing diagnostic confidence.