High agreement but low kappa: II. Resolving the paradoxes

High agreement but low kappa: II. Resolving the paradoxes

Cicchetti DV; Feinstein AR

Journal Of Clinical Epidemiology

1990

Sensitivity and Specificity; Observer Variation; Q3 Literature Search; Statistics as Topic

An omnibus index offers a single summary expression for a fourfold table of binary concordance among two observers. Among the available other omnibus indexes, none offers a satisfactory solution for the paradoxes that occur with p0 and kappa. The problem can be avoided only by using ppos and pneg as two separate indexes of proportionate agreement in the observers' positive and negative decisions. These two indexes, which are analogous to sensitivity and specificity for concordance in a diagnostic marker test, create the paradoxes formed when the chance correction in kappa is calculated as a product of the increment in the two indexes and the increment in marginal totals. If only a single omnibus index is used to compared different performances in observer variability, the paradoxes of kappa are desirable since they appropriately "penalize" inequalities in ppos and pneg. For better understanding of results and for planning improvements in the observers' performance, however, the omnibus value of kappa should always be accompanied by separate individual values of ppos and pneg.

1990

Article information provided for research and reference use only. PedPalASCNET does not hold any rights over the resource listed here. All rights are retained by the journal listed under publisher and/or the creator(s).

Journal Article

Backlog