High agreement but low kappa: II. Resolving the paradoxes

Title

High agreement but low kappa: II. Resolving the paradoxes

Creator

Cicchetti DV; Feinstein AR

Publisher

Journal Of Clinical Epidemiology

Date

1990

Subject

Sensitivity and Specificity; Observer Variation; Q3 Literature Search; Statistics as Topic

Description

An omnibus index offers a single summary expression for a fourfold table of binary concordance among two observers. Among the available other omnibus indexes, none offers a satisfactory solution for the paradoxes that occur with p0 and kappa. The problem can be avoided only by using ppos and pneg as two separate indexes of proportionate agreement in the observers' positive and negative decisions. These two indexes, which are analogous to sensitivity and specificity for concordance in a diagnostic marker test, create the paradoxes formed when the chance correction in kappa is calculated as a product of the increment in the two indexes and the increment in marginal totals. If only a single omnibus index is used to compared different performances in observer variability, the paradoxes of kappa are desirable since they appropriately "penalize" inequalities in ppos and pneg. For better understanding of results and for planning improvements in the observers' performance, however, the omnibus value of kappa should always be accompanied by separate individual values of ppos and pneg.
1990

Rights

Article information provided for research and reference use only. PedPalASCNET does not hold any rights over the resource listed here. All rights are retained by the journal listed under publisher and/or the creator(s).

Type

Journal Article

Citation List Month

Backlog

Citation

Cicchetti DV; Feinstein AR, “High agreement but low kappa: II. Resolving the paradoxes,” Pediatric Palliative Care Library, accessed March 29, 2024, https://pedpalascnetlibrary.omeka.net/items/show/12535.