This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Haber, M. Articles by Barnhart, H. X Search for Related Content PubMed PubMed Citation Articles by Haber, M. Articles by Barnhart, H. X Social Bookmarking                         What's this?

Coefficients of agreement for fixed observers

Department of Biostatistics, Rollins School of Public Health, Emory University, Atlanta, GA, USA, mhaber{at}sph.emory.edu

Huiman X Barnhart

Department of Biostatistics and Bioinformatics, Duke Clinical Research Institute, Duke University, PO Box 17969, Durham, NC 27715, USA

Agreement between fixed observers or methods that produce readings on a continuous scale is usually evaluated via one of several intraclass correlation coefficients (ICCs). This article presents and discusses a few related issues that have not been raised before. ICCs are usually presented in the context of a two-way analysis of variance (ANOVA) model. We argue that the ANOVA model makes inadequate assumptions, such as the homogeneity of the error variances and of the pairwise correlation coefficients between observers. We then present the concept of observer relational agreement which has been used in the social sciences to derive the common ICCs without making the restrictive ANOVA assumptions. This concept did not receive much attention in the biomedical literature. When observer agreement is defined in terms of the difference of the readings of different observers on the same subject (absolute agreement), the corresponding relational agreement coefficient coincides with the concordance correlation coefficient (CCC), which is also an ICC. The CCC, which has gained popularity over the past 15 years, compares the mean squared difference between readings of observers on the same subject with the expected value of this quantity under the assumption of ‘chance agreement’, which is defined as independence between observers. We argue that the assumption of independence is unrealistic in this context and present a new coefficient that is not based on the concept of chance agreement.

Statistical Methods in Medical Research, Vol. 15, No. 3, 255-271 (2006)
DOI: 10.1191/0962280206sm441oa


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				M. A. Virji, S. R. Woskie, M. Waters, S. Brueck, D. Stancescu, R. Gore, C. Estill, and M. Prince Agreement between Task-Based Estimates of the Full-Shift Noise Exposure and the Full-Shift Noise Dosimetry Ann. Hyg., April 1, 2009; 53(3): 201 - 214. [Abstract] [Full Text] [PDF]

				M. Haber and H. X Barnhart A general approach to evaluating agreement between two observers or methods of measurement from quantitative data with replicated measurements Statistical Methods in Medical Research, April 1, 2008; 17(2): 151 - 169. [Abstract] [PDF]