This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Bland, J M. Articles by Altman, D. G Search for Related Content PubMed PubMed Citation Articles by Bland, J M. Articles by Altman, D. G Social Bookmarking                   What's this?

Measuring agreement in method comparison studies

Department of Public Health Sciences, St George's Hospital Medical School, London, UK

Douglas G Altman

ICRF Medical Statistics Group, Centre for Statistics in Medicine, Institute of Health Sciences, Oxford, UK

Agreement between two methods of clinical measurement can be quantified using the differences between observations made using the two methods on the same subjects. The 95% limits of agreement, estimated by mean difference 1.96 standard deviation of the differences, provide an interval within which 95% of differences between measurements by the two methods are expected to lie. We describe how graphical methods can be used to investigate the assumptions of the method and we also give confidence intervals. We extend the basic approach to data where there is a relationship between difference and magnitude, both with a simple logarithmic transformation approach and a new, more general, regression approach. We discuss the importance of the repeatability of each method separately and compare an estimate of this to the limits of agreement. We extend the limits of agreement approach to data with repeated measurements, proposing new estimates for equal numbers of replicates by each method on each subject, for unequal numbers of replicates, and for replicated data collected in pairs, where the underlying value of the quantity being measured is changing. Finally, we describe a nonparametric approach to comparing methods.

CiteULike    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				P. N. Ainslie, J. D. Cotter, K. P. George, S. Lucas, C. Murrell, R. Shave, K. N. Thomas, M. J. A. Williams, and G. Atkinson Elevation in cerebral blood flow velocity with aerobic fitness throughout healthy human ageing J. Physiol., August 15, 2008; 586(16): 4005 - 4010. [Abstract] [Full Text] [PDF]

				Y. Wang, R. J. van Klaveren, H. J. van der Zaag-Loonen, G. H. de Bock, H. A. Gietema, D. M. Xu, A. L. M. Leusveld, H. J. de Koning, E. T. Scholten, J. Verschakelen, et al. Effect of Nodule Characteristics on Variability of Semiautomated Volume Measurements in Pulmonary Nodules Detected in a Lung Cancer Screening Program Radiology, August 1, 2008; 248(2): 625 - 631. [Abstract] [Full Text] [PDF]

				A. D. Leonard, C. M. Allsager, J. L. Parker, A. Swami, and J. P. Thompson Comparison of central venous and external jugular venous pressures during repair of proximal femoral fracture Br. J. Anaesth., August 1, 2008; 101(2): 166 - 170. [Abstract] [Full Text] [PDF]

				B. H. Amundsen, J. Crosby, P. A. Steen, H. Torp, S. A. Slordahl, and A. Stoylen Regional myocardial long-axis strain and strain rate measured by different tissue Doppler and speckle tracking echocardiography methods: a comparison with tagged magnetic resonance imaging Eur J Echocardiogr, July 23, 2008; (2008) jen201v1. [Abstract] [Full Text] [PDF]

				O. Oladeji, C. Stackhouse, E. Gracely, and M. Orlin Comparison of the Two-Step and Midgait Methods of Plantar Pressure Measurement in Children J Am Podiatr Med Assoc, July 1, 2008; 98(4): 268 - 277. [Abstract] [Full Text] [PDF]

				T. I. Leakey, J. Zielinski, R. N. Siegfried, E. R. Siegel, C.-Y. Fan, and C. A. Cooney A simple algorithm for quantifying DNA methylation levels on multiple independent CpG sites in bisulfite genomic sequencing electropherograms Nucleic Acids Res., June 1, 2008; 36(11): e64 - e64. [Abstract] [Full Text] [PDF]

				C. Schneider, R. Malisius, K. Krause, F. Lampe, E. Bahlmann, S. Boczor, M. Antz, S. Ernst, and K.-H. Kuck Strain rate imaging for functional quantification of the left atrium: atrial deformation predicts the maintenance of sinus rhythm after catheter ablation of atrial fibrillation Eur. Heart J., June 1, 2008; 29(11): 1397 - 1409. [Abstract] [Full Text] [PDF]

				M. Haber and H. X Barnhart A general approach to evaluating agreement between two observers or methods of measurement from quantitative data with replicated measurements Statistical Methods in Medical Research, April 1, 2008; 17(2): 151 - 169. [Abstract] [PDF]

				A. M. Arafat, M. Mohlig, M. O. Weickert, F. H. Perschel, J. Purschwitz, J. Spranger, C. J. Strasburger, C. Schofl, and A. F. H. Pfeiffer Growth Hormone Response during Oral Glucose Tolerance Test: The Impact of Assay Method on the Estimation of Reference Values in Patients with Acromegaly and in Healthy Controls, and the Role of Gender, Age, and Body Mass Index J. Clin. Endocrinol. Metab., April 1, 2008; 93(4): 1254 - 1262. [Abstract] [Full Text] [PDF]

				H. Werner, L. Molinari, C. Guyer, and O. G. Jenni Agreement Rates Between Actigraphy, Diary, and Questionnaire for Children's Sleep Patterns Arch Pediatr Adolesc Med, April 1, 2008; 162(4): 350 - 358. [Abstract] [Full Text] [PDF]

				T Piche, M C Saint-Paul, R Dainese, E Marine-Barjoan, A Iannelli, M L Montoya, J F Peyron, D Czerucka, F Cherikh, J Filippi, et al. Mast cells and cellularity of the colonic mucosa correlated with fatigue and depression in irritable bowel syndrome Gut, April 1, 2008; 57(4): 468 - 473. [Abstract] [Full Text] [PDF]

				S. G. Chan, S. C. Ho, N. Kreiger, G. Darlington, E. M. Adlaf, K. F. So, and P. Y. Y. Chong Validation of a Food Frequency Questionnaire for Assessing Dietary Soy Isoflavone Intake among Midlife Chinese Women in Hong Kong J. Nutr., March 1, 2008; 138(3): 567 - 573. [Abstract] [Full Text] [PDF]

				C. L. Gorman, A. I. Russell, Z. Zhang, D. Cunninghame Graham, A. P. Cope, and T. J. Vyse Polymorphisms in the CD3Z Gene Influence TCR{zeta} Expression in Systemic Lupus Erythematosus Patients and Healthy Controls J. Immunol., January 15, 2008; 180(2): 1060 - 1070. [Abstract] [Full Text] [PDF]

				K. Staier, C. Wiesenack, L. Gunkel, and C. Keyl Cardiac output determination by thermodilution and arterial pulse waveform analysis in patients undergoing aortic valve replacement: [Determination du debit cardiaque par thermodilution et par analyse du contour de pression arterielle chez des patients subissant un remplacement de la valve aortique] Can J Anesth, January 1, 2008; 55(1): 22 - 28. [Abstract] [Full Text] [PDF]

				J. L. Chen, A. Gittleman, P. D. Barnes, and K. W. Chang Utility of Temporal Bone Computed Tomographic Measurements in the Evaluation of Inner Ear Malformations Arch Otolaryngol Head Neck Surg, January 1, 2008; 134(1): 50 - 56. [Abstract] [Full Text] [PDF]

				F. Y. Obese, S. Humphrys, K. L. Macmillan, and A. R. Egan Measuring Concentrations of Insulin-Like Growth Factor-I with an Enzyme-Linked Immunosorbent Antibody Assay in Plasma Samples from Holstein Cows J Dairy Sci, January 1, 2008; 91(1): 160 - 168. [Abstract] [Full Text] [PDF]

				I. Decramer, P. K. Vanhoenacker, G. Sarno, L. Van Hoe, O. Bladt, W. Wijns, and P. M. Parizel Effects of Sublingual Nitroglycerin on Coronary Lumen Diameter and Number of Visualized Septal Branches on 64-MDCT Angiography Am. J. Roentgenol., January 1, 2008; 190(1): 219 - 225. [Abstract] [Full Text] [PDF]

				N. Khuseyinova, S. Greven, R. Ruckerl, G. Trischler, H. Loewel, A. Peters, and W. Koenig Variability of Serial Lipoprotein-Associated Phospholipase A2 Measurements in Post Myocardial Infarction Patients: Results from the AIRGENE Study Center Augsburg Clin. Chem., January 1, 2008; 54(1): 124 - 130. [Abstract] [Full Text] [PDF]

				J. W. Castellani, C. O'Brien, P. Tikuisis, I. V. Sils, and X. Xu Evaluation of two cold thermoregulatory models for prediction of core temperature during exercise in cold water J Appl Physiol, December 1, 2007; 103(6): 2034 - 2041. [Abstract] [Full Text] [PDF]

				E. Hypponen, S. Turner, P. Cumberland, C. Power, and I. Gibb Serum 25-Hydroxyvitamin D Measurement in a Large Population Survey with Statistical Harmonization of Assay Variation to an International Standard J. Clin. Endocrinol. Metab., December 1, 2007; 92(12): 4615 - 4622. [Abstract] [Full Text] [PDF]

				M. Eichinger, M. Puderbach, H-J. Smith, R. Tetzlaff, A. Kopp-Schneider, M. Bock, J. Biederer, and H-U. Kauczor Magnetic resonance-compatible-spirometry: principle, technical evaluation and application Eur. Respir. J., November 1, 2007; 30(5): 972 - 979. [Abstract] [Full Text] [PDF]

				M. Causse, A. D. Garcia-Mayorgas, J. B. Gutierrez, and M. Casal Preliminary Evaluation of a Multiplex Reverse Transcription-PCR Assay Combined with a New DNA Chip Hybridization Assay for Detecting Respiratory Syncytial Virus J. Clin. Microbiol., November 1, 2007; 45(11): 3811 - 3813. [Abstract] [Full Text] [PDF]

				R. S. Parekh, W.H. L. Kao, L. A. Meoni, E. Ipp, P. L. Kimmel, J. La Page, C. Fondran, W. C. Knowler, M. J. Klag, and and the Family Investigation of Nephropathy and Di Reliability of Urinary Albumin, Total Protein, and Creatinine Assays after Prolonged Storage: The Family Investigation of Nephropathy and Diabetes Clin. J. Am. Soc. Nephrol., November 1, 2007; 2(6): 1156 - 1162. [Abstract] [Full Text] [PDF]

				C. Lefoll-Masson, C. Fermanian, I. Aime, N. Verroust, G. Taylor, P.-A. Laloe, N. Liu, P. Aegerter, and M. Fischler The Comparability of Bispectral Index and State Entropy Index During Maintenance of Sufentanil-Sevoflurane-Nitrous Oxide Anesthesia Anesth. Analg., November 1, 2007; 105(5): 1319 - 1325. [Abstract] [Full Text] [PDF]

				E. C. Uong, M. Epperson, S. A. Bathon, and D. B. Jeffe Adherence to Nasal Positive Airway Pressure Therapy Among School-aged Children and Adolescents With Obstructive Sleep Apnea Syndrome Pediatrics, November 1, 2007; 120(5): e1203 - e1211. [Abstract] [Full Text] [PDF]

				R. M. Determann, E. K. Wolthuis, P. E. Spronk, M. A. Kuiper, J. C. Korevaar, M. B. Vroom, and M. J. Schultz Reliability of Height and Weight Estimates in Patients Acutely Admitted to Intensive Care Units Crit. Care Nurse, October 1, 2007; 27(5): 48 - 55. [Full Text] [PDF]

				M. E. Johnson and P. J. Murphy Measurement of Ocular Surface Irritation on a Linear Interval Scale with the Ocular Comfort Index Invest. Ophthalmol. Vis. Sci., October 1, 2007; 48(10): 4451 - 4458. [Abstract] [Full Text] [PDF]

				L. S. Kim, J. J. McAnany, K. R. Alexander, and G. A. Fishman Intersession Repeatability of Humphrey Perimetry Measurements in Patients with Retinitis Pigmentosa Invest. Ophthalmol. Vis. Sci., October 1, 2007; 48(10): 4720 - 4724. [Abstract] [Full Text] [PDF]

				P. S. Myles and J. Cui I. Using the Bland Altman method to measure agreement with repeated measures Br. J. Anaesth., September 1, 2007; 99(3): 309 - 311. [Full Text] [PDF]

				M. Friden, A. Gupta, M. Antonsson, U. Bredberg, and M. Hammarlund-Udenaes In Vitro Methods for Estimating Unbound Drug Concentrations in the Brain Interstitial and Intracellular Fluids Drug Metab. Dispos., September 1, 2007; 35(9): 1711 - 1719. [Abstract] [Full Text] [PDF]

				A. Ruggeri, E. Grisan, and J. Schroeter Evaluation of repeatability for the automatic estimation of endothelial cell density in donor corneas Br. J. Ophthalmol., September 1, 2007; 91(9): 1213 - 1215. [Abstract] [Full Text] [PDF]

				M. Crilly, C. Coch, M. Bruce, H. Clark, and D. Williams Indices of cardiovascular function derived from peripheral pulse wave analysis using radial applanation tonometry: a measurement repeatability study Vascular Medicine, August 1, 2007; 12(3): 189 - 197. [Abstract] [PDF]

				R. Cortes, M. Rivera, A. Salvador, V. Bertomeu, F. G. de Burgos, E. Rosello-Lleti, M. Portoles, R. Paya, L. Martinez-Dolz, and V. Climent Variability of NT-proBNP plasma and urine levels in patients with stable heart failure: a 2-year follow-up study Heart, August 1, 2007; 93(8): 957 - 962. [Abstract] [Full Text] [PDF]

				R. Lavanya, L. Teo, D. S Friedman, H. T Aung, M. Baskaran, H. Gao, T. Alfred, S. K Seah, K. Kashiwagi, P. J Foster, et al. Comparison of anterior chamber depth measurements using the IOLMaster, scanning peripheral anterior chamber depth analyser, and anterior segment optical coherence tomography Br. J. Ophthalmol., August 1, 2007; 91(8): 1023 - 1026. [Abstract] [Full Text] [PDF]

				E. Selvin, C. M. Crainiceanu, F. L. Brancati, and J. Coresh Short-term Variability in Measures of Glycemia and Implications for the Classification of Diabetes Arch Intern Med, July 23, 2007; 167(14): 1545 - 1551. [Abstract] [Full Text] [PDF]

				N. M. Kayes, K. M. McPherson, D. Taylor, P. J. Schluter, B.-J. K. Wilson, and G. S. Kolt The Physical Activity and Disability Survey (PADS): reliability, validity and acceptability in people with multiple sclerosis Clinical Rehabilitation, July 1, 2007; 21(7): 628 - 639. [Abstract] [PDF]

				H.-M. Chen, C.-L. Hsieh, Sing Kai Lo, L.-J. Liaw, S.-M. Chen, and J.-H. Lin The Test-Retest Reliability of 2 Mobility Performance Tests in Patients With Chronic Stroke Neurorehabil Neural Repair, July 1, 2007; 21(4): 347 - 352. [Abstract] [PDF]

				N. Janzen, M. Peter, S. Sander, U. Steuerwald, M. Terhardt, U. Holtkamp, and J. Sander Newborn Screening for Congenital Adrenal Hyperplasia: Additional Steroid Profile using Liquid Chromatography-Tandem Mass Spectrometry J. Clin. Endocrinol. Metab., July 1, 2007; 92(7): 2581 - 2589. [Abstract] [Full Text] [PDF]

				M. Rynning, T. Wentzel-Larsen, and B. J. Bolann A Model for an Uncertainty Budget for Preanalytical Variables in Clinical Chemistry Analyses Clin. Chem., July 1, 2007; 53(7): 1343 - 1348. [Abstract] [Full Text] [PDF]

				J. M. Nielsen, S. B. Kristiansen, S. Ringgaard, T. T. Nielsen, A. Flyvbjerg, A. N. Redington, and H. E. Botker Left ventricular volume measurement in mice by conductance catheter: evaluation and optimization of calibration Am J Physiol Heart Circ Physiol, July 1, 2007; 293(1): H534 - H540. [Abstract] [Full Text] [PDF]

				A. Boonen, D. van der Heijde, R. Landewe, A. van Tubergen, H. Mielants, M. Dougados, and S. van der Linden How do the EQ-5D, SF-6D and the well-being rating scale compare in patients with ankylosing spondylitis? Ann Rheum Dis, June 1, 2007; 66(6): 771 - 777. [Abstract] [Full Text] [PDF]

				M. Kverka, J. Burianova, R. Lodinova-Zadnikova, I. Kocourkova, J. Cinova, L. Tuckova, and H. Tlaskalova-Hogenova Cytokine Profiling in Human Colostrum and Milk by Protein Array Clin. Chem., May 1, 2007; 53(5): 955 - 962. [Abstract] [Full Text] [PDF]

				C. Arzola, S. Davies, A. Rofaeel, and J. C. A. Carvalho Ultrasound Using the Transverse Approach to the Lumbar Spine Provides Reliable Landmarks for Labor Epidurals Anesth. Analg., May 1, 2007; 104(5): 1188 - 1192. [Abstract] [Full Text] [PDF]

				D. J. C. Alders, R. N. Cornelussen, F. W. Prinzen, P. A. C. Specht, M. I. M. Noble, A. J. Drake-Holland, F. J. J. de Kanter, and J. H. G. M. van Beek Heart/Cardiac Muscle: Regional sympathetic denervation affects the relation between canine local myocardial blood flow and oxygen consumption Exp Physiol, May 1, 2007; 92(3): 541 - 548. [Abstract] [Full Text] [PDF]

				F. Zulian, D. Meneghesso, E. Grisan, F. Vittadello, A. Belloni Fortina, B. Pigozzi, A. C. Frigo, G. Martini, and A. Ruggeri A new computerized method for the assessment of skin lesions in localized scleroderma Rheumatology, May 1, 2007; 46(5): 856 - 860. [Abstract] [Full Text] [PDF]

				J. Thompson, M. Reid, I. Hambleton, and G. R. Serjeant Albuminuria and Renal Function in Homozygous Sickle Cell Disease: Observations From a Cohort Study Arch Intern Med, April 9, 2007; 167(7): 701 - 708. [Abstract] [Full Text] [PDF]

				Z. Jobbagy, R. van Atta, K. M. Murphy, J. R. Eshleman, and C. D. Gocke Evaluation of the Cepheid GeneXpert BCR-ABL Assay J. Mol. Diagn., April 1, 2007; 9(2): 220 - 227. [Abstract] [Full Text] [PDF]

				J. E. DeLeon Ortega, L. M. Sakata, B. Kakati, G. McGwin Jr, B. E. Monheit, S. N. Arthur, and C. A. Girkin Effect of Glaucomatous Damage on Repeatability of Confocal Scanning Laser Ophthalmoscope, Scanning Laser Polarimetry, and Optical Coherence Tomography Invest. Ophthalmol. Vis. Sci., March 1, 2007; 48(3): 1156 - 1163. [Abstract] [Full Text] [PDF]

				E Rydwik, C Karlsson, K Frandin, and G Akner Muscle strength testing with one repetition maximum in the arm/shoulder for people aged 75 + - test-retest reliability Clinical Rehabilitation, March 1, 2007; 21(3): 258 - 265. [Abstract] [PDF]

				G. T. Allison Statistical and Clinical Interpretation of Research Results J Am Podiatr Med Assoc, March 1, 2007; 97(2): 165 - 170. [Abstract] [Full Text] [PDF]

				B. K. Kojonazarov, B. Z. Imanov, T. A. Amatov, M. M. Mirrakhimov, R. Naeije, M. R. Wilkins, and A. A. Aldashev Noninvasive and invasive evaluation of pulmonary arterial pressure in highlanders Eur. Respir. J., February 1, 2007; 29(2): 352 - 356. [Abstract] [Full Text] [PDF]

				BASEM 2006 Annual Conference Br. J. Sports Med., February 1, 2007; 41(2): 118 - 124. [Full Text] [PDF]

				M. Guilarte, J. Santos, I. de Torres, C. Alonso, M. Vicario, L. Ramos, C. Martinez, F. Casellas, E. Saperas, and J. R. Malagelada Diarrhoea-predominant IBS patients show mast cell activation and hyperplasia in the jejunum Gut, February 1, 2007; 56(2): 203 - 209. [Abstract] [Full Text] [PDF]

				S. D. Allen, A. R. Padhani, A. S. Dzik-Jurasz, and R. Glynne-Jones Rectal Carcinoma: MRI with Histologic Correlation Before and After Chemoradiation Therapy Am. J. Roentgenol., February 1, 2007; 188(2): 442 - 451. [Abstract] [Full Text] [PDF]

				O. I.I. Soliman, D. A.M.J. Theuns, M. L. Geleijnse, A. M. Anwar, A. Nemes, K. Caliskan, W. B. Vletter, L. J. Jordaens, and F. J. T. Cate Spectral pulsed-wave tissue Doppler imaging lateral-to-septal delay fails to predict clinical or echocardiographic outcome after cardiac resynchronization therapy Europace, February 1, 2007; 9(2): 113 - 118. [Abstract] [Full Text] [PDF]

				A. Lavinio, E. A. Schmidt, C. Haubrich, P. Smielewski, J. D. Pickard, and M. Czosnyka Noninvasive Evaluation of Dynamic Cerebrovascular Autoregulation Using Finapres Plethysmograph and Transcranial Doppler Stroke, February 1, 2007; 38(2): 402 - 404. [Abstract] [Full Text] [PDF]

A.S. Turk, K.M. Johnson, D. Lum, D. Niemann, B. Aagaard-Kienitz, D. Consigny, J. Grinde, P. Turski, V. Haughton, and C. Mistretta Physiologic and Anatomic Assessment of a Canine Carotid Artery Stenosis Model Utilizing Phase Contrast with Vastly Undersampled Isotropic Projection Imaging AJ