J Manipulative Physiol Ther. 1991 Feb; 14(2): 119-32.

Statistical methodology for reliability studies.

Haas M.

Research Department, Western States Chiropractic College, Portland, OR 97230.

Although a considerable number of reliability studies have appeared in the chiropractic literature in the last 10 yr, there is still no consensus on the definition of reliability, appropriate statistics for reliability testing or interpretation of results. This paper offers suggestions for the standardization of experimental design, analysis and evaluation for reliability studies. The primary focus is on appropriate concordance statistics, i.e., Kappa and intraclass correlation. These are presented in detail, together with a discussion of their limitations. Less suitable, but often used indices such as Pearson's r are also discussed. A measure of precision, the interexaminer measurement error, is introduced and a methodology for evaluating precision is presented. Clinical issues which must be considered in reliability studies are discussed; specifically, those issues relevant to studies of diagnostic indicators for spinal manipulation.