Rorschach interrater agreement estimates: An empirical evaluation

Journal/Book: Scand J Psychol. 1999; 40: PO Box 2959 Toyen, Journal Division Customer Service, N-0608 Oslo, Norway. Scandinavian University Press. 115-120.

Abstract: A standardized estimation of Rorschach interrater agreement is needed. Percentage agreement, although widely used, is found to be unsuitable. Forty-one protocols from adults in both a normal and a psychiatric sample were scored by two or three scorers, making 85 scoring pairs. Percentage agreement, correlations (phi and Pearson's r), and kappa were computed on single response, total score, and category level. Percentage agreement shows minimal variation. Even when exceeding 0.80, it can obscure major disagreements. Kappa and correlations both vary in a similar way with level of disagreement. Total score level does not give additional information compared to single score and category levels. Kappa proved to be conservative and reliable and is therefore suggested as a standard estimate.

Note: Article Gronnerod C, Univ Oslo, Dept Psychol, PB 1094 Blindern, N-0383 Oslo, NORWAY

Keyword(s): Rorschach; interrater agreement; kappa; percentage agreement; PSYCHIATRIC-DIAGNOSIS; INTERSCORER AGREEMENT; CATEGORICAL DATA; RELIABILITY; KAPPA; RATERS; COEFFICIENT; VARIANCE; SYSTEM; MODEL

Zurück | Weiter