What is inter-rater reliability?

What is inter-rater reliability?

• A. Comparing the ratings of two or more observers to check for agreement in their measurements.
• B. The consistency of a test over time.
• C. The correlation between two parallel forms.
• D. The bias of raters.

Answer: (A)

Inter-rater reliability is about how consistently different observers rate the same thing. In clinical psychology, many measures rely on judgments or coding of behavior, so you want observers to agree rather than produce idiosyncratic differences. A high level of agreement suggests the rating system is clear and the scores reflect the phenomenon being measured, not just one rater’s opinion. This is different from other reliabilities: test-retest reliability looks at consistency of a score over time, parallel-forms reliability checks consistency between two versions of a test, and rater bias refers to systematic errors introduced by a rater’s own tendencies. When two raters assess the same behavior, metrics like Cohen’s kappa or intraclass correlation can quantify how much they agree, indicating strong inter-rater reliability.

Inter-rater reliability is about how consistently different observers rate the same thing. In clinical psychology, many measures rely on judgments or coding of behavior, so you want observers to agree rather than produce idiosyncratic differences. A high level of agreement suggests the rating system is clear and the scores reflect the phenomenon being measured, not just one rater’s opinion. This is different from other reliabilities: test-retest reliability looks at consistency of a score over time, parallel-forms reliability checks consistency between two versions of a test, and rater bias refers to systematic errors introduced by a rater’s own tendencies. When two raters assess the same behavior, metrics like Cohen’s kappa or intraclass correlation can quantify how much they agree, indicating strong inter-rater reliability.