Three related concepts: Interobserver Agreement, Interobserver Correlation, and Interobserver Reliability Coefficient

Relationship Between Interobserver Agreement, Interobserver Correlation, and Interobserver Reliability Coefficient

Percent Agreement

Interobserver Reliability Correlation

(r)

Interobserver Reliability Coefficient

(r²)

Example of Data With This Degree of Interobserver Agreement

100%

1

1

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	1
5	0	0
6	1	1
7	0	0
8	1	1
9	0	0
10	1	1

90%

0.8

0.64

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	1
5	0	0
6	1	1
7	0	0
8	1	1
9	0	0
10	1	0

80%

0.6

0.36

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	1
5	0	0
6	1	1
7	0	0
8	1	1
9	0	1
10	1	0

70%

0.4

0.17

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	1
5	0	0
6	1	1
7	0	0
8	1	0
9	0	1
10	1	0

60%

0.2

0.04

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	1
5	0	0
6	1	1
7	0	1
8	1	0
9	0	1
10	1	0

50%

0

0

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	1
5	0	0
6	1	0
7	0	1
8	1	0
9	0	1
10	1	0

40%

-0.2

0.04

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	1
5	0	1
6	1	0
7	0	1
8	1	0
9	0	1
10	1	0

30%

-0.4

0.17

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	0
4	1	0
5	0	1
6	1	0
7	0	1
8	1	0
9	0	1
10	1	0

20%

-0.6

0.36

Participant	Observer 1’s score	Observer 2’s score
1	0	0
2	1	1
3	0	1
4	1	0
5	0	1
6	1	0
7	0	1
8	1	0
9	0	1
10	1	0

10%

-0.8

0.64

<

Participant

Observer 1’s score