Exploring inter-rater reliability and measurement properties of environmental ratings using kappa and colocation quotients

Table 4 Index – nearest neighbor rater agreement on serenity in the close outdoor environment

	Rel	Nearest neighbor rater
	freq^a	1 - 2. Disagree completely/Disagree		3. Agree		4. Agree completely		5 - 6. Cannot say/Not answered
	(%)	Obs/Exp^b	CLQ (K %)	Obs/Exp^b	CLQ (K %)	Obs/Exp^b	CLQ (K %)	Obs/Exp^b	CLQ (K %)
Index rater
1-2 Disagree completely/Disagree	23	8.5/5.4	1.6 (17)^***	9.5/9.7	0.97 (−1.2)	3.3/6.5	0.51 (−16)^***	2.0/1.6	1.2 (2.9)^**
3 Agree	42			19/17	1.1 (4.6)^***	11/12	0.92 ( −3.7)^***	2.9/2.9	0.98 (−0.2)
4 Agree completely	28					12/7.8	1.6 (23)^***	1.4/2.0	0.68 (−3.9)^***
5-6 Cannot say/Not answered	7.0							0.77/0.49	1.6 (4.4)^**

**p <0.01; ***p <0.001.
^aRelative frequency of index ratings in each category.
^bObserved/Expected relative frequency (%) used in the calculation of the colocation quotient (CLQ).

ISSN: 1476-069X