6.5.10. Kappa Test for InterObserver Variation
This version will calculate a test statistic to measure the degree of agreement between two raters. The present implementation is the original form of Kappa test as introduced by Cohen, J. A. (1960) and Cohen, J. A. (1968), and also takes into account the correction in the calculation of large sample variances by Fleiss, J. L., Cohen, J., Everitt, B. S., (1969). A weighted Kappa is computed if weights are given and an unweighted Kappa otherwise.
The data for this test must be in the form of a square matrix. Columns of the matrix can be selected from the Variables Available list by clicking on [Variable]. If the number of rows is not equal to the number of columns the test cannot be performed. Missing values are not allowed.
If a weighted analysis is to be performed weights should also be arranged in the form of a square matrix with the same dimensions. Columns of the weights matrix can be selected from the Variables Available list by clicking on [Weight].
Example
The data is taken from Fleiss, J. L., Cohen, J., Everitt, B. S., (1969).
Open NONPARM2, select Statistics 1 → Nonparametric Tests (Multisample) → Kappa Test InterObserver Variation and select Diagnosis 1, Diagnosis 2 and Diagnosis 3 (C31 to C33) as [Variable]s to obtain the following results:
Kappa Test (InterObserver Variation)
Variables Selected: Diagnosis1, Diagnosis2, Diagnosis3
Expected Proportion = 
0.7000 
Observed Proportion = 
0.4750 

Value 
Standard Error 
ZStatistic 
1Tail Probability 
Kappa 
0.4286 
0.0537 
7.9792 
0.0000 

2Tail Probability 
Lower 95% 
Upper 95% 
Kappa 
0.0000 
0.3233 
0.5338 
Keeping the Diagnosis columns as [Variable]s, select Weight 1, Weight 2 and Weight 3 (C34 to C36) as [Weight]s:
Kappa Test (InterObserver Variation)
Variables Selected: Diagnosis1, Diagnosis2, Diagnosis3
Weights: Weight1, Weight2, Weight3
Expected Proportion = 
0.5672 
Observed Proportion = 
0.7867 

Value 
Standard Error 
ZStatistic 
1Tail Probability 
Weighted Kappa 
0.5071 
0.0570 
8.8967 
0.0000 

2Tail Probability 
Lower 95% 
Upper 95% 
Weighted Kappa 
0.0000 
0.3954 
0.6188 