by

Kilem Li Gwet, PhD

I wrote this book, primarily to assist researchers and students with the calculation of various inter-rater reliability coefficients using the SAS system. For categorical ratings, you will learn how to use SAS for computing various chance-corrected agreement coe cients (CAC) such as Cohen's kappa, Gwet's $\mbox{AC}_1$/$\mbox{AC}_2$, Krippendorff's alpha and many others. For quantitative ratings, you will learn how SAS can be used to compute different variants of the Intraclass Correlation Coefficient (ICC).

**Preliminaries**: Preliminaries include the preface, the table of contents**Chapter 1**: The SAS Solution and Its Problems**Chapter 2**: Analysis of Categorical Ratings from 2 Raters**Chapter 3**: Analysis of Categorical Ratings for Multiple Raters**Chapter 4**: Analysis of Quantitative Ratings**Chapter 5**: Analysis Techniques for Categorical Ratings

**SAS PROGRAMS FOR DOWNLOAD**

*Chance-corrected Agreement Coefficients (CAC)*

CAC SAS/IML Librairies

- agreecoeff2.sas - analysis of contingency tables / no missing rating -
- agreecoeff3raw.sas - analysis of raw ratings -
- agreecoeff3dist.sas - analysis of distribution of raters by subject & categories -
- benchmarking.sas - Benchmarking agreement coefficients -
- pairedttest.sas - Paired t-test for testing the difference of agreement coefficients for statistical significance
- weights.sas - Weight functions for generating various predefined sets of weights -

CAC Programs in Chapters 2 & 3

- prg2contingency.sas: Basic
SAS program for computing the kappa coefficient based on a

contingency table. - prg2raw.sas: Basic SAS program for computing the kappa coefficient based on raw scores.
- prg2xtab3cats.sas: SAS program that fixes the diagonal and imbalance problems.
- prg2raw2r.sas: SAS program that xes the diagonal and imbalance problems.
- prg2freq.sas: The FREQ procedure with an unbalanced rating table.
- prg2xtabiml.sas: SAS program that computes alternative agreement coefficients from a contingency table, using SAS/IML functions.
- prg3raw.sas: Using SAS/IML functions in agreecoeff3raw.sas for performing an unweighted analysis of raw ratings.
- prg3dist.sas: Using SAS/IML functions in agreecoeff3dist.sas for analyzing raw data
- prg3wted.sas: Using SAS/IML functions in agreecoeff3raw.sas for computing weighted agreement coefficients based on ordinal weights.
- prg3missfreq.sas: Using the MISSING option in the TABLES Statement of the FREQ procedure for handling missing ratings.
- prg3miss2raters.sas: Calculating several agreement coefficients based on 2 raters with missing ratings - An alternative to the FREQ procedure.
- prg3magree.sas: SAS program that
computes Fleiss' generalized kappa and

Gwet's AC1 using the MAGREE.SAS SAS macro. - prg3raw.sas: Using SAS/IML functions in agreecoeff3raw.sas for performing an unweighted analysis of raw ratings.
- prg3raw10x4.sas: Using SAS/IML functions in agreecoeff3raw.sas for performing an unweighted analysis of Table 3.1 ratings.
- prg5pttest.sas: SAS program for testing the difference between two agreement coefficients for statistical significance
- prg5benchmark.sas: SAS program
for computing cumulative membership probabili-

ties to benchmark agreement coefficients.

*Intraclass Correlation Coefficients (ICC)*

1) ICC SAS/IML Librairies

- icc1factor.sas - Intraclass Correlation Coefficients (ICC) for one-factor models.
- icc2x3.sas - Intraclass Correlation Coefficients (ICC) for two-factor models.

2) ICC Programs in Chapter 4

- prg4shrout.sas: Using the INTRACC.SAS SAS macro and Table 2 of Shrout & Fleiss (1979), for computing intraclass correlation coefficients.
- prg4icc1.sas: Calculating Intraclass Correlation Coefficients (ICC) under the one-factor ANOVA model
- prg4icc2.sas: Calculating
Intraclass Correlation Coe cients (ICC) under the

two-way ANOVA model with random effects (Model 2). - prg4icc3.sas: Calculating
Intraclass Correlation Coe cients (ICC) under the

two-way ANOVA model with mixed effects (Model 3). - prg4finn.sas: SAS program for calculating Finn's agreement coefficient and the associated p-value.