logoInter-Rater Reliability Discussion Corner
by Kilem L. Gwet, Ph.D.

Back to the Inter-Rater Reliability Discussion Corner's Home Page

Calculating Intraclass Correlation with AgreeStat 2011.1
Posted : October 24, 2011

AgreeStat 2011.1 for Excel Windows provides the simplest way for researchers to compute the Intraclass Correlation Coefficient (ICC). It is a self-automated workbook containing a Visual Basic for
Applications program, and requires no installation. You simply need to have MS Excel 2007 or 2010 for Windows. The Mac version of AgreeStat 2011.1 will be released in the beginning of 2012. Download the trial version of AgreeStat 2011.1 here, and you will be surprised to see how easy and intuitive it is.

Numerous versions of the intraclass correlation coefficient are offered for the purpose of evaluating the extent of agreement among multiple raters based on interval or ratio data. Here are 3 key advantages for using AgreeStat 2011.1 to compute the intraclass correlation coefficients:

Figure 1 shows AgreeStat 2011.1's main menu, which allows to specify a 2-rater or 3-rater analysis. Although this post focuses on intraclass correlation, AgreeStat 2011.1 allows you to compute chancecorrected agreement coefficients such as Cohen's kappa, Fleiss' generalized kappa, Gwet's AC1, Scott's π, and many more, along with their standard errors.

Figure 2 shows how you would capture your data for the purpose of calculating the intraclass correlation coefficient of your choice. You would first select your workbook, then the worksheet within your workbook, before specifying the data itself.

Figure 3 shows the "Options (ICC)" form that allows you to specify the model you want to use for calculating the intraclass correlation coefficient. Depending on the model of your choice, you would calculate Shrout-Fleiss' ICC(1,1), ICC(1,k), ICC(2,1), ICC(2,k), ICC(3,1), or ICC(3,k) coefficients (see Shrout and Fleiss, 1979 for more information regarding these coefficients.

AgreeStat Main Menu
Figure 1: AgreeStat's Main Menu

Calculation of Intraclass Correlation
Figure 2: Capturing Data for the Calculation of Intraclass Correlation Coefficients

Specifying Inter-Rater Reliability Model
Figure 3: Specifying Inter-Rater Reliability Model



Back to the Inter-Rater Reliability Discussion Corner's Home Page

Please use the form shown below to communicate your comments, questions, or suggestions to me regarding this post. I will be happy to get back to you as soon as possible. Thanks. K.L. Gwet, Ph.D.

Name* . :
E-Mail* .:



Submit Your Message: