Intraclass Correlation Sample Size Determination
Confidence Interval Approach/2-Way Mixed Effects

Input Data

Assume you are in the planning stage of an inter-rater or intra-rater reliability experiment, and that the two-way mixed effects design will be adopted.  However, you do not know how many subjects, raters and replicates should be used to achieve a target confidence interval width.

AgreeStat360 can be used to determine the optimal number of subjects, raters and replicates that will satisfy the prescribed confidence interval width.  The input data needed to run this module is described in the figure below. To allow the software to suggest the most meaningful recommendations, it is essential to provide the following:

• Using the radio buttons at the top of the input form, indicate whether it is the inter-rater or the intra-rater reliability experiment that is being planned.

• The desired confidence level; 95% being the most widely-used value.

• The desired confidence interval length. Specifiying a range of values here allows AgreeStat360 to explore several possibilities around the interval width of interest.

Analysis with AgreeStat/360

To see how AgreeStat360 processes this dataset to produce various agreement coefficients, please play the video below.  This video can also be watched on youtube.com for more clarity if needed.

Results

The output that AgreeStat360 produces is shown below.  It contains a 5-column "Power Table" on the right side showing the confidence interval width associated with the number of subjects, raters and replicates in the second, third, and forth columns.

• Note that the first column of the power table shows the predicted ICC.

• You can change the range of values that the predicted ICC can take on the left side of the power table.

• Scroll down the power table until you find the combination of predicted ICC, number of subjects, number of raters, number of replicates and power that is right for you.