agree AgreeStat Analytics

Intraclass Correlation Sample Size Determination
Retrospective Power Analysis/2-Way Random Effects Model

Problem

Assume you have already conducted an inter-rater or intra-rater reliability experiment following the two-way random effects design.  Now, you want to know what statistical power you have been able to achieve.

AgreeStat360 can be used to determine the achieved statistical power for various values of the Minimum Detectable Difference (MDD).  The input data needed to run AgreeStat360 is described in the figure below. Essentially you need to provide the following:

  • Using the radio buttons at the top of the input form, indicate whether it is the inter-rater or the intra-rater reliability experiment that is being planned.

  • The standard significance level is 5% and needs not be modified. The baseline ICC value could be modified and represents the value the researcher aims at exceeding.

  • The number of subjects, raters and replicates (i.e. number of ratings per subject and by rater) used in the experiment must be specified as well.

intraclass correlation sample size calculation with AgreeStat360

Analysis with AgreeStat/360

To see how AgreeStat360 processes this dataset to produce various agreement coefficients, please play the video below.  This video can also be watched on youtube.com for more clarity if needed.

Results

The output that AgreeStat360 produces is shown below.

  • The power table on the right hand side shows how the power changes as the MDD changes given the parameters specified previously.

  • You may also modify the MDD range values of interest and the power table will update accordingly.

intraclass correlation sample size calculation with AgreeStat360