Date of this Version
Zweifel, M. (2014). The power and type I error rate of Holm's procedure when the assumptions of normality and variance homogeneity are violated (Unpublished Masters Thesis). University of Nebraska, Lincoln.
When multiple hypothesis tests are conducted on a single data set, it is necessary to control for the inflation of the Type I error rate. This is done through the use of multiple comparison procedures. Holm’s procedure is a potentially attractive multiple comparison procedure because it makes no assumptions about the data and it is simple to implement. Holm’s procedure is conducted by adjusting the p-values obtained from a prior statistical test. As a result, the power and Type I error rate of Holm’s procedure may be tied to the assumptions of the statistical test from which the p-values are obtained. In the case of making all pairwise comparisons across means, the independent samples t-test is typically used, which assumes normally distributed data and homogeneous variances across the groups being compared. The present study sought to examine how violating the assumptions of normality and variance homogeneity affected the power and Type I error rate of Holm’s procedure across several effect sizes, mean configurations and sample sizes. The results indicated that Holm’s procedure maintains the Type I error rate below α for all combinations of variance heterogeneity, nonnormality, sample size, effect size, and pattern of mean difference. As expected, the power of Holm’s procedure decreases as the sample size becomes smaller and the effect size increases. Nonnormality had a negligible effect on the power of Holm’s procedure. However, the presence of even moderate variance heterogeneity severely decreased the power of Holm’s procedure. Future research will investigate whether these results hold for situations other than testing pairwise mean differences, such as when multiple correlations are being tested. In addition, the power and Type I error rate of Holm’s procedure will be compared to alternative multiple comparison procedures.
Advisor: Rafael De Ayala