Education and Human Sciences, College of (CEHS)
Date of this Version
12-2014
Citation
Zweifel, M. (2014). The power and type I error rate of Holm's procedure when the assumptions of normality and variance homogeneity are violated (Unpublished Masters Thesis). University of Nebraska, Lincoln.
Abstract
When multiple hypothesis tests are conducted on a single data set, it is necessary to control for the inflation of the Type I error rate. This is done through the use of multiple comparison procedures. Holm’s procedure is a potentially attractive multiple comparison procedure because it makes no assumptions about the data and it is simple to implement. Holm’s procedure is conducted by adjusting the p-values obtained from a prior statistical test. As a result, the power and Type I error rate of Holm’s procedure may be tied to the assumptions of the statistical test from which the p-values are obtained. In the case of making all pairwise comparisons across means, the independent samples t-test is typically used, which assumes normally distributed data and homogeneous variances across the groups being compared. The present study sought to examine how violating the assumptions of normality and variance homogeneity affected the power and Type I error rate of Holm’s procedure across several effect sizes, mean configurations and sample sizes. The results indicated that Holm’s procedure maintains the Type I error rate below α for all combinations of variance heterogeneity, nonnormality, sample size, effect size, and pattern of mean difference. As expected, the power of Holm’s procedure decreases as the sample size becomes smaller and the effect size increases. Nonnormality had a negligible effect on the power of Holm’s procedure. However, the presence of even moderate variance heterogeneity severely decreased the power of Holm’s procedure. Future research will investigate whether these results hold for situations other than testing pairwise mean differences, such as when multiple correlations are being tested. In addition, the power and Type I error rate of Holm’s procedure will be compared to alternative multiple comparison procedures.
Advisor: Rafael De Ayala
Comments
A Thesis Presented to the Faculty of The Graduate College at the University of Nebraska In Partial Fulfillment of the Requirements For the Degree of Master of Arts, Major: Educational Psychology, Under the Supervision of Professor Rafael De Ayala. Lincoln, Nebraska: December, 2014
Copyright (c) 2014 Michael Zweifel