Off-campus UNL users: To download campus access dissertations, please use the following link to log into our proxy server with your NU ID and password. When you are done browsing please remember to return to this page and log out.
Non-UNL users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
The effects of simplifying assumptions in power analysis
Abstract
In experimental research, planning studies that have sufficient probability of detecting important effects is critical. Carrying out an experiment with an inadequate sample size may result in the inability to observe the effect of interest, wasting the resources spent on an experiment. Collecting more participants than is necessary may unnecessarily put too many participants at risk and potentially detect an effect size that is not clinically meaningful. Therefore, obtaining the most accurate estimates of necessary sample size prior to applying for research funding and carrying out experiments is of utmost importance. Educational studies often select whole classrooms for participation. When clusters of individuals are assigned to experimental conditions, the design is referred to as a cluster randomized clinical trial. An over-estimation or under-estimation of the number of clusters needed can have large budgetary or logistical consequences. Accurate sample size estimates, especially in large-scale studies, can help researchers carry out high-quality research and make valid conclusions. Statistical power is the probability of rejecting a false null hypothesis so that a researcher can correctly conclude that an effect has been observed in a study. Three different methods of estimating power are examined in this study, including (a) formulaic power functions, (b) empirical simulation studies, and (c) constructing exemplary datasets. Formula-based methods make assumptions that may not be met in practice. This study assessed (a) the extent to which failure to account for practical data conditions such as attrition, unequal treatment group sizes, and unequal cluster sizes bias estimates of anticipated power and sample size; and (b) if these effects were moderated by the amount of variability that is attributable to between-cluster differences. The empirical simulation study and exemplary dataset methods showed that attrition and unequal treatment group sizes have substantial effects on estimates of power and sample size. Unequal cluster sizes did not appreciably affect power. Higher levels of bias were found when less variability was attributable to between-cluster differences. Power analyses based on a formulaic approach that fail to account for attrition or treatment group imbalance may severely overestimate power and underestimate the sample size necessary to observe important effects.
Subject Area
Educational psychology|Quantitative psychology
Recommended Citation
Kupzyk, Kevin A, "The effects of simplifying assumptions in power analysis" (2011). ETD collection for University of Nebraska-Lincoln. AAI3450211.
https://digitalcommons.unl.edu/dissertations/AAI3450211