Education and Human Sciences, College of (CEHS)
First Advisor
James A. Bovaird
Committee Members
Jordan M. Wheeler, Rafael J. de Ayala, Calvin P. Garbin
Date of this Version
11-2024
Document Type
Dissertation
Citation
A dissertation presented to the faculty of the Graduate College at the University of Nebraska in partial fulfillment of requirements for the degree of Doctor of Philosophy
Major: Psychological Studies in Education (Quantitative, Qualitative, and Psychometric Methods)
Under the supervision of Professor James A. Bovaird
Lincoln, Nebraska, November 2024
Abstract
Researchers understand that conducting numerous pairwise comparisons between group means increases the Type I error rate, prompting the use of planned contrasts like orthogonal contrast sets. Implicit to orthogonal contrast sets is the principal assumption that groups are balanced in size. Further, when dealing with complex variables like latent constructs, specialized modeling is necessary. Understanding how violations of the assumptions of an orthogonal contrast set and the latent variable model approach can help identify variation in the accuracy of group mean difference. However, hindrances to the accuracy of latent mean orthogonal comparisons when this assumption is untenable remains unsettled. This study's purpose was to identify how diverging proportions of subgroup sample sizes and two approaches to measurement modeling influence the accuracy of latent mean group difference estimates. Monte Carlo simulations generated data based on a common factor model, manipulating overall sample size, group size imbalance, and population latent mean difference. Two structural equation modeling approaches, MIMIC and re-parameterized MGCFA models, estimated latent group mean differences using orthogonal contrasts. Estimate accuracy was assessed using statistical power, Type I error, bias, and parameter efficiency measures. The analysis showed that as sample sizes became more imbalanced (e.g., group 1 = 90%, group 2 = 10%), measures of accuracy declined, with some estimates falling below acceptable thresholds for power, Type I error, and bias. This effect was most pronounced in small samples (N = 240), where latent mean differences were highly inaccurate in extremely unbalanced groups. The MIMIC approach consistently outperformed the MGCFA model in parameter accuracy; however, when the MGCFA model was re-specified to mirror the MIMIC model’s underlying assumptions, both methods produced nearly equivalent estimates of latent mean differences. The findings suggest that researchers using orthogonal contrasts to compare groups on a unidimensional latent variable continuum should (a) be aware of examined group’s sample size proportions and the impact of group size inequalities on estimate accuracy, and (b) carefully consider the costs and benefits of the chosen latent variable modeling approach, including how each model addresses measurement non-invariance.
Advisor: James A. Bovaird
Comments
Copyright 2024, Jay B. Jefries. Used by permission