Dependent effect sizes in MASEM: The current state of affairs
Wed-03
Presented by: Zeynep Bilici
The current meta-analytic structural equation modeling (MASEM) techniques cannot properly deal with cases where there are multiple effect sizes available for the same relationship from the same study. Existing applications either treat these effect sizes as independent, randomly select one effect size amongst many or create an average effect size. None of these approaches deal with the inherent dependency in effect sizes, and either leads to biased estimates or loss of information and power. An alternative technique is to use univariate three-level modeling in the two-stage approach to model these dependencies (Wilson et al., 2016). These different strategies for dealing with dependent effect sizes in the context of MASEM have not been previously compared in a simulation study. This study aims to compare these strategies to evaluate their performance before establishing new and better methods to tackle the problem of dependent effect sizes. We assessed the performance of these strategies across different conditions, varying the number of studies, number of dependent effect sizes within studies, sample size, the magnitude of the correlation between the dependent effect sizes and the between studies variance. We examine the relative bias in parameter estimates and standard errors, coverage proportions of confidence intervals, as well as mean standard error and power as measures of efficiency. Preliminary results suggest that there is not one method that performs well across all these criteria, pointing to the need for better methods.