Back to Projects List
Pursuing Causal Inferences With Complex Survey Data
Research Team
Principal Investigator: Natalie Koziol
Funding Information
Funding Agency: Office of Research and Economic Development—Layman Award
Award Date: May 1, 2019
End Date: Oct 31, 2020
Abstract
Secondary analysis of large federally funded studies has garnered national and institutional attention. These studies afford strong evidence of external validity, as the samples are typically nationally representative. Unfortunately, because the studies are not randomized experiments, they lack evidence of internal validity. As a result, estimates of treatment effects may be confounded by selection bias — pre-existing differences between groups.
Methods developed to control selection bias have not been adequately tested for use with the complex sampling designs used in federal studies. Hybrid approaches that adjust for such complexities can generate inaccurate results in some contexts, so methodological studies are needed to develop and identify appropriate statistical approaches.
This project aims to advance statistical methodology for controlling selection bias in analyses of complex survey studies. The study will test the statistical validity of a novel method for controlling selection bias in complex designs, i.e., sample weighted multilevel propensity score analysis, by conducting a secondary analysis of the Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 and a Monte Carlo simulation.
The study will contribute substantively and methodologically to the social sciences by providing analytic guidance to applied researchers, and introducing advanced statistical theory underlying propensity score analysis and complex designs.