Research Projects

Back to Projects List

Pursuing Causal Inferences With Complex Survey Data

Research Team


Principal Investigator: Natalie Koziol

Funding Information

Funding Agency: Office of Research and Economic Development—Layman Award

Award Date: May 1, 2019

End Date: Oct 31, 2020


Secondary analysis of large federally funded studies has garnered national and institutional attention. These studies afford strong evidence of external validity, as the samples are typically nationally representative. Unfortunately, because the studies are not randomized experiments, they lack evidence of internal validity. As a result, estimates of treatment effects may be confounded by selection bias — pre-existing differences between groups.

Methods developed to control selection bias have not been adequately tested for use with the complex sampling designs used in federal studies. Hybrid approaches that adjust for such complexities can generate inaccurate results in some contexts, so methodological studies are needed to develop and identify appropriate statistical approaches. 
This project aims to advance statistical methodology for controlling selection bias in analyses of complex survey studies. The study will test the statistical validity of a novel method for controlling selection bias in complex designs, i.e., sample weighted multilevel propensity score analysis, by conducting a secondary analysis of the Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 and a Monte Carlo simulation.

The study will contribute substantively and methodologically to the social sciences by providing analytic guidance to applied researchers, and introducing advanced statistical theory underlying propensity score analysis and complex designs.

Research, Measurement and Evaluation Methods