As demand increases for employees with expertise in science, technology, engineering and mathematics, representation of Latinos in STEM fields continues to lag.
As of 2011, Latinos represented only 7 percent of the American STEM workforce, despite accounting for more than 16 percent of the U.S. population.
Natalie Koziol, CYFS research assistant professor, and Marc Goodrich, assistant professor of special education and communication disorders, are exploring whether the STEM training gap is due in part to challenges in measuring the academic achievement of language-minority students.
They are taking a fresh look at an old challenge: how to test for differential item functioning (DIF) — the extent to which items differentially discriminate among different groups — and how to better control for such bias.
Traditional methods for evaluating DIF do not control for potentially important pre-test differences between groups, which may confound results.
Researchers are examining a restricted-use dataset from the “Early Childhood Longitudinal Study, Kindergarten Class of 1998-99.” Their focus is kindergartners’ mathematics achievement, where previous research found Latino students have significantly lower performance than their white peers in both the fourth and eighth grades. The gap is even wider for students identified as English language learners.
Koziol and Goodrich are working to operationalize a new approach for investigating DIF. Once it is ready, they will apply the approach to evaluate DIF of English vs. Spanish-translated kindergarten math items within a subsample of Spanish-speaking students.
You don’t want students’ math proficiency to be confounded by a language barrier. We need to be able to measure the achievement and skills of all students, not just those who speak English proficiently.”
Natalie Koziol, CYFS research assistant professor
The goal is to learn whether Spanish-translated math items function differently for Spanish-speaking kindergartners after accounting for potential selection bias.
Their design is an evaluation that draws on a pre-test measure to assign individuals to treatment groups. In this case, an English language screener was used to determine whether students should receive an English or Spanish-translated math test.
Their design makes it possible to estimate causal effects in environments where randomization is impractical.
Following the empirical application, Koziol and Goodrich will conduct a simulation that involves generating data for a hypothetical known population, then evaluating how well the newly proposed and traditional approaches correctly identify the DIF items. They will then be able to determine which approach is most effective.
The challenge, Koziol said, is to ensure the English and Spanish items measure the same construct.
“If we have two students with the same math ability, with one student taking a test in English and the other taking the Spanish version, they should have the same probability of answering correctly,” she said. “If they don’t, that’s indicative of differential item functioning — the items don’t function the same in both languages.”
Once differently functioning items are identified, adjustments can be made. For example, English-to-Spanish translations of math word problems or instructions can be modified, or avoided altogether.
“You don’t want students’ math proficiency to be confounded by a language barrier,” Koziol said. “We need to be able to measure the achievement and skills of all students, not just those who speak English proficiently.”
The project is funded by a grant from the American Educational Research Association.