09:30 - 11:10
P1-S20
Room: 1A.04
Chair/s:
Heiner Janus
Discussant/s:
Alicia Noëllie Saes-Louarn
Measuring Scientific Reasoning for Public Policy
P1-S20-3
Presented by: Simon Munzert
Sebastian Ramirez-RuizSimon Munzert
Hertie School
Scientific reasoning is a critical component in shaping effective public policy, enabling both policymakers and the public to make informed decisions on complex issues. However, existing measures often fail to meet the specific methodological and applied demands of public policy contexts. This paper addresses this gap by developing a comprehensive measurement tool to assess scientific literacy in relation to public policy. The tool integrates knowledge of statistical reasoning, data literacy, causal inference, and the scientific method, alongside the ability to critically evaluate scientific information pertinent to policy debates. To ensure flexibility for researchers working under practical constraints such as survey time, we incorporate Item Response Theory (IRT) scaling to develop an adaptive inventory. This battery dynamically optimizes learning rates while adhering to survey length constraints, offering an efficient yet rigorous solution for data collection. Validation of the tool is undertaken using a diverse sample of policy professionals and lay respondents (n ≈ 500), enabling robust assessment across varied competency levels and demographic contexts. Preliminary findings indicate strong internal consistency and construct validity, with the tool effectively distinguishing between conceptual knowledge and applied competencies. Results also suggest that the adaptive format can enhance respondent engagement without compromising measurement precision. These findings have significant implications for public policy research, providing a scalable method to gauge scientific literacy and decision-making capacity amidst evolving scientific issues.
Keywords: Scientific reasoning, Public policy, Computer adaptive testing (CAT), Survey methodology

Sponsors