09:30 - 11:10
P1
Room:
Room: Terrace 2B
Panel Session 1
Kevin Munger - Temporal Validity
Lukas Linsi - Measuring the unmeasurable: the politics driving the production of "bad" statistics
William Lowe - Statistical artifacts when scaling count data in multiple dimensions
Temporal Validity
P1-1
Presented by: Kevin Munger
Kevin Munger
Penn State University
The ``credibility revolution" has forced social scientists to confront the limits of our methods for creating general knowledge. The current approach aims to aggregate valid but local knowledge. ``Temporal validity" is a form of external validity in which the target setting is in the future---which, of course, is always the case. Positivist social science has until recently been hamstrung with other, more immediate threats to validity and inference, but I argue that the cutting edge of non-parametric statistical approaches to the problem of external validity lay bare the inability of these approaches to inform human decision-making in, or make predictions about, the future. Using a large database of 32,000 RCTs conducted by a media firm between 2013 and 2015, I simulate the process of social science knowledge production and demonstrate that at the current margin, temporal validity is a first-order problem.