09:30 - 11:10
P1
Room: Terrace 2B
Panel Session 1
Kevin Munger - Temporal Validity
Lukas Linsi - Measuring the unmeasurable: the politics driving the production of "bad" statistics
William Lowe - Statistical artifacts when scaling count data in multiple dimensions
Measuring the unmeasurable: the politics driving the production of "bad" statistics
P1-02
Presented by: Lukas Linsi
Lukas Linsi, Seiki Tanaka, Francesco Giumelli
University of Groningen
International organizations produce statistics on a large range of social phenomena. Whereas some of these are relatively straightforward to measure, others do not lend themselves to easy quantification (for example because the underlying processes are too complex or deliberately hidden from state monitoring). Statistics on hard-to-measure cross-border transactions frequently represent vague guesses rather than valid, accurate measures. Nonetheless, more often than not, they are being published as if they were precise, perfectly valid measures. What is it that makes numbers attractive even when they fail to measure what they purport to measure? To shed light on this question we empirically study the politics behind the production of two distinct types of international trade statistics: "standard" merchandise trade statistics (relatively straightforward to measure) and estimates of volumes of illicit trade (difficult to measure). Building on interviews with statisticians and IO officials, an original large-scale survey experiment with a random sample of 2,000 American citizens, as well as expert surveys with economists and IO staff, we study the role of measurement (in-)accuracy in the production, use and communication of these numbers. Contrary to expectations, we find that the deceptive precision of “bad” statistics is not effective in raising greater awareness for a policy issue or signaling greater competence of the agency producing it. Instead, the phenomenon appears to be primarily the outcome of professional norms and a fetish of mathematics prominent among policy analysts trained in economics.