SRA-E Lisbon 2017

Survey methodology is often used when risk analysis depends on preferences and behaviors that cannot be measured directly by independent observation. These situations arise frequently in environmental and health disputes for which reliable risk estimates are needed. Even well-designed surveys can produce unreliable results when the preferences or behaviors of interest are illicit or poorly articulated by respondents. In such cases, the risk analyst is challenged to find corroborating evidence to support or refute survey results. This presentation reviews examples of survey applications in recent disputes over water resource allocation, tobacco usage, and product-labeling claims to illustrate the potential and limitations of frequently used survey designs to collect stated preferences and self-reported behaviors. The examples serve to highlight different issues raised by consideration of the standard two categories of survey error: measurement error (concerning the data of interest), and representation error (concerning the population of interest). In each example, the reliability of the survey results was susceptible to serious criticism based on conflicts with independent measures of behavior or economic rationality. The discussion concludes with some suggestions for improving survey design and better aligning the methodology with the demands of the analysis.