Submission 252
The Effect of Explicit Prior Judgment Elicitation on Belief Change
SymposiumTalk-01
Presented by: Mark Himmelstein
In standard judge advisor system (JAS) studies a judge reports a prior belief, receives advice, and then revises their initial estimate, providing a clear and tractable measure of quantitative belief revision. However, what happens if prior beliefs are not elicited? Past research has identified clear differences in posterior beliefs depending on whether priors are elicited or not, implying the mere elicitation of a prior has a treatment-like effect. However, without prior judgments, we are restricted to studying posterior judgments, rather than belief change. We propose a new method that treats judges’ priors as planned missing data and employes imputation techniques to generate estimates of those priors, and thus estimates of how their beliefs change, without ever having to ask them to directly report their priors, thereby circumventing this treatment-like effect. We first use simulation studies to demonstrate the feasibility of our method, then apply it in two advice taking experiments. In a calorie estimation task we show judges are both more willing to consider advice and weigh it more heavily when they aren’t anchored on an explicit prior. However, in a probability forecasting task, neglecting to elicit a prior induced latent confirmation bias, causing detrimental effects on judgment accuracy.