`Identify the Expert': an Experimental Study in Economic Advice
We ask whether laypeople are able to distinguish the opinions of experts from those of other laypeople. First, we distributed an economics questionnaire to randomly chosen academic economists in Europe and the US. We select questions with at least 70% agreement as those on which there is sufficient consensus. Then, we administer a two-stage experiment with participants from the general population in England and Wales to test our pre-registered hypotheses. In Stage 1 of the Treatment 1 (the Baseline), participants provide their own answer and then see the suggested answers of two computerized advisors on the same questions. One advisor is the 'Expert', who is designed to always give the answer deemed correct by the academic consensus. The other advisor is the 'Populist' who always proposes the most popular answer from a pilot study run with participants from the same pool. Participants do not know which advisor is the Expert. They see in words the recommendations which the two advisors make and they know that one of them is always correct. Furthermore, they receive no other information that could operate as a cue, such as credentials, visual characteristics or exhibited confidence. Subsequently, in Stage 2, after observing a summary of the recommendations by both advisors, participants are asked to pick one advisor to answer all of the questions on their behalf. They are financially incentivized to answer correctly in Stage 1 and to select the Expert in Stage 2.
We administer two additional treatments which only differ from the Baseline in Stage 2. In Treatment 2, participants learn in addition how many questions they answered correctly in Stage 1. This treatment intends to correct mis-perceptions of performance and address participants' overconfidence. In Stage 2 of Treatment 3 each participant sees the summary table of answers and the two advisors' recommendations referring to another participant from a prior experimental session. They also see how many correct answers that third participant had in Stage 1. This treatment intends to additionally address ego-involvement and the inability to admit mistakes. We develop a theoretical model which shows that Treatments 2 and 3 provide sufficient information for paricipants to always be able to identify the Expert.
in all three treatments, the percentage of participants who chose the Expert in Stage 2 is significantly below 50%. In Treatments 2 and 3 performance improves only marginally in comparison to the Baseline, and it does not exceed 50% in either treatment. Overconfidence seems to play a role, since in Treatment 2, where participants receive feedback on their performance in Stage 1, they perform significantly better than in Treatment 1. Ego-involvement does not seem to play a role: in Treatment 3 participants performed no better than in Treatment 2. We test for several pre-registered demographic characteristics but find no consistent effects. In exploratory analyses we find that addressing overconfidence helps particularly those who had strictly more answers in common with the Populist, hence they faced the hardest problem. Our results are robust to a number of pre-registered robustness checks.
We administer two additional treatments which only differ from the Baseline in Stage 2. In Treatment 2, participants learn in addition how many questions they answered correctly in Stage 1. This treatment intends to correct mis-perceptions of performance and address participants' overconfidence. In Stage 2 of Treatment 3 each participant sees the summary table of answers and the two advisors' recommendations referring to another participant from a prior experimental session. They also see how many correct answers that third participant had in Stage 1. This treatment intends to additionally address ego-involvement and the inability to admit mistakes. We develop a theoretical model which shows that Treatments 2 and 3 provide sufficient information for paricipants to always be able to identify the Expert.
in all three treatments, the percentage of participants who chose the Expert in Stage 2 is significantly below 50%. In Treatments 2 and 3 performance improves only marginally in comparison to the Baseline, and it does not exceed 50% in either treatment. Overconfidence seems to play a role, since in Treatment 2, where participants receive feedback on their performance in Stage 1, they perform significantly better than in Treatment 1. Ego-involvement does not seem to play a role: in Treatment 3 participants performed no better than in Treatment 2. We test for several pre-registered demographic characteristics but find no consistent effects. In exploratory analyses we find that addressing overconfidence helps particularly those who had strictly more answers in common with the Populist, hence they faced the hardest problem. Our results are robust to a number of pre-registered robustness checks.