Is a distrust instruction helpful when interacting with a fallible AI?
Mon—Casino_1.811—Poster1—2106
Presented by: Tobias Peters
One use case of Artificial Intelligence (AI) is to support humans in their decision-making, for example in the context of medical image classification. Often a combination of an expert’s knowledgeable perception and an AI-generated classification as advice are required. Another requirement is that the AI’s suggestions are explainable in order to ensure humans’ understanding and to foster their trust. Given that AI models can err, we argue that the possibility to critically review, thus to distrust, an AI decision and its explanation is an equally interesting target of research.
We studied task performance, trust, and distrust in an image classification experiment in which the participants were supported by a mock-up AI. The quality of the AI’s classifications decreased for a phase of the experiments. We tested whether the instruction to remain sceptical and to critically review the AI advice leads to a better decision performance, compared to a control condition with neutral instructions. To improve generalisability, we ran two parallel studies, using one set of stimuli, which participants encounter regularly, and one set of abstract stimuli that the participants needed to be familiarized with.
A Bayesian Signal Detection Theory analysis shows the expected difference in the participants’ performance for the abstract stimulus set, but not for the familiar stimuli, where a tendency in the opposite direction is observed. Repeated single-item self-report of trust and distrust shows an increase in trust and a decrease in distrust after the drop of the AI’s classification quality, with no difference between conditions.
We studied task performance, trust, and distrust in an image classification experiment in which the participants were supported by a mock-up AI. The quality of the AI’s classifications decreased for a phase of the experiments. We tested whether the instruction to remain sceptical and to critically review the AI advice leads to a better decision performance, compared to a control condition with neutral instructions. To improve generalisability, we ran two parallel studies, using one set of stimuli, which participants encounter regularly, and one set of abstract stimuli that the participants needed to be familiarized with.
A Bayesian Signal Detection Theory analysis shows the expected difference in the participants’ performance for the abstract stimulus set, but not for the familiar stimuli, where a tendency in the opposite direction is observed. Repeated single-item self-report of trust and distrust shows an increase in trust and a decrease in distrust after the drop of the AI’s classification quality, with no difference between conditions.
Keywords: Human-AI interaction, Distrust, Image Classification, Signal Detection Theory, Appropriate Trust, Explainable AI