ANALYSIS OF THE IMPACT OF RECOMMENDATIONS
FROM INTELLIGENT EXPERT SYSTEMS ON
PERSONAL TRUST AND
DECISION-MAKING PERFORMANCE
Tue-03
Presented by: Peter Rötzel
(a) Background
AI is increasingly taking on a significant role in managerial decision-making (Elson et al. 2018; Guangming et al. 2021; Alufaisan et al. 2021; Wang et al. 2022). The use of AI decision support might improve the information processing of decision-makers, as Roetzel (2019) and Eppler and Mengis (2008) suggest that AI decision support increases individual's information processing capacity and, thus, increase the decision making performance.
But individuals might tend to be suspicious regarding AI decision support due unwanted learning behavior such as the Amazon-HR case (Dastin 2018) or inexplicable recommendations (Glikson and Williams Woolley 2020; Gastounioti and Kantos 2020). Initial trust is given high importance in many previous studies to induce willingness to use a technology (e.g., McKnight et al. 2002). Early research on the topic assumed that at stages when knowledge is still low, trust is mainly determined by the perceived predictability of a technology (Muir 1987; Moray and Inagaki 1999; Lee et al. 2021). Previous studies fail to address the effect of deteriorating or improving AI performance on individual’s trust. The consideration of changing AI performance is very important, as the listed examples show that modern and future AI will be able to change, to interact with people or the environment and thus develop worse or better performance.
(b)Objectives
This study examines how human decision-making performance is influenced by AI performance. It raises the question of how an individual’s trust in AI decision support develops in a multi-round lab experiment, how cooperation behavior changes over time, especially with the decreasing quality of AI recommendations and whether findings from creativity research can have an influence on the construct.
(c)Hypothesis and research question
This study examines how changing AI performance – deteriorating or improving – affects individual’s trust in AI decision support, decision-making performance, and individual’s confidence in AI.
H1: AI performance influences individual’s performance.
H2: AI performance influences individual’s confidence in AI.
H3: Trust influences the individual’s performance.
(d)Method
Two studies were conducted. The purpose of the preliminary study is to test and evaluate the experimental material and the measuring instruments and then to improve the design of the main study. The main study involves a lab experiment with eight different groups.
(e) Results/findings
We find that AI recommendation increases individual’s performance compared to the control group’s performance without AI recommendation. Surprisingly, we find that even a deteriorating AI increases individual’s performance but for the cost of trust. The performance level of individuals facing a deteriorating AI is significantly higher than the performance level of individuals without AI recommendation.
(f) Conclusions and implications (expected)
Trust in the AI recommendation is a basic prerequisite for improving the performance of decision-makers. Decision-makers will use AI-generated decision recommendations if the performance of the AI is sufficient to avoid mistrust. We also find that deteriorating AI recommendation increases the individual’s performance but reduces performance. Drawing on cognitive load theory, the insufficient support of AI recommendation might drive the allocated cognitive resources of decision-makers (Roetzel 2019, Paas and van Merriënboer 2020).
AI is increasingly taking on a significant role in managerial decision-making (Elson et al. 2018; Guangming et al. 2021; Alufaisan et al. 2021; Wang et al. 2022). The use of AI decision support might improve the information processing of decision-makers, as Roetzel (2019) and Eppler and Mengis (2008) suggest that AI decision support increases individual's information processing capacity and, thus, increase the decision making performance.
But individuals might tend to be suspicious regarding AI decision support due unwanted learning behavior such as the Amazon-HR case (Dastin 2018) or inexplicable recommendations (Glikson and Williams Woolley 2020; Gastounioti and Kantos 2020). Initial trust is given high importance in many previous studies to induce willingness to use a technology (e.g., McKnight et al. 2002). Early research on the topic assumed that at stages when knowledge is still low, trust is mainly determined by the perceived predictability of a technology (Muir 1987; Moray and Inagaki 1999; Lee et al. 2021). Previous studies fail to address the effect of deteriorating or improving AI performance on individual’s trust. The consideration of changing AI performance is very important, as the listed examples show that modern and future AI will be able to change, to interact with people or the environment and thus develop worse or better performance.
(b)Objectives
This study examines how human decision-making performance is influenced by AI performance. It raises the question of how an individual’s trust in AI decision support develops in a multi-round lab experiment, how cooperation behavior changes over time, especially with the decreasing quality of AI recommendations and whether findings from creativity research can have an influence on the construct.
(c)Hypothesis and research question
This study examines how changing AI performance – deteriorating or improving – affects individual’s trust in AI decision support, decision-making performance, and individual’s confidence in AI.
H1: AI performance influences individual’s performance.
H2: AI performance influences individual’s confidence in AI.
H3: Trust influences the individual’s performance.
(d)Method
Two studies were conducted. The purpose of the preliminary study is to test and evaluate the experimental material and the measuring instruments and then to improve the design of the main study. The main study involves a lab experiment with eight different groups.
(e) Results/findings
We find that AI recommendation increases individual’s performance compared to the control group’s performance without AI recommendation. Surprisingly, we find that even a deteriorating AI increases individual’s performance but for the cost of trust. The performance level of individuals facing a deteriorating AI is significantly higher than the performance level of individuals without AI recommendation.
(f) Conclusions and implications (expected)
Trust in the AI recommendation is a basic prerequisite for improving the performance of decision-makers. Decision-makers will use AI-generated decision recommendations if the performance of the AI is sufficient to avoid mistrust. We also find that deteriorating AI recommendation increases the individual’s performance but reduces performance. Drawing on cognitive load theory, the insufficient support of AI recommendation might drive the allocated cognitive resources of decision-makers (Roetzel 2019, Paas and van Merriënboer 2020).