The concept of algorithm aversion was first introduced in 2015 by Dietvorst et al. [1] named. The authors conducted a laboratory experiment in which subjects were asked to assess the likelihood of success of applicants for an MBA programme. Before the test subjects finalised their own predictions, some were given access to the predictions of a human expert, an algorithmic agent or the predictions of both. The experiment shows: In particular, when people can observe the algorithm at work and therefore also when it makes a mistake or two, they tended to rely more on human expertise – even though the algorithmic agent displayed significantly better forecasting performance overall than the human expert. These findings laid the foundation for the first definition of the phenomenon of algorithm aversion.
While these early definitions of algorithm aversion were mostly related to the (incorrect) behaviour of an algorithmic agent, further studies showed that humans do not necessarily have to witness the performance of this agent in order to be averse to it [2]. This further emphasises the inherent irrationality of this phenomenon. This quickly led to a broader definition of algorithm aversion, which is generally used in research today. According to this definition, algorithm aversion describes a “biased evaluation of an algorithm that manifests itself in negative behaviour and attitudes towards the algorithm in comparison to a human agent” [3, p. 4]. In this broader definition, algorithm aversion does not only refer to the area of predictions, but can also be reflected in the evaluation of the algorithm or its results as well as in the general use of an algorithmic agent. An example: If an algorithmic agent makes a mistake in a personnel or lending decision, the associated negative emotions are more pronounced than when a human colleague makes a mistake [4].
Research has already demonstrated the phenomenon of algorithm aversion in many different fields of application. An irrational aversion to algorithms can be seen, for example, in medicine, where algorithmic systems are used as decision support or even decision makers in diagnostics [2]. Aversion is also evident in the delegation of business decisions to algorithmic agents, for example with regard to making personnel decisions, such as invitations to job interviews [4]. Algorithm aversion can also manifest itself in the area of algorithmically created products, such as algorithmically created music or art, or towards algorithmic agents in the area of customer service, such as chatbots [5]. In particular, the creation of digital products by algorithmic systems stands out as having significant efficiency potential, as these systems can take over the entire value creation and sales process.
The increasing diffusion of algorithmic systems into the digital economy and the associated efficiency potential for companies means that algorithm aversion can be firmly anchored there. However, it is also important to emphasise that algorithm aversion also occurs outside of an economic framework, for example in the context of legal decisions [3]. Nevertheless, the use of algorithmic systems in the digital economy is extremely important and will continue to increase significantly in the future, making algorithm aversion particularly relevant in this area. It is also interesting to note that an aversion to algorithms cannot be recognised in all cases and the effect can even be the opposite: For example, some studies describe people preferring algorithmic agents over other people [6]. This is called algorithm appreciation. Algorithm appreciation usually occurs when the underlying task is non-judgemental in nature. This is based on the fact that humans regard machines as particularly capable for objective tasks, but less so for subjective tasks [7]. Overall, there are currently indications that the underlying circumstances (e.g. characteristics of the algorithmic system or the user) have a significant influence on the development of algorithm aversion [3]. Nevertheless, research is still ongoing to better understand the development of algorithm aversion and the influences on it. For example, a recent study suggests that algorithm aversion does not arise automatically, but is rather based on a (rational!) cost-benefit analysis [8].
Comparability with analogue phenomena
Information systems and related disciplines have long been concerned with the acceptance of new digital solutions. Numerous studies have been conducted in the field of technology acceptance. These studies are based on a singular, i.e. stand-alone consideration of a solution. Research on algorithm aversion deliberately takes a different approach by focussing on comparative scenarios (i.e. human vs. algorithm) [3]. This comparative approach emerged against the backdrop of the increasing diffusion of artificial intelligence (AI), which allows systems to take over tasks from humans much more than before. Studies on algorithm aversion therefore always focus on the trade-off between a human and an algorithmic agent, as they compare how the interaction of a person with a human differs from the interaction with algorithms.
Since a digital solution is a mandatory part of the phenomenon and no comparable analogue phenomenon can be observed, the digital specificity of the phenomenon must be classified as very high. In addition, only their digital, intelligent capabilities allow the systems to take over tasks previously performed by humans in the first place. The need for intelligence in algorithmic systems thus fundamentally distinguishes this phenomenon from analogue phenomena. The enabler of the phenomenon of algorithm aversion is therefore also the automation and autonomisation of processes, as algorithm aversion only occurs when humans witness algorithmic systems taking over tasks (both of an advisory or executive nature).
Social relevance
Society has often been confronted with algorithmic systems in recent decades, which means that the phenomenon of algorithm aversion has been known for some time.
Current technological progress makes algorithm aversion particularly relevant for society. In the past, society was mostly confronted with algorithmic systems that made passive recommendations. For example, research into algorithm aversion has often focussed on the fields of medicine (e.g. decision support for diagnostics) or finance (e.g. decision support for lending). However, the relevance of algorithm aversion for society as a whole will continue to increase as technological progress continues apace: In perspective, especially against the backdrop of current technological developments in the field of non-linear machine learning models, more and also increasingly performative systems will enter our everyday lives. These advanced systems can perform tasks independently thanks to their higher degree of autonomy. This means that people not only use these systems, but also delegate tasks to them. Examples of this include voice assistants in customer service or algorithmic agents that carry out medical triage without human intervention. In the future, we will therefore be increasingly confronted with algorithmic agents, with many other fields of application opening up as technology advances (e.g. the creation of journalistic articles or the design of fashion).
Higher degrees of autonomy and thus the ability of algorithms to act can deepen the aversion to these algorithmic agents [2], [3]. The extent and frequency of the occurrence of algorithm aversion is therefore also likely to increase. Whether algorithm aversion as a social phenomenon can be reduced over time with increasing confrontation with these systems is an important field for future research.
Further links and literature
Sources
- Dietvorst, B.J./Simmons, J.P./Massey, C. (2015). Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them Err. In: Journal of Experimental Psychology: General 144(1),114–126.
- Longoni, C./Bonezzi, A./Morewedge, C.K. (2019). Resistance to Medical Artificial Intelligence. In: Journal of Consumer Research (46(4), pp. 629-650.
- Jussupow, E./Benbasat, I./Heinzl, A. (2020). Why Are We Averse Towards Algorithms? A Comprehensive Literature Review on Algorithm Aversion. Proceedings of the 28th European Conference on Information Systems, A Virtual AIS Conference.
- Renier, L.A./Mast, M.S./Bekbergenova, A. (2021). To Err Is Human, Not Algorithmic – Robust Reactions to Erring Algorithms. In: Computers in Human Behavior 124, 1–12.
- Jago, A.S. (2019). Algorithms and Authenticity. In: Academy of Management Discoveries 5(1), 38–56.
- Logg, J.M./Minson, J.A./Moore, D.A. (2019). Algorithm Appreciation: People Prefer Algorithmic to Human Judgment. In: Organizational Behavior and Human Decision Processes 151, 90–103.
- Castelo, N./Bos, M.W./Lehmann, D.R. (2019). Task-Dependent Algorithm Aversion.In: Journal of Marketing Research 56(5), 809–825.
- Schaap, G./Bosse, T./Vettehen, H.P. (2023). The ABC of Algorithmic Aversion: Not Agent, but Benefits and Control Determine the Acceptance of Automated Decision-Making. In: AI & Society (Open Forum), 1–14.
