Algorithm Aversion
Category: Trust & Fairness
Related Concepts: Automation Bias, Trust Calibration, Loss Aversion, Error Sensitivity
Behavioral Mechanisms: Error Salience, Negative Updating, Perceived Lack of Human Judgment
Definition
Algorithm aversion is the tendency for individuals to distrust or reject algorithmic or automated systems—especially after observing them make even a single mistake—while tolerating comparable or greater error rates from humans. People often perceive algorithmic errors as more severe, less forgivable, and more indicative of systemic failure, leading them to prefer human judgment even when algorithms perform better on average.
In Plain Language
People are quick to lose trust in algorithms. When a human makes a mistake, we see it as normal; when an algorithm makes a mistake, we see it as proof that the system is flawed. This leads users to abandon tools after minor errors, override recommendations, or revert to familiar manual processes. Algorithm aversion isn’t about actual accuracy—it’s about how errors feel. Machine mistakes seem colder, more rigid, and harder to explain, so people react more strongly to them.
Why It Happens
Algorithm aversion arises from several interacting mechanisms:
Error salience: Algorithmic errors stand out more and feel more consequential.
Loss aversion: Mistakes made by machines feel like losses of control or competence.
Mental models: People often lack a clear understanding of how algorithms work, making errors seem mysterious or unpredictable.
Perceived inflexibility: Algorithms are seen as unable to adapt, contextualize, or “use judgment.”
Negative updating: A single error disproportionately reduces trust, even when overall performance is strong.
These mechanisms combine to create a fragile trust dynamic: algorithmic trust is easy to lose and hard to rebuild.
Implications for Design, Governance, and Decision-Making
Algorithm aversion has major implications for how AI systems should be introduced, explained, and governed:
Error transparency: Showing how often humans err helps contextualize algorithmic mistakes.
Reversibility: Allowing users to easily override or adjust recommendations reduces perceived risk.
Progressive trust-building: Start with low‑stakes tasks and gradually increase autonomy as confidence grows.
Human–machine teaming: Combining algorithmic suggestions with human judgment increases acceptance.
Feedback loops: Allowing users to correct or influence the system reduces feelings of rigidity.
Communication: Framing the algorithm as a support tool—not a replacement—reduces defensiveness.
Designers and leaders must anticipate that trust in algorithms is fragile and requires deliberate reinforcement.
Applications Across Domains
Healthcare: Clinicians may reject diagnostic tools after a single incorrect recommendation, even if the tool is statistically more accurate than human judgment.
Finance: Customers distrust automated financial advice when they see small inconsistencies or unexpected outputs.
Education: Students avoid algorithmic grading tools if they perceive one unfair or incorrect evaluation.
Consumer behavior: Users abandon recommendation engines after receiving one irrelevant or odd suggestion.
Workplace technology: Employees override automated workflows or alerts because machine errors feel more consequential than human ones.
References
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018). Overcoming algorithm aversion: People will use imperfect algorithms if they can (even slightly) modify them. Management Science, 64(3), 1155–1170.
Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.