Algorithm Aversion

An AI system makes hundreds of recommendations. Most are useful. Then it makes a visible mistake. A user notices the error, loses confidence, and begins avoiding the system. They may prefer their own judgment, even when the AI continues to perform better overall. This reaction can seem irrational. Why abandon a tool because of an occasional mistake when its overall performance remains strong? But people's responses to algorithmic errors are not determined by accuracy alone.

One visible failure can change how people perceive the reliability, appropriateness, and trustworthiness of an entire system.

  • What is Algorithm Aversion?

    Algorithm aversion refers to people's reluctance to rely on algorithmic recommendations or decisions, particularly after observing an algorithm make mistakes.

    Research has identified situations in which people prefer human judgment over algorithmic advice, even when the algorithm has demonstrated stronger overall performance.

    Importantly, algorithm aversion is not universal. People can also prefer algorithmic advice, and responses depend on the task, the user's experience, the nature of the error, and the perceived ability to influence the system.

    For organizations, the practical concern is that a system's actual performance and employees' willingness to use it may diverge.

Why Does It Happen?

  • Expectations of Consistency

    People may expect algorithms to be systematic and consistent. When an algorithm makes an obvious mistake, the error can violate expectations about how a machine should perform.

  • Error Salience

    A visible mistake can be more memorable than many successful recommendations. People may give disproportionate weight to an error that is easy to recall.

  • Loss of Perceived Control

    When people believe they cannot understand, influence, or correct a system, a mistake can reinforce the feeling that relying on it is unsafe.

  • Human Comparison

    People may judge an algorithm more harshly than a human making the same error. Human mistakes can be attributed to circumstances or individual limitations, while algorithmic mistakes may undermine confidence in the system itself.

  • Professional Identity & Expertise

    Employees with substantial domain expertise may be reluctant to rely on a system that appears to contradict their judgment. An AI mistake can reinforce the belief that their own approach is preferable.

  • Connection to the AI Trust Axis

    The AI Trust Axis maps Algorithm Aversion to three dimensions:

    • Cognitive Alignment: Does the AI's reasoning make sense, particularly when it produces an unexpected result?

    • Fairness Comprehension: Do users understand why the system produced a particular outcome, and do its decisions appear consistent?

    • Failure Recovery Intelligence: Can users detect, correct, and recover from errors?

    Failure Recovery Intelligence is especially relevant. An isolated error doesn't have to produce lasting distrust. But when users cannot understand what went wrong, correct the result, or obtain a reliable recovery path, one failure can undermine confidence in the entire system.

    This distinction is important: improving accuracy is not the only way to address algorithm aversion. The experience of failure and recovery also matters.

    Learn More about the AI Trust Axis

  • What Does It Look Like?

    • Employees stop using an AI tool after one or two highly visible mistakes.

    • A single failure receives more attention than a long record of successful performance.

    • Users revert to manual methods even when AI remains more accurate for the task.

    • Employees selectively use AI for tasks where it confirms their own judgment.

    • Confidence in AI declines without any corresponding deterioration in measured system performance.

  • Why Conventional Solutions Often Fail

    Organizations may respond by emphasizing overall accuracy, reminding employees that humans make mistakes too, or providing more examples of successful AI performance.

    These responses may help, but they can miss the psychological significance of the error.

    If users don't understand why the mistake happened, how it can be detected, or what safeguards exist, simply presenting an accuracy statistic may not restore confidence.

    Likewise, insisting that employees continue using the system can deepen resistance if they feel their concerns are being dismissed.

    The goal is not to persuade people that AI is infallible. It is to help them develop an appropriate understanding of what the system can and cannot do.

  • What Organizations Can Do Differently

    • Explain where the system performs well and where errors are more likely.

    • Investigate visible failures rather than dismissing them as isolated incidents.

    • Give users meaningful ways to correct, challenge, and report problematic outputs.

    • Demonstrate how the system and workflow respond when errors occur.

    • Measure confidence alongside technical performance.

    • Examine whether the organization can restore appropriate reliance after a failure.

  • Questions Worth Asking

    • Are employees avoiding AI because its performance is genuinely inadequate or because a particular failure changed their perceptions?

    • How do users interpret AI mistakes compared with equivalent human mistakes?

    • Can employees understand, challenge, and recover from system errors?