Workers Skip AI Explanations When Bonuses Are on the Line

The transparency paradox

Employees using AI decision-support tools frequently choose not to see explanations for algorithmic recommendations—particularly when their compensation depends on outcomes or when those explanations might reveal bias, according to new research from Harvard Business School.

In a study of 2,512 participants acting as loan officers, Assistant Professor Alex Chan found that while 80% wanted to see AI risk scores for borrowers, only 46% chose to view explanations of how those scores were calculated. The gap widened significantly when financial incentives entered the picture.

Participants reviewing real $10,000 loan applications were nearly 20% more likely to skip explanations when their bonuses depended on loan repayment. When told an explanation might reveal whether race or gender influenced the AI's recommendation, avoidance rates jumped by more than 10 percentage points.

Why it matters

As organizations embed AI into high-stakes decisions—from credit approval to hiring and medical diagnostics—this research challenges the assumption that transparency alone ensures responsible use. The findings suggest that disclosure requirements and explainable AI features may be insufficient if employees have incentives to remain willfully ignorant of how algorithms reach conclusions.

When knowing complicates deciding

The study, detailed in Chan's working paper "Preference for Explanations: Case of Explainable AI," revealed that explanations changed behavior when people did engage with them. Participants who viewed AI reasoning were about six percentage points more likely to override the algorithm's recommendation and approve both loans in a pair.

This pattern suggests people avoid explanations precisely because the information creates moral discomfort or complicates decisions they'd prefer to keep simple.

"Humans interacting with AI are not perfectly rational Bayesian agents," Chan notes in the research, first reported by Harvard Business School Working Knowledge. "They are strategic, motivated, and sometimes willfully ignorant."

Building better oversight

Chan's findings point to three organizational imperatives for companies deploying AI decision tools:

First, passive disclosure isn't enough. While regulations like the EU's AI Act and GDPR require explanations for algorithmic decisions, organizations need active oversight mechanisms that ensure employees actually engage with that reasoning rather than treating it as "checkbox transparency."

Second, incentive structures matter enormously. When compensation models reward outcomes without accounting for decision quality or fairness considerations, employees will rationally avoid information that might complicate their path to higher pay.

Third, organizations must actively cultivate critical engagement with AI recommendations through training and cultural reinforcement. "The biggest risk of AI isn't just bad answers or lack of adoption," Chan warns. "It's training people to stop asking why."

The research arrives as U.S. regulators increase scrutiny of algorithmic lending. In 2023, the Consumer Financial Protection Bureau reminded lenders they must provide "specific" and "accurate" reasons for AI-assisted adverse decisions like credit denials—a requirement that assumes those explanations will actually influence human decision-makers.

Chan's work suggests that assumption needs testing. The details were first reported by Harvard Business School Working Knowledge.