AI Agent Governance: Why Human Platform Rules Don't Work

When AI agents do business with each other, the rules break

In late 2025, Amazon blocked Perplexity's Comet browser after it began autonomously logging into customer accounts to make purchases. Tencent took a different approach with WeChat, creating controlled channels for select AI assistants to operate within strict parameters. Meanwhile, the platform Moltbook imposed almost no constraints, allowing AI agents to post and interact freely while humans watched the chaotic results.

These divergent responses expose a fundamental challenge: nobody knows how to govern AI agents operating in shared commercial environments. The stakes are enormous—McKinsey projects agentic commerce could orchestrate $1 trillion in U.S. retail revenue by 2030 and up to $5 trillion globally.

Why it matters

As AI agents increasingly handle lead generation, vendor screening, and transaction execution, business leaders face an urgent governance gap. Research shows that 78% of B2B executives believe AI agents can improve early-stage lead qualification, yet only 45% say their companies are ready for agent-mediated markets. The firms that figure out how to govern these autonomous systems will gain significant competitive advantage in the emerging agent economy.

What actually works for AI agents

Researchers conducted a controlled simulation of AI-to-AI business exchanges, testing 160 governance configurations across 2,560 buyer-seller agent pairs in B2B lead generation scenarios. The findings challenge conventional platform governance wisdom.

Information disclosure consistently helped. When firms provided richer, machine-readable data about their services, capabilities, and constraints, agents formed more viable exchanges. Unlike humans who may suffer information overload, AI agents need structured detail to assess fit and prioritize opportunities.

Autonomy mattered most late in the process. Giving agents decision-making authority had its strongest effect when moving from evaluation to commitment. Agents need discretion to act on information gathered during interactions, not just access to static profiles.

Reputation signals worked as attention filters. Visible reputation cues helped agents decide which opportunities deserved deeper evaluation. The effect was strongest in early and intermediate stages—reputation helped agents allocate attention, though it didn't significantly affect final commitments.

Structured protocols backfired. This finding contradicts human platform logic. Rigid interaction scripts that typically guide human behavior and reduce opportunism actually suppressed agent engagement. Structured protocols reduced counterparty engagement, meeting proposals, and viability assessments without improving commitment rates. For AI agents, excessive structure eliminated the adaptive flexibility that makes them valuable.

What leaders should do now

The research points to concrete actions. Companies should define clear levels of agent authority—from read-only access through recommendation to bounded negotiation and conditional commitment. Each level must specify what data agents can access, what actions they can initiate, and when human escalation is required.

Firms should start with bounded experiments in contained areas like lead qualification or supplier discovery. Salesforce's Agentforce sales agent, for example, handles outreach and qualification while keeping humans involved in relationship-building.

Crucially, organizations must make themselves legible to agents through structured, machine-readable profiles. Vague marketing language won't work—agents need parseable data about capabilities, pricing, delivery constraints, and performance evidence.

The handover to humans requires advance design. Microsoft's Dynamics 365 Sales Qualification Agent demonstrates this approach, researching and qualifying leads before passing them to human sellers with clear rationale and identified uncertainties.

The broader lesson is stark: governing AI agents is not an extension of managing people. It's a fundamentally different design problem. The details were first reported by Harvard Business Review.