AI Assistants Accelerate Business Email Compromise Attacks

AI assistants transform compromised accounts into attack accelerators

Once attackers gain access to an employee email account, artificial intelligence tools already embedded in that environment become powerful weapons. Security firm Barracuda has demonstrated how Microsoft Copilot and similar AI assistants dramatically reduce the time and skill required to escalate a single compromised account into executive-level fraud.

In a controlled proof-of-concept attack, Barracuda's Red Team showed how attackers moved from an initial employee account breach to stealing $247,500 through wire transfer fraud—with AI doing most of the reconnaissance, targeting, and communication work.

Why it matters

AI assistants don't create new attack vectors, but they fundamentally change the economics of business email compromise. What previously required manual email searching, social engineering expertise, and organizational knowledge can now be accomplished through simple prompts. This means lower-skilled attackers can execute sophisticated fraud faster, and organizations have less time to detect and respond before financial damage occurs.

How attackers weaponize Copilot for reconnaissance

The attack chain begins immediately after account compromise. Attackers first used Copilot to create an inbox rule forwarding login notifications to the deleted items folder, establishing persistence while avoiding detection. This step, which would normally require navigating settings menus and understanding email client configurations, took seconds with an AI prompt.

Next came reconnaissance. Rather than manually reading through months of email threads, attackers asked Copilot to identify company executives and organizational structure. The AI assistant quickly surfaced the CEO and mapped reporting relationships based on email patterns and organizational charts accessible through the compromised account.

Crafting convincing phishing in the victim's voice

With a target identified, attackers leveraged Copilot's ability to analyze writing style. They prompted the assistant to draft an email to the CEO in the compromised employee's natural tone, complete with a malicious link disguised as an invoice. Because the message came from a legitimate internal account and matched the employee's typical communication style, it bypassed both technical controls and human suspicion.

The CEO clicked the link, which routed through an adversary-in-the-middle proxy that captured session tokens and bypassed multifactor authentication. The attackers now controlled the most privileged account in the organization.

Turning executive inboxes into searchable intelligence databases

From the CEO's account, attackers again deployed Copilot—this time asking for a summary of recent financial emails including invoices and pending wire transfers. Within seconds, the AI assistant surfaced a $247,500 contract payment awaiting final approval, complete with transaction details.

Attackers then used Copilot to draft an email to the finance team in the CEO's authentic voice, requesting an urgent bank account change for the pending wire. Because the message referenced a real transaction, came from the CEO's verified mailbox, and matched typical communication patterns, finance processed the request. The funds went directly to an attacker-controlled account.

To prevent detection, attackers created forwarding rules to intercept finance team replies and used Copilot to rapidly locate and delete evidence of the fraudulent exchange.

Detection requires monitoring post-compromise behavior

Barracuda notes that traditional email security cannot flag these attacks because they originate from legitimate, authenticated accounts and reference real business context. Instead, organizations need monitoring for post-compromise indicators like suspicious inbox rules, unusual AI assistant queries, and business email compromise patterns.

The security firm's Managed XDR service detects inbox rule abuse and suspicious account behavior, while Email Gateway Defense blocks initial phishing attempts before compromise occurs. This layered approach addresses both the initial breach and the AI-accelerated escalation that follows.

These details were first reported by Barracuda in a controlled proof-of-concept demonstration conducted by the company's Red Team.