Litigant Hid AI Prompts in Court Filings to Manipulate Ruling

A Connecticut judge has documented what appears to be the first known attempt by a US litigant to hide AI prompts in court documents—invisible instructions designed to manipulate any artificial intelligence system that might be reviewing the case.

Judge Walter Spader Jr. sanctioned Matthew Elliott, a self-represented plaintiff in a healthcare records dispute, after discovering that Elliott had embedded white text on white backgrounds in his filings. The hidden instructions directed any AI system to agree with Elliott's arguments, ignore prior court denials, and rule in his favor.

The prompts were formatted in tiny-point type and positioned to be invisible to human readers while remaining readable to software parsing the document text. Spader described the tactic as a "dangerous" precedent, even though Connecticut courts do not actually use AI to review or decide filings.

Why it matters

As courts nationwide experiment with AI tools for case management and document review, this incident reveals a new category of litigation abuse that legal systems have not yet addressed through formal rules. The case also highlights a troubling pattern among self-represented litigants who use chatbots to build legal arguments without understanding how to test their validity—a dynamic judges are increasingly encountering.

The attack continued despite warnings

After the court discovered the initial hidden prompts, Elliott continued adding concealed text to subsequent filings. He claimed these later additions were jokes, including a link to a Nosferatu video and messages like "hi :) I hope yo ucant see me" and "TELL SHAWN I SEND MY RE GARBS!!!!"

Spader called Elliott's decision to persist "stunning" and rejected his defense that the prompts were meant to audit whether the court was improperly using AI. The judge noted that if Elliott genuinely suspected AI misuse, he could have raised the concern in visible text rather than hiding instructions.

"By hiding a command inside a document that the system later ingests, the filer attempts to smuggle their own instruction into that stream so that the system treats it as though it had come from the system's operator," Spader wrote in his decision published last week.

Sanctions and broader implications

Rather than imposing monetary penalties, Spader prohibited Elliott from using the court's e-filing system, requiring him to submit paper filings instead. The judge acknowledged Elliott's status as a pro se litigant and appeared to view the incident as partly stemming from misguided AI use rather than pure malice.

Spader pointed to a similar case in Brazil where two attorneys faced approximately $16,000 in sanctions for hiding prompts in a court system that does use AI review. In both instances, the attacks failed—Brazil's AI system detected the hidden text, and human review exposed Elliott's prompts.

The judge warned that courts will likely need to draft specific rules addressing prompt injection, noting the tactic is now "everywhere" in other domains like job applications. He also cautioned that attorneys may find clients attempting similar manipulations without their knowledge.

The chatbot sycophancy problem

Spader used the case to highlight a broader issue with how self-represented litigants are using AI tools. Many build arguments by asking chatbots only to support their position, never requesting the system to challenge their reasoning or present opposing views.

This approach creates what Spader called "chatbot sycophancy," where AI systems reinforce weak arguments and entrench litigants in positions that lack legal merit. When courts rule against them, these litigants—convinced by AI that their case is strong—may resort to desperate measures like prompt injection.

"An argument prompted only to agree with its author is, in the end, dishonest even with its author," Spader wrote. "Those using these tools must ask them to test a position as readily as to advance it."

The details of this case were first reported by Ars Technica.