Proving AI Didn't Write It: The Hidden Cost of Denying Bots Free Speech

A legal argument gaining traction in courts and scholarship holds that large language model outputs aren't "speech" under the First Amendment—and therefore can be regulated without constitutional scrutiny. The theory sounds clean: machines don't have rights, so their words don't get protection. But administering that rule across the internet creates a thornier problem: how to tell human expression from machine output when the words themselves rarely reveal their origin.

Why it matters

Making human attribution the line between protected and unprotected expression doesn't just affect AI companies—it creates pressure on platforms, users, and governments to verify authorship at scale using tools that can't reliably do the job. The likely substitutes are identity checks and personhood credentials that establish who is speaking, not what they wrote. That shift could erode online anonymity and burden the human speech the rule claims to protect.

The legal landscape shifts

U.S. District Judge Anne Conway recently wrote in Garcia v. Character Technologies that she was "not prepared to hold that [LLM] output is speech," allowing product liability claims to proceed in a wrongful-death suit involving a teenager's interactions with an AI chatbot. The case, first reported by Lawfare, settled in January 2026, but similar suits continue.

Legal scholars including Mackenzie Austin, Max Levy, Peter Salib, and others have built frameworks arguing that machine output lacks the human intent, certainty, or speaker necessary for First Amendment coverage. The approaches differ in detail but converge on one point: human attribution determines constitutional protection.

The tools don't match the task

Text classifiers that detect AI-generated content are unreliable and answer the wrong question—they show whether text resembles model output, not whether a person adopted it. OpenAI withdrew its own detector in 2023 due to low accuracy, and research shows these tools disproportionately flag non-native English speakers.

Content provenance systems like the C2PA standard can document a file's editing history but don't prove who's responsible for its message. Behavioral detection systems identify likely bots but classify accounts, not individual texts, and grow more invasive as they seek more certainty.

Personhood credentials—cryptographic proofs that a unique human controls an account—address a real problem in an internet populated by AI agents. But they establish presence, not authorship. A verified person can post machine-generated text; an automated system can distribute human writing. The credential answers "Is someone real here?" not "Did they write this?"

When verification becomes coercion

The burden doesn't require a legal mandate. Once verified speech receives preferential treatment or visibility, unverified speech becomes suspect. Platforms facing liability for unprotected machine output have concrete incentives to demand proof of human authorship—or its closest available proxy.

Biometric systems like World's iris-scanning orbs offer one path, certifying unique personhood through hardware verification. The company reports nearly 18 million users but has faced bans and investigations across Europe, Asia, Africa, and Latin America over privacy concerns. Unlike passwords, biometric identifiers can't be reset after a breach.

The distribution problem intensifies if Section 230 protections erode. Bipartisan proposals including the Sunset Section 230 Act would expose platforms to liability for hosted content. Under a no-speech rule, that exposure peaks for machine output—giving every intermediary reason to sort human from machine expression across millions of posts.

The paradox at scale

A rule designed to regulate machines without burdening human speech may do the opposite. When attribution becomes the constitutional test but existing tools can't prove it, institutions substitute questions they can answer. The result: pressure for identity disclosure, personhood verification, and surveillance systems that establish who speakers are rather than what they wrote.

These details were first reported by Becca Branum in Lawfare.