Flock Safety's AI Watchlist Tools Let Police Search by Description

Police Can Set Automated Alerts for People Matching Written Descriptions

Flock Safety's surveillance platform now includes AI-powered tools that let police officers search for people on camera using written descriptions, not just license plates. An officer can define an area of a city and instruct cameras to run continuous automated searches for anyone matching text like "person wearing scrubs," triggering alerts when the system finds a match.

WIRED obtained and analyzed the code underlying Flock's user interface, revealing how the company's content moderation system evaluates officer searches before they run. The examination shows that while Flock's guardrails flag certain sensitive searches, they often warn officers rather than blocking queries outright—and the final decision to proceed rests with the officer.

How the Moderation System Works

Before a search executes, an AI model on Flock's servers scores the officer's written description against eight categories of sensitive content. The system returns one of three verdicts: allow, block, or warn. Categories that trigger blocks include race, ethnicity, religion, nationality, and offensive content. However, searches involving "political, social, and cultural expression" generate only a warning that officers can click through.

The confidence scores the AI model uses to reach these verdicts remain hidden on company servers. Departments see only standardized warnings or blocks, with no visibility into why the system made a particular call. When officers proceed past a warning, Flock logs the search and notifies department administrators—but does not prevent the query from running.

According to WIRED's reporting, one California officer last year typed "American flag" into the system. The search was blocked when aimed at people but ran across 11,000 cameras when redirected to vehicles instead.

Dozens of Stalking Cases Tied to the Platform

At least 46 recent U.S. cases of officers accused of misusing license plate readers involve Flock's technology, according to The Washington Post. In more than half, the alleged target was a romantic partner, ex-partner, or woman the officer wanted to meet. One Milwaukee officer searched for a woman he was dating 124 times, logging each as an "investigation." A Georgia police chief ran searches on his ex-girlfriend 600 times before being charged; he later took his own life.

Flock announced changes in August aimed at curbing abuse: shorter default data retention periods, mandatory case codes for every search, and automated auditing. The company says these controls will be required for all departments by year's end. However, WIRED's code analysis found that some departments can currently configure the system to allow searches without case numbers or reasons tied to real investigations.

Why It Matters

Flock's surveillance technology sits at the intersection of AI capability and constitutional limits. Experts who study automated screening say no technology reliably performs the job Flock has given its AI—deciding which descriptions of people officers may search for at scale. The company itself warns that results may be inaccurate but places "the risk of any inaccuracies" on officers. Meanwhile, the one category of expression courts have protected most strongly under the First Amendment—political and cultural speech—receives the weakest guardrail: a warning officers can dismiss with a click.

Tom Bowman, policy counsel with the Center for Democracy and Technology, noted that political expression "seems like that actually has the least concern from a policing perspective" in Flock's system, despite being "one of the most protective areas" under constitutional law.

Researchers studying AI safety systems emphasize that effective moderation requires examining both what users input and what models return. Flock's approach focuses primarily on filtering search terms before queries run, leaving the accuracy and bias of returned results largely unexamined by anyone outside the company.

These details were first reported by WIRED, which built custom software to collect and analyze the files Flock sends to users' browsers.