Penn Engineering Publishes Open-Access Guide to AI Audits

A Framework for Understanding Black-Box AI Systems

Researchers at the University of Pennsylvania have released an open-access book that provides a systematic approach to evaluating artificial intelligence systems whose inner workings remain hidden from public view.

"Auditing AI," co-authored by Danaé Metaxa, Raj and Neera Singh Term Assistant Professor in Computer and Information Science at Penn Engineering, addresses a growing challenge: how to measure the real-world behavior and impacts of large language models, recommendation algorithms, and other automated systems that increasingly shape human decisions and outcomes.

According to details first reported by Penn Today, the book outlines methods for probing AI systems externally—testing their outputs and responses without requiring access to proprietary code or training data.

Why It Matters

As AI systems take on more consequential roles in content moderation, hiring, lending, and other domains, independent evaluation becomes essential. Organizations and regulators need ways to verify whether these systems function as intended, identify design flaws, and detect unanticipated harms—particularly when developers may be unwilling or unable to provide transparency. External auditing offers a path forward when internal access is restricted.

From Legal Disputes to Systematic Methods

Metaxa's work on AI auditing gained urgency through real-world cases. She recounts being contacted by a lawyer from a human rights organization negotiating a legal settlement with a social media platform. The platform's AI-powered content filters were allegedly discriminating against certain user groups, and the company claimed it couldn't improve the bias in its tools or track such improvements over time.

"As an expert in AI auditing, I could confirm that improving these tools was possible, and describe to him the kind of evaluations that he should ask the company to provide over time to demonstrate improvement," Metaxa explained.

This experience highlighted a critical gap: policymakers, advocates, and users lacked accessible frameworks for understanding and evaluating AI systems.

Rigorous Probing Without Internal Access

Metaxa describes AI audits as "a rigorous scientific method of systematically probing those systems and measuring their outputs to make inferences about how they work and what social impact they are having."

The approach allows researchers and regulators to:

• Test whether systems behave as their creators claim
• Identify patterns of bias or discrimination in outputs
• Measure consistency and reliability across different user groups
• Document changes in system behavior over time

By making the book freely available, the authors aim to equip a broader audience—from policy professionals to concerned citizens—with tools to scrutinize the AI systems affecting their lives.

The full book and additional details are available through Penn Engineering.