Cornell launches AI security initiative to vet agent-generated code

Cornell tackles security risks in AI-generated code

Cornell University has established a new research initiative to address security vulnerabilities in code produced by AI agents, according to an announcement from the university. The AI4AI program, funded by Amazon, will develop verification frameworks to ensure that autonomous AI systems generate safe, secure software.

Alexandra Silva, professor of computer science at Cornell's Ann S. Bowers College of Computing and Information Science, and Vitaly Shmatikov, professor at Cornell Tech, are leading the project titled "Assured Integrity for AI-Based Software." The initiative brings together faculty specializing in machine learning, security, formal methods, and verification.

The vulnerability problem

Agentic AI systems can now build and deploy software applications from simple text prompts, working across multiple tools, searching the internet, and interacting with code repositories. However, these systems lack the critical judgment to evaluate security implications, researchers note.

Because AI agents follow instructions without skepticism, they can introduce security weaknesses when requirements are unclear or when prompts contain malicious intent. The consequences of vulnerable code can extend throughout software supply chains.

"Agentic AI has great potential to change how software is written, but we need stronger assurance that agents will not generate harmful code," said Andrew Myers, professor of computer science at Cornell. "In this project we are exploring how to get the real benefits of agentic AI but in a safe and secure way."

Building a security framework

The Cornell team will develop a security framework that makes AI agents more cautious about their code outputs. This framework will include rules and verification checks that run before code is deployed.

The research draws on Cornell's strengths in both artificial intelligence and programming languages. Participating faculty include Saikat Dutta, Kevin Ellis, Greg Morrisett (who serves as dean of Cornell Tech), and Myers.

"AI4AI brings together our strengths in AI and in programming languages, and we're very excited about this partnership with Amazon," said Thorsten Joachims, Cornell's vice provost for AI strategy and director of the Cornell AI Initiative.

Why it matters

As organizations increasingly adopt AI coding assistants and autonomous development tools, the security of machine-generated code becomes a critical infrastructure concern. Vulnerabilities introduced during the coding phase can persist throughout an application's lifecycle, creating attack surfaces that may not be discovered until after deployment. Establishing verification methods now—while agentic AI is still emerging—could prevent a new generation of systemic security flaws.

Industry collaboration

Debashis Das, principal in the Office of the Chief Information Security Officer at Amazon Web Services, emphasized that AI security agents will eventually protect software across its entire lifecycle, not just during initial development. "Industry-academia collaboration like this is essential to advancing the tools and frameworks the developers need to innovate with confidence," Das said.

Details of the initiative were first reported by Cornell University.