CMS Shifts AI Strategy from Usage Metrics to Outcome Tracking

The Centers for Medicare and Medicaid Services is recalibrating how it evaluates artificial intelligence deployments, shifting from tracking how many employees use AI tools to measuring whether those tools produce meaningful results for the populations it serves.

CMS Chief Information Officer Patrick Newbold outlined the strategic pivot during a panel at GovCIO's Health IT Summit in Bethesda, Maryland. The agency previously emphasized adoption metrics — CMS has reported that roughly 80% of its workforce uses AI tools, including the internal "CMS Chat" chatbot. But that approach is changing.

"Instead of leading with AI, we're going back and we're leading with outcomes," Newbold said. "And if it just so happens that AI is helping meet those outcomes, that's a good thing."

Why it matters

As federal agencies face the end of deeply discounted AI licensing deals and pressure to demonstrate return on investment, CMS's shift represents a maturing approach to enterprise AI adoption. Measuring outcomes rather than seat counts could provide a template for other agencies navigating similar questions about AI value and sustainability.

Tracking results through Tech Pulse

CMS monitors technology outcomes through an internal forum called Tech Pulse, which launched approximately 20 months ago and was modeled after a similar structure at the Social Security Administration. The forum uses objectives and key results as its primary metrics.

Newbold described Tech Pulse as "a performance management-based forum to keep in track and push a lot of the key outcomes and objectives that we had from an IT perspective at CMS." One current objective being monitored: transitioning to a multi-cloud environment by the end of the fiscal year. In that framework, specific tools and capabilities are viewed as enablers rather than end goals.

The CIO emphasized that usage metrics remain relevant but are no longer the primary measure of success. What usage figures don't capture, he noted, is whether technology is actually improving outcomes for Medicare and Medicaid beneficiaries.

Cost concerns as discounts expire

CMS's strategic recalibration comes as the federal AI landscape enters a period of uncertainty. The General Services Administration's OneGov initiative provided deeply discounted access to major generative AI platforms across federal agencies, but those deals are expected to end soon.

CMS currently has access to more than 70 AI use cases documented in the Department of Health and Human Services' 2025 AI use case inventory, plus department-wide access to generative AI tools through the OneGov agreements.

Newbold acknowledged concerns about long-term costs. "There's not a lot of information coming from the foundational models on what the costs are going to be long term," he said. "We took advantage of some of the OneGov deals, but we know those are not going to be forever."

To manage potential cost escalation, CMS is implementing governance structures to understand and control usage patterns. Newbold said the agency currently has the capabilities to support that level of oversight, though he expressed concern about limited transparency from AI model providers regarding future pricing.

These details were first reported by Madison Alder at FedScoop.