OpenAI's Jalapeño chip challenges Nvidia in AI inference market

OpenAI unveiled its first custom AI chip this week, a semiconductor designed specifically for inference workloads that analysts say can match or exceed the efficiency of Nvidia's flagship Blackwell GPUs.

The chip, named Jalapeño, represents a strategic shift for OpenAI and underscores a broader industry trend: major AI companies are building their own silicon rather than relying exclusively on Nvidia's hardware. The announcement comes as Google, Meta, Amazon Web Services, and Anthropic have all committed to custom chip programs in recent months.

Performance claims and benchmarking

OpenAI developed Jalapeño in partnership with Broadcom and plans to deploy it within its compute infrastructure by year's end. The company says the chip will deliver faster responses and more reliable access as demand scales, and is already developing second- and third-generation versions.

According to SemiAnalysis, which visited OpenAI's labs to test the hardware, Jalapeño outperformed Blackwell on performance per watt in nearly all scenarios. However, the research firm noted the comparison isn't entirely apples-to-apples: Jalapeño uses newer HBM4 memory, while Blackwell relies on an earlier generation. Nvidia's upcoming Rubin platform, which also incorporates HBM4, represents a more direct competitor.

Why it matters

Custom AI chips threaten to erode Nvidia's profit margins in the fastest-growing segment of the AI infrastructure market. Inference—the process of running trained models to handle real-world queries—is expanding rapidly as AI applications move from research labs to production environments. While Nvidia maintains dominance in model training through its CUDA software ecosystem and broad programmability, inference workloads are more specialized and easier for competitors to target with purpose-built silicon. For OpenAI, which has been one of Nvidia's largest GPU customers, developing its own chips could fundamentally reshape that commercial relationship.

Nvidia's position under pressure

Adrien Sanchez, a technology analyst at Yole Group, told CNBC that while Nvidia still controls the vast majority of AI compute, Jalapeño poses "a threat to Nvidia's inference margins, which is the field growing the most at the moment."

Alexander Harrowell, senior principal analyst at Omdia, characterized Jalapeño as an "impressive achievement, most of all in terms of efficiency." He noted that in large-scale deployments, the power and cooling savings would significantly improve unit economics.

Omdia projects that custom application-specific integrated circuits like Jalapeño will exceed GPUs in volume by 2028, though revenue will lag given GPUs' higher price points. Harrowell identified this as "the biggest competitive threat to Nvidia," since roughly half of AI infrastructure spending comes from hyperscale cloud providers capable of launching their own chip programs.

Training versus inference

Analysts expect Nvidia to maintain its edge in compute-intensive training workloads. TrendForce analyst Fion Chiu told CNBC that for large-scale model training and frontier AI development, "we believe Nvidia GPUs will remain important given their broad programmability, performance, software ecosystem, and ability to handle a wide range of workloads."

But the inference market represents a different competitive landscape—one where specialized chips can deliver better economics for specific tasks.

Beyond the tech giants, startups including Cerebras, SambaNova, D-Matrix, Etched, and Fractile are also developing AI-focused semiconductors, further fragmenting what was once Nvidia's near-monopoly.

These details were first reported by CNBC.