Google Makes First External Sale of Full TPU Racks, Inference Performance per Dollar Leads GB200 by 50%

Google Makes First External Sale of Full TPU Racks, Inference Performance per Dollar Leads GB200 by 50%

2026-11-16 12:06

Introduction: TorchTPU Open-Sourced, Google Catches Up on Software Ecosystem to Challenge CUDA. NVIDIA’s Headaches Go Beyond Just Hwang.

NVIDIA’s headaches extend far beyond just Hwang’s personal troubles.

Within this week, Google unveiled two new TPU chips, targeting confidential inference and long-context scenarios. Adding to the four TPU chips launched earlier this year, the eighth-generation TPU now comes in four distinct configurations.

This year, Google has released four TPU systems within a single calendar year for the first time. Yet Google’s ambitions for TPU may be even greater than commonly perceived—

Also this year, Google made a historic move by launching its first external sales of TPU systems.

According to insiders at Google, internal confidence is high, with executives stating, “TPU market share will rise rapidly next year,” and that “more large enterprises will adopt TPUs at scale.”

Regarding performance, they remain highly confident.

At least in the view of Google’s senior leadership, “the era of NVIDIA’s singular dominance is coming to an end.”

(Rumors: What? .jpg)

△ Image generated by AI

“Like NVIDIA,” Selling TPU Systems Externally

This year, Google made a heretical decision—breaking from tradition:

For the first time, Google began selling complete TPU systems to external customers.

QuantumBit learned from sources close to Google that the decision was driven by a senior executive who joined Google less than three years ago.

Amid intense internal debate, he championed a strategic shift in TPU commercialization, ultimately convincing leadership to “sell TPUs externally like NVIDIA”.

△ Image generated by AI

Originally (in 2013), TPUs were designed as custom accelerators for Google’s internal AI workloads to reduce compute costs. It wasn’t until 2018 that Google Cloud began offering TPU compute rentals to external users.

The cloud rental model meant Google retained full control over all underlying hardware infrastructure.

Fast forward to this year’s Q1 earnings call, CEO Sundar Pichai noted strong demand from AI labs, capital markets firms, and high-performance computing applications, announcing support for deploying TPUs within clients’ own data centers—targeting select customers only.

Internally, this external rollout is viewed as a structural pivot toward “sales-driven R&D”—using external orders to offset massive capex in data centers, while scaling demand drives down the cost curve of self-developed hardware.

Starting in Q2, Google began delivering full TPU systems to customers.

The system includes compute, storage, networking hardware, and software stack, along with training and inference services.

△ Image generated by AI

This business model is already showing revenue potential.

Since the beginning of the year, Google Cloud’s growth has accelerated. Q1 saw a 63% YoY increase, Q2 rose 82%, and by Q3, revenue reached $30.7 billion, up 103% YoY; operating profit surged from $3.594 billion to $11.121 billion.

(Whisper: Google Cloud might be the only part of the Q3 earnings report that doesn’t make you frown.)

Alphabet, Google’s parent company, explicitly stated that TPU system sales were one of the key contributors to Google Cloud’s Q3 growth.

In the latest 10-Q filing, Alphabet has formally categorized “product sales, primarily TPU systems” under Google Cloud revenue. Management expects TPU system sales to continue expanding into Q4.

Although most TPU system contract revenues will not hit the books until 2027, and are not yet reflected in current financials, this trend appears irreversible.

Why Now?

Google’s decision to sell TPUs externally is the inevitable outcome of both natural evolution and external conditions.

While at the Q2 earnings call, Pichai emphasized that Google’s internal demand for frontier AGI compute remains an absolute red line, the past two years have seen persistent shortages of high-end GPUs, forcing buyers unable to procure NVIDIA products to seek alternatives.

Google aims to capitalize on this window.

The seventh-generation TPU Ironwood, launched in 2025, was already optimized specifically for inference. By this year’s eighth generation, Google further split training, real-time inference, confidential computing, and ultra-long context into distinct system configurations.

Google’s chip lead Amin Vahdat hinted publicly that “the future belongs to specialization,” aiming precisely to undermine NVIDIA’s narrative of “one universal chip dominating all.”

△ Image generated by AI

Secondly, Google has been aggressively building up the TPU software ecosystem—hammering away at gaps with relentless pace.

One of NVIDIA’s biggest advantages over the years has been its CUDA ecosystem. NVIDIA pre-equipped AI developers with a full toolset early on, and vast amounts of software have been optimized and adapted around CUDA.

Among mainstream AI frameworks, PyTorch offers the most mature CUDA support.

The industry standard path has been: build models with PyTorch, invoke NVIDIA GPUs via CUDA, forming an entrenched, hard-to-replicate ecosystem moat.

For years, however, Google’s TPU remained tightly coupled with TensorFlow and JAX—its software-hardware stack differs significantly from GPU-based workflows, resulting in high migration barriers.

By mid-2025, Google finally began collaborating with Meta’s PyTorch team to develop TorchTPU, aiming to enable running PyTorch models on TPU with minimal code changes.

Last month, the collaboration’s outcome—TorchTPU—was officially open-sourced.

The impact? No need for me to elaborate.

△ Image generated by AI

In terms of performance and cost, on November 5, SemiAnalysis released new Agent inference benchmark results.

The results showed that Google’s seventh-generation TPU Ironwood achieved higher token output per dollar than NVIDIA GPUs in certain tests on models like Muse Spark.

Previously, NVIDIA had only lost to AMD in rare cases; the cost-performance pressure from TPU represents a scenario NVIDIA hasn’t seen—and certainly doesn’t want to face.

SemiAnalysis reported that after Google submitted AgentX test results demonstrating TPUv7 outperforming GB300 NVL72 FP4 on per-dollar performance, NVIDIA swiftly assembled an engineering team to counterattack.

NVIDIA engineers and agents worked non-stop, without weekends, pushing through until 2 AM local time.

After four optimization rounds, on one specific model, GB300 achieved 1.5 to 2.6 times higher token output per dollar than TPU.

On one front, AMD continues to close the gap within the GPU camp.

On the other, Google seeks to challenge GPU’s default status as the go-to compute substrate—at the architectural level.

NVIDIA can no longer avoid self-cannibalizing.

Just when Hwang thought his problems were solved, the AI chip market adds another layer of “dilemma.”

Truly, a season of trials for NVIDIA—truly a season of trials…

Source: QuantumBit

#agent#Launch#google#Inference#Hash Rate#meta#chip

Disclaimer: Contains third-party opinions, does not constitute financial advice

Share To
X
Telegram
WeChat
QQ
Link
Recommended Reading

Doubao Launches Paid Model: Can AI Subscriptions Work?

05-06
Doubao Launches Paid Model: Can AI Subscriptions Work?

Lending My Face to AI Shorts, But I Can't Get It Back

05-06
Lending My Face to AI Shorts, But I Can't Get It Back

Liáng Wénfēng's peers have made 200 billion in profits

05-06
Liáng Wénfēng's peers have made 200 billion in profits

The AlphaGo Moment in Cybersecurity, or Starting from These 466 Agent Matchups

11 days ago
The AlphaGo Moment in Cybersecurity, or Starting from These 466 Agent Matchups

The Trump administration plans to offer Constellation Nuclear a $5 billion loan for capacity expansion, with the additional power mostly pre-allocated to AI giants such as Google

19 days ago
The Trump administration plans to offer Constellation Nuclear a $5 billion loan for capacity expansion, with the additional power mostly pre-allocated to AI giants such as Google

Silicon Valley Executives Spark Controversy Over Using AI for Household Tasks; Survey Reveals Half of American Adults Worry About Impact on Relationships

19 days ago
Silicon Valley Executives Spark Controversy Over Using AI for Household Tasks; Survey Reveals Half of American Adults Worry About Impact on Relationships

Tensorlake npm package suffers supply chain attack, malicious version 0.5.144 injected with Shai-Hulud data-exfiltration worm

10-08
Tensorlake npm package suffers supply chain attack, malicious version 0.5.144 injected with Shai-Hulud data-exfiltration worm