Investing.com -- Google could save billions of dollars annually on its custom AI chips, known as Tensor Processing Units (TPUs), thanks to rising competition among application-specific integrated circuit (ASIC) service providers, according to analysts at Bernstein.

In a new report this week, Bernstein estimates that TPUs currently offer about “~4x performance per dollar vs. current-gen GPU,” though this measure only reflects chip acquisition cost.

Advertisement: High Yield Savings Offers

Earn 4.10% APY** on balances of $5,000 or more  View Offer

Earn up to 4.00% APY with Savings Pods  View Offer

Earn up to 3.80% APY¹ & up to $300 Cash Bonus with Direct Deposit  View Offer

Powered by Money.com - Yahoo may earn commission from the links above.

When accounting for total system costs—including software, R&D, and packaging—savings may still range from “1.5-4x,” underscoring TPUs’ economic advantage in certain AI workloads, according to Bernstein.

Much of this advantage is said to stem from high margins on rival graphics processing units (GPUs).

Bernstein writes that while the bill of materials (BOM)—covering memory and foundry costs—differs by only “1.3-2.4x between TPU & GPU,” the remaining cost gap is “from NVIDIA’s gross margins which can run 70% or higher.”

As the size and cost of training large AI models continue to rise faster than improvements in hardware efficiency, Bernstein believes increased use of ASICs like TPUs will be necessary.

“Moore’s Law isn’t sufficient,” the analysts noted, as it cuts cost per operation by just 25%–40% annually, while model size grows roughly 3.5x per year.

Although some hyperscalers like Amazon (NASDAQ:AMZN) and Google (NASDAQ:GOOGL) are building ASICs in-house, others—including Meta (NASDAQ:META), Microsoft (NASDAQ:MSFT), OpenAI, and xAI—remain dependent on external providers.

Bernstein concludes that despite potential “demand lumpiness” and the addition of new customers, “will keep the overall ASIC market growing in the long run.”

Related articles

How competition among ASIC service providers can help Google save billions in TPU

Nvidia’s Huang sees no evidence of AI chip leakage to China

BT set to sell TNT Sports stake to Warner Bros Discovery – FT

View Comments