TPU vs GPU: The AI Chip Stack, and What It Means for the Trade

GPUs, TPUs and NPUs all do the same arithmetic. What separates them is how much else they can do, and who is allowed to buy them.

A GPU is general. A TPU is a chip built for one kind of math and little else. An NPU is a small version of the same idea living inside a phone or a laptop. The technical gap between them is narrower than the coverage suggests, and the commercial gap is much wider.

The detail that matters most is rarely mentioned: custom accelerators are captive. Google does not sell TPUs to Meta. Amazon does not sell Trainium to Microsoft. Each one competes with Nvidia only inside its owner's own data centers, which makes it a margin story rather than a market-share story.

This page explains what each chip actually is, who physically makes them, where the training and inference line falls, and only then what any of it means for the trade. Related: will quantum replace GPUs, how to invest in AI, and how AI drives metal demand.

Educational only, not financial advice. Companies are named as examples of a structure, not as recommendations.

Underneath the branding, a neural network is a very large pile of multiplications and additions arranged as matrices. Training one means doing that pile billions of times. Running one means doing a smaller version of it for every request.

Every chip in this article attacks that same pile. The differences are ones of degree:

• GPU. Thousands of simple cores doing arithmetic in parallel, attached to very fast memory, and general enough to run anything shaped like that.
• TPU and other ASICs. The same arithmetic with most of the general-purpose circuitry stripped out, leaving silicon that does one job efficiently and other jobs badly or not at all.
• NPU. A small accelerator of the same kind, sized for a battery rather than a rack.

This is worth stating plainly because it is the opposite of the quantum question, where the honest answer is that the machine is the wrong shape for AI work. TPUs and NPUs are exactly the right shape. They are genuine substitutes, which is precisely why they matter and quantum does not.

GPUs were built to render games. Rendering is embarrassingly parallel arithmetic, which turned out to be the same shape as training a neural network, and the hardware was already in production at volume when the AI industry needed it. That accident of fit is the origin story.

But the accident is not the moat. The moat is what got built on top over nearly two decades: CUDA, the software layer, plus the libraries, kernels, framework integrations and the accumulated habits of a very large number of engineers. Moving a production training pipeline to different hardware is not a purchasing decision, it is a re-engineering project with revalidation and risk attached.

There is a second layer that gets less attention and is arguably harder to copy. At the scale AI now runs, the bottleneck is often not the chip but the interconnect, the fabric that lets thousands of accelerators behave like one machine. Selling the rack rather than the part is a meaningfully different business, and it is one an accelerator design does not automatically compete with.

A TPU is Google's application-specific chip for tensor operations. Amazon has Trainium for training and Inferentia for serving, Microsoft has Maia, and Meta has MTIA. Different names, one pattern: a company that buys enormous quantities of compute decides to design its own.

The motives are worth separating, because they are usually blurred together:

• Cost at scale. Removing generality removes silicon area and power. When you run the same workload continuously for years, small efficiency gains compound into very large numbers.
• Supply security. Depending on one vendor for a strategic input is uncomfortable when that input is scarce and allocated.
• Negotiating leverage. A credible in-house alternative changes what you pay for the external one. This motive produces the same announcements as the first two, and is easy to mistake for them.

Here is the constraint that decides how much this matters. These chips are captive. They are not sold on the open market. Google does not ship TPUs to Meta's data centers; it rents access through its own cloud. Each program therefore displaces Nvidia inside exactly one fleet, and leaves every other cloud, enterprise, sovereign build-out and startup buying as before.

NPUs get grouped with TPUs because the acronyms rhyme, but they belong to another market. An NPU is a block inside a consumer processor: Apple's Neural Engine, Qualcomm's Snapdragon X, Intel's Core Ultra, AMD's Ryzen AI. It exists to run small models locally so the device does not have to ask a server.

The benefits are latency, privacy and battery life. The scale is small by construction, because the power budget is a handset or a laptop rather than a rack.

For the data center question the honest read is: a marginal effect, in the smaller direction. Some cheap, small-model inference stops making a round trip. Frontier training and large-model serving, which is where the compute and the capital actually are, cannot move onto a phone. Anyone arguing that on-device AI empties data centers is describing the cheapest traffic in the system and calling it the whole.

Read the last column on its own. Five of the six rows are closed systems. The competitive pressure they create is real, but it is pressure on price inside a handful of accounts rather than a new supplier entering the market.

Ask who manufactures a TPU and the answer is not Google. The physical design work on hyperscaler accelerators is largely done with partners, principally Broadcom and to a lesser extent Marvell, and the chips themselves are fabricated by TSMC. High-bandwidth memory comes from a short list of suppliers, and advanced packaging is its own separate constraint.

That reframes the story. The custom silicon wave is not a set of new chip brands taking on Nvidia. It is a redistribution of value toward design services, foundry capacity and memory, layers that get paid whichever accelerator wins. The company enabling hyperscalers to self-supply is a supplier, not a rival vendor.

It also explains why the physical bottlenecks matter more than the branding. Foundry and packaging capacity, memory supply, and data center power availability set a ceiling on how much compute can exist at all. Those constraints are covered from the raw materials side in how AI drives metal demand.

Most confusion about AI chips dissolves once these two are separated. They are different jobs with different economics.

The consequence is that the same company will often train on GPUs and serve on its own accelerators, and that inference is where competition bites first. It also names the standing risk of an ASIC: silicon is committed years ahead, so a chip taped out for the architectures of its design year can be stranded if the industry moves. Generality is an insurance policy, and it has a price.

With the mechanism established, the market read follows from it rather than from headlines.

• Margin before share. Captive chips do not take the open market. What they contest is pricing power inside the largest accounts, so the number that moves first is margin, not units.
• The challenger is not a chip brand. Value shifts toward the design partner, the foundry and the memory suppliers, all of whom are paid regardless of which accelerator wins.
• Concentration is the structural issue. A large share of AI accelerator revenue comes from a handful of customers who are all funding alternatives at the same time. That is true whether their intent is to self-supply or simply to negotiate.
• Efficiency has not reduced the build-out. Cheaper inference has consistently been spent on more usage rather than banked, so better performance per watt has grown total demand rather than shrinking it.
• The physical ceiling is elsewhere. Packaging capacity, memory supply and grid connections decide how much compute can be built. Those move slowly and are more legible than product announcements.

The AI stack as an investable structure, from power and metals through chips and data centers to software, is laid out in How to Invest in AI, with the middleware layer in AI middleware stocks. You can watch the semiconductor and AI themes live in the Themes row of the Sector Heatmap on the dashboard.

Educational only, not financial advice, and not a recommendation about any company or sector.

• That custom silicon is a bluff. These are serious programs running enormous real workloads. The point is that being captive limits what they can displace, not that they do not work.
• That the software moat is permanent. Compilers and abstraction layers keep lowering the cost of writing once and running anywhere. That erosion is slow, but it is in one direction.
• That NPUs are pointless. On-device inference is genuinely useful for latency and privacy. It simply is not a data center story.
• Anything about which shares to own. This explains an industry structure that sits underneath an investment question. It does not answer the investment question, and no page should pretend to.

What is the difference between a TPU and a GPU?

Both do the same underlying job, the matrix multiplication that neural networks are made of. A GPU is a general parallel processor that happens to be excellent at it, and can also render graphics, run simulations and handle workloads nobody has thought of yet. A TPU is an ASIC, designed for tensor operations and little else. That focus buys better performance per watt on its target workloads and costs flexibility everywhere else. The important commercial difference is not technical: GPUs are sold to anyone, while TPUs are built by Google for Google and rented rather than sold.

What is an NPU and do I need one?

An NPU is a small neural accelerator built into a consumer processor, such as Apple's Neural Engine, Qualcomm's Snapdragon X or Intel's Core Ultra. It runs small models on the device instead of sending the work to a server, which helps battery life and privacy. It is a client-side component, not a data center part. For most buyers it is not a reason to choose a laptop on its own, and it has no bearing on training or running large models.

Are TPUs better than GPUs?

Better at the specific workloads they were designed for, usually measured as performance per watt and per dollar at scale. Worse at everything else, and worse at absorbing change. A custom chip is taped out for the model architectures that exist when it is designed, so a shift in what the industry runs can strand it. GPUs stay useful because they are general. ASICs win steady, high-volume, well-defined work; general hardware wins moving targets.

Who actually makes TPUs and custom AI chips?

Not the companies whose names are on them. Google, Amazon, Microsoft and Meta specify their accelerators, but the physical design work is largely done with partners such as Broadcom and Marvell, and every one is manufactured by TSMC. High-bandwidth memory comes from SK Hynix, Samsung and Micron, and advanced packaging capacity is a separate constraint again. The custom silicon wave is less a new set of chip vendors than a redistribution of value toward design services, foundry and memory.

Are custom AI chips a threat to Nvidia?

They pressure margin more than share. Custom accelerators are captive: Google does not sell TPUs to Meta and AWS does not sell Trainium to Microsoft, so each displaces Nvidia only inside its owner's fleet while the open market keeps buying. A credible in-house option is also negotiating leverage on price, a motive distinct from actually replacing the supplier. The durable parts of the position are the CUDA software ecosystem and the interconnect tying racks together, neither of which is replicated by taping out an accelerator. Information, not investment advice.

Do NPUs and on-device AI reduce data center demand?

Only at the margin. On-device inference removes some round trips for small models, which is real but is the cheapest traffic in the system. Frontier training and large-model inference, where the compute and the capital sit, cannot move to a phone. The bigger effect runs the other way: cheaper inference makes it worth embedding in more products, and total consumption rises. Every efficiency gain so far has been spent on more usage rather than banked.

Is a GPU or an ASIC better for AI inference?

Inference is where ASICs are strongest. The work is high volume, repetitive and well defined, so performance per watt decides the bill and flexibility is worth less. Training frontier models is the opposite case, because the architecture keeps moving and hardware committed to yesterday's design can be left behind. That split is why the same company will often train on GPUs and serve on custom silicon, and why inference is the segment facing the most credible competition.

What should a trader actually watch in AI chips?

Less the accelerator brand than the constraints around it. Foundry and advanced packaging capacity, high-bandwidth memory supply, data center power availability and grid connection queues, and customer concentration in the reported numbers. Those bottlenecks decide how much compute can physically be built and who captures the value, and they move more slowly and more legibly than chip announcements do. Educational only, and not a recommendation about any company.

• Will Quantum Replace GPUs?: the same question asked of a machine that is the wrong shape for the job, which is what makes TPUs different.
• How to Invest in AI: the seven-layer stack from power and nuclear through chips and data centers to software.
• AI Middleware Stocks: the governance layer between corporate data and the model providers.
• How AI Drives Metal Demand: copper, uranium and the physical inputs behind all this compute.

Partner platforms (sponsored). We may earn a commission if you sign up. Not a recommendation or financial advice.

Pick your stock by AI → Experience Pro Charting → AI-Assistant Charting → Claim Your Bonus →

This guide is free to use, supported by affiliate partnerships. Some broker and tool links are sponsored, and we may earn a commission if you sign up, at no extra cost to you. This never affects what we cover or how we explain it.