Jensen Huang Says Memory Is Now AI's Biggest Bottleneck. Here's What That Means for Nvidia.
Key Points
Nvidia has begun building systems that rely on memory chips that are packed into its hardware.
Nvidia can use its billions in cash and purchasing scale to corner the market and have first dibs on memory chips as they become available.
The underperformance of Nvidia's stock is largely due to overall sentiment toward major AI companies, like the "Magnificent Seven."
Nvidia (NASDAQ: NVDA) has been around since 1993, but it came into the spotlight for its graphics processing units (GPUs), which are crucial for training and scaling artificial intelligence (AI). They provide much of the compute power that powers AI workloads.
At the beginning of the current AI boom, the goal for tech giants was simply acquiring as much compute power (i.e., GPUs) as possible. Now, the focus has shifted to memory chips, but as Nvidia's CEO, Jensen Huang, highlighted, those memory chips are now AI's biggest bottleneck.
Missed Nvidia in 2009? This Rare Signal Is Flashing Again. In 2009, a "Double Down" signal flashed for a little-known chipmaker called Nvidia. For the first time in years, that same "Total Conviction" signal is flashing for a company 1/100th the size of Nvidia. Continue »
Why memory is important to AI and Nvidia
AI training and application rely on trillions of data points, and it wouldn't be possible to store and quickly retrieve them without specialized memory chips. As AI is used for handling more complex tasks -- such as running autonomous agents or processing complicated context instead of providing recipes or travel recommendations -- the need for high bandwidth memory has become increasingly important.
Nvidia began its reign selling its GPUs and AI hardware, but now it's building systems with multiple working parts, including memory chips that are packed into its hardware. That means relying on memory chipmakers, such as Micron, SK Hynix, and Samsung, for a continuous, high-volume supply.
Unfortunately, making those specialized memory chips is far from simple, which is why only a handful of companies make the vast bulk of them.
Nvidia CEO Jensen Huang. Image source: Nvidia.
What does it mean for Nvidia?
The downside to the shortage is that Nvidia is at the mercy of memory chip suppliers for its own supply chain. If the suppliers can't make memory chips fast enough, Nvidia will have to wait, potentially affecting its own business.
The positive is that Nvidia has the cash and purchasing scale to have priority on the memory chips being made. In its most recent quarter (ended April 26), it generated $48.6 billion in free cash flow and finished the quarter with $13.2 billion in cash and cash equivalents. It can easily pay a premium to buy them in bulk, shutting out smaller competitors and further cementing its stronghold on the industry.
Investors shouldn't hear Huang's message and become concerned; it's just the next chapter of the AI evolution. If anything, it should be encouraging that Nvidia can use its leadership position to be a long-term force, regardless of the current hiccup.
Nvidia's stock has been a disappointment this year, up only 0.60% year to date as of market close on July 29, but much of that has to do with overall sentiment surrounding big tech and the "Magnificent Seven" stocks as a whole, versus disappointing business performance from Nvidia.
Stefon Walters has no position in any of the stocks mentioned. The Motley Fool has positions in and recommends Micron Technology and Nvidia. The Motley Fool has a disclosure policy.