Highlighting 4 potential risks for the boom in memory stocks and semis. Memory stocks are most exposed here.
1. Efficiency Improvements: If frontier models figure out how to run agentic AI with 80% less KV cache or memory overhead, the shortage disappears.
2. China Supply: If they successfully mass-produce High Bandwidth Memory or even flood the mid-range market, the Western companies lose pricing power.
3. Companies Scale back on AI Spend: Cost of running AI at companies is exploding. As of now, AI first is more expensive than the old human + SAAS stack.
4. Construction Delays and Grid Constraints: Physical construction and grid/regulatory problems are already causing delays.
I'm not saying this will happen, but all of the above are at least somewhat likely to happen and would have a massive impact on demand or supply. Be careful out there.
Posted by m4329b
8 Comments
Makes sense. But still calls it is
2. China supply isn’t coming online anytime soon.
3. Hyperscalers have already committed to spending and signed multiyear contracts with memory companies. There just isn’t enough supply to meet demand.
4. Construction delays don’t address the supply issue. There isn’t enough demand to satisfy supply for the next two years.
> 80% less KV cache or memory overhead, the shortage disappears.
This is simply not true. Attention has already gone through multiple algorithmic and constant-time performance improvements, and memory demand has only increased. The key question is not “how much memory will be used” but “how much memory overhead occurs per unit of compute” and that metric only means we need more, faster, higher bandwidth memory as GPUs get faster, not less.
I would add politics. There is growing hatred towards AI and especially data centers and it seem to be bipartisan. Blocking the buildout drops demand dramatically.
I don’t know 2,3,4, but I guarantee 1 is not true.
We could magically have 10x more CPU, GPU and VRAM and we’d just make models that are 10x bigger and more demanding. The demand for compute is unimaginable, like being in 1850 and trying to predict the demand for electricity in 2026.
Jevons Paradox
1 and 2 are simply not factors in the near term.
3 is the most probable scenario but investing in AI is currently driving strong ROI for the hyperscalers, not to mention its existential for them.
4. Demand/supply gap is huge at the moment. Even if we just maintain the current rate of DC building, it will take years for demand and supply to even out.
Even a hint of a thought of a headline for point 3 would blow a huge hole in their prices. Pricing is entirely based on faith right now.