How the AI bubble is going to pop

LLM training and inference are horrifically inefficient. It's fairly easy to count the number of tokens a human and LLM need to learn a language fluently. LLM needs around 150X more than a human. Inference efficiency gap estimates I've seen are similar. Memory usage is the same story.

If you ask an ML researcher why it takes so many more tokens to train an LLM than a human, or why 200kb of text needs 10 gigabytes of attention state in ram, they'll say "we don't know". The hardware rich bigcos don't know and don't care. They think they're going to scale everyone else out of business with their bigger hardware budget.

This entire CAPEX bubble makes a bad assumption that's already been proven wrong once. Remember Deepseek? One paper that showed models could be made 4X more efficient nearly crashed the market.

Someone is going to figure out a way to train and inference language models 100X faster or with 100X less memory. Or both. And all these data centers will be worthless overnight.

There's trillions of dollars riding on the assumption that nobody will figure this out. More money than the Manhattan Project. Big labs are trying to build a "hardware moat". And that's the exact kind of moat that got bulldozed when people figured out alternatives to mainframes in the 1980's.

This reminds me of the Dotcom bubble. Everyone knew that the web would be transformative. Yet 90% of investors bet on the wrong horses.

Big labs have a vested interest in NOT figuring out how to make models more efficient. Hardware cost is their entire moat. Some upstart is gonna figure it out and leave all of big tech spiraling.