I've been using and exploring the current crop of "AI", which is really just various iterations of LLMs for a good two years now. Given all my experiences, this is my conclusion: LLMs are useful in a narrow set of use-cases, none of which will justify the ridiculous investments. Let me explain.
First, let's get illusions out of the way: LLMs are very fancy database-queries that yield complex and often useful results with very simple input. That is some impressive technology by any means, but it's about as "intelligent" as SQL.
So what are working use-cases?
Programming support LLMs are great at making the job of programming a bit easier. Much like powerful IDEs, they can take some of the burden off good programmers by helping with simple, repetitive tasks here and there. By their very nature, they will never replace human programmers, since they lack any understanding of the task at hand. Yes, we've all seen pretty impressive "vibe-coding" results, but these have no chance of standing up in the world of real usage. There is also quickly dimishing returns when refining prompts takes about as long as just straight up programming against requirements. Nevertheless, I wouldn't want to miss Github Copilot in my day to day programming, it's often useful and just generally speeds things up. How much is this worth, though? Possibly a couple of billions, however big the market for professional programming is. And it's not a market that can be monopolized anymore, so prices will be capped pretty hard, but this a definitely one of the things where business can and will be made. Just not trillions.
Adult content creation I don't think I need to explain the business case here. One does have to wonder how big the margins are, though. People are cheap and developing targeted LLMs is expensive. And obviously for the majority of investors this is not a business they can or want to get into. However, this too will stick around as a business, because of course it will.
Text editing and translation LLMs are very good at transforming text into other languages. But, of course, every bit of output has to be validated. There are definitely savings to be had here, but again humans are needed to get actual results that will stand up to scrutiny in the real world. There is also a case for using LLMs for error correction – ironically – and generally improving the quality of text. But computer aided translation had been around for years and LLMs are at best a marginal improvement, so it's not a disruptive business that will yield insane profits.
Cheap visuals If you don't have specific requirements, visual models can create a lot of pleasing output very fast. But, since the results are never quite specific and prone to errors, this really only works for cheep content that can be easily discarded. There's also the open question of copyright. Should courts or lawmakers decide that ingesting artwork into LLMs does in fact require the original creator's consent, this will be over in an instant (which would be true for the adult content case as well). And even if this use-case persists, it will stay narrow. It's really analogous to the programming one, where it might help to reasonably speed up workflows without replacing the majority of the work.
And that's about it. All LLM generated results always have to be fully verified by humans, they are full of errors and hallucinations and they get worse the more complex the requirements. Some of this can be countered by having them take considerably more iterations to refine the results, but ultimately you run into diminishing returns. The "mind blowing" demos of the latest crop are always very carefully prompted and don't reflect real-life requirements that would bring tangible value customers would be willing to pay for.
LLMs will stick around and generate reasonable revenue, but the trillions invested will be largely lost. Getting into this hype now, financially, is a losing bet.
Props to Nvidia, they sold the shovels and made a killing.