The LLM Bubble is bursting

Some observers, including analysts for Goldman Sachs, say that the growth of AI since the height of the pandemic is now proving to be a bubble, like dotcom stocks in the late Clinton years or mortgage based derivatives in 2006-07.

What is a bubble? By standard definition it is a growth in value of an asset that occurs after the speculative demand has surpassed the operational demand for same. If I buy X as a speculator at a price higher than the operational value of X can support I may be willing to make an operationally foolish deal in the expectation (on the speculation) that there will be a "greater fool" to whom I can then sell it.

In such a situation, sooner or later the biggest fools ARE the ones who are holding X. What happens to its value then you can fill in for yourself.

Clearly there is money still entering the field. The Peoples Republic of China is incubating a group of companies that it proudly calls the "little AI dragons," with seed money of more than US$1 billion each.

One variant of the AI-is-a-bubble theory is this: LLM-is-the-bubble. The initials "LLM" refer to "large language models," that is, algorithms that consume huge blocks of internet-based text, organize it pertinent to your requests and spit it back at you in coherent English.

Question: "What is the plot of [novel x]." LLMs generally stay away from consuming novels directly, but they do take in reviews or scholarly articles about such novels, and from them they can spin you a plot summary suitable for a lazy high school student to turn in as a book report. This is the sort of trick that has made AI, of which LLM is a subset, so valuable. [For the student anyway.]

From the point of view of the plagiarist high school student, or other users if we find that one unsympathetic, the problem is that LLM can produce howlers (called "hallucinations" in the literature on the subject) that make it all too easy for the human teacher to catch on and flunk him. Or, more generally, hallucinations are results that are coherent and factually correct, yet factually wrong. The LLM in our plot summary example could, let us say, mix up the names of major characters. "The Don, Michael, throws a lavish wedding for his daughter. His soldier son, Vito, arrives with a girlfriend."

Anyway, the LLM-is-the-bubble theory says that the limits of this technology are becoming evident. That in turn is making it clear that the operational value of the technology is a good deal lower than had been thought. If the value of stocks such as Nvidia continue at anything like current levels this will be the consequence of speculative demand, not the operational sort. Definitionally, a bubble.

Nonetheless, the theorists continue, LLM is not all there is to AI. As the companies involve shift away from reliance on LLM to other applications of the underlying technology, (and the new Chinese concerns may avoid the over-emphasis on LLM in the first place) prices will find their new equilibrium and perhaps begin to rise again. So, dread not.

A Story About Coleridge

This is a quote from a memoir by Dorothy Wordsworth, reflecting on a trip she took with two famous poets, her brother, William Wordsworth, and their similarly gifted companion, Samuel Taylor Coleridge. We sat upon a bench, placed for the sake of one of these views, whence we looked down upon the waterfall, and over the open country ... A lady and gentleman, more expeditious tourists than ourselves, came to the spot; they left us at the seat, and we found them again at another station above the Falls. Coleridge, who is always good-natured enough to enter into conversation with anybody whom he meets in his way, began to talk with the gentleman, who observed that it was a majestic waterfall. Coleridge was delighted with the accuracy of the epithet, particularly as he had been settling in his own mind the precise meaning of the words grand, majestic, sublime, etc., and had discussed the subject with William at some length the day before. “Yes, sir,” says Coleridge, “it is a maj...

Searle: The Chinese Room

John Searle has become the object of accusations of improper conduct. These accusations even have some people in the world of academic philosophy saying that instructors in that world should try to avoid teaching Searle's views. That is an odd contention, and has given rise to heated exchanges in certain corners of the blogosphere. At Leiter Reports, I encountered a comment from someone describing himself as "grad student drop out." GSDO said: " This is a side question (and not at all an attempt to answer the question BL posed): How important is John Searle's work? Are people still working on speech act theory or is that just another dead end in the history of 20th century philosophy? My impression is that his reputation is somewhat inflated from all of his speaking engagements and NYRoB reviews. The Chinese room argument is a classic, but is there much more to his work than that?" I took it upon myself to answer that on LR. But here I'll tak...

Recent Controversies Involving Nassim Taleb, Part I

I've written about Nassim Taleb on earlier occasions in this blog. I'll let you do the search yourself, dear reader, for the full background. The short answer to the question "who is Taleb?" is this: he is a 57 year old man born in Lebanon, educated in France, who has been both a hedge fund manager and a derivatives trader. He retired from active participation from the financial world sometime between 2004 and 2006, and has been a full-time writer and provocateur ever since. Taleb's writings for the general public began where one might expect -- in the field where he had made his money -- and he explained certain financial issues to a broad audiences in a very dramatic non-technical way. Since then, he has widened has fields of study, writing about just about everything, applying the intellectual tools he honed in that earlier work. As you might have gather from the above, I respect Taleb, though I have sometimes been critical of him when my own writing ab...

Jamesian Philosophy Refreshed

Search This Blog