The novelty effect that started with ChatGPT in November 2022 is winding down (1). Meanwhile, massive AI investments are ramping up. Only Alphabet, Amazon, META, and Microsoft are expected to spend $210B (roughly 20% of Spain's GDP). This apparent misalignment is making many (1, 2) question whether there is an AI bubble, i.e., will the investment be justified, or will we see a dramatic slowdown in investment, capacity build-up, and general AI interest?
The thesis of this post is that the outcome of the AI bubble depends on whether the value per token generated (the utility of the model) increases before a significant write-off event occurs. Moreover, the non-linearity or even exponential nature of this value per token, which grows across multiple dimensions in both use cases and complexity, makes justifying the hype very challenging. A major write-off event could be triggered by several factors, such as defaults on the billions of dollars companies are borrowing to create GPUs as a service (CoreWeave borrowed $2.4B), market corrections affecting the top tech companies, or the new Nvidia generation accelerating depreciation curves (Nvidia has changed their new generation deployment cycle to 1-year!).
<aside> š Caveat: I do not like nor trust expert predictions on almost any topic - they are generally wrong - . Neither am I an expert, nor is this an attempt at making a prediction. This is simply me trying to make sense of the news and distill it down to what I believe are its first principles. Writing helps (and forces) me to delve deeper and be more rigorous, which is why I publish it.
</aside>
We could spend hours debating the potential of AIāand I will touch on this laterābut if we accept that this technology applies broadly across various use cases, both old and new, we can use Carlota Perezās interpretation of the traditional āSā curve as a helpful framework (not a theory). This concept, first applied to technology by Everett Rogers, helps us better understand the situation.
Carlota Perez S-curve
In Carlotaās framework, every new technological wave (or techno-economic paradigm, but let's go with Mustafa Suleymanās terminology for simplicity) starts when several important technologies converge, creating a general-purpose technology with the potential to transform the economy. Think about the Industrial Revolution (1771), the Age of Steam and Railways (1829), the Age of Oil and Mass Production (1908), and the Information Technology revolution (1971).
GenAI, with all its potential, represents the convergence of many technologies that have been building since the IT revolution began in the 1970s. Each eraāthe personal computer (1970s), the internet (1990s), mobile computing (2007), and cloud computing (2010s)āhas laid the groundwork, supported by extensive AI research in Neural Networks, Deep Learning, and the transformative transformer architecture. Simply put, these technologies have provided the two essential elements an LLM needs: compute and data, all enhanced by a robust architecture.
For GenAI, this was November 30, 2022, the day OpenAI launched ChatGPT. People transitioned from using Alexa to play music to having fully fleshed conversations with ChatGPT, Luzia, and other assistants that emerged. š¤Æš¤Æš¤Æ
Interest over time of the term āGenerative AIā
At this point, argues Carlota, financial capital is poured into āthe new thingā and a phase of intense innovation and infrastructure development begins.
"When we go through a curve like this, the risk of underinvesting is dramatically greater than the risk of over-investing for us here, even in scenarios where if it turns out that we are over-investing.ā - Sundar Pichai
Sundar's feelings about the CapEx investmentāquoted from the 2024 Q1 earnings callāreflect the industry's general sentiment.
Alphabet, Amazon, Meta, and Microsoft are projected to spend around $200-210 billion combined on AI-related CapEx in 2024āa staggering 38% increase year-over-year. This massive investment will go towards GPU acquisition, data centers, and energy infrastructure to support AI operations.
This spending spree will likely continue as long as the scaling laws hold true (more compute + more data = better models). Essentially, the idea is to throw money at the problem and produce better models. As long as these scaling laws don't hit a ceiling, this sets the stage for an arms race towards AGI.