Every time Lisa Su, chief executive officer at AMD, announces a new Instinct GPU accelerator, the addressable market for AI acceleration in the datacenter seems to expand. But, we observe, the growth rate in the AI datacenter accelerator forecast has slowed even as the ramp rate for AMD and Nvidia GPUs is speeding up.
Based on the prognostications of Su, then the inevitable math is that the world is going to consume somewhere around $700 billion to $900 billion in datacenter AI accelerators in the six years encompassing the first wave of the GenAI boom (2023 through 2028). How much depends on which AMD forecast you use. This implies worldwide spending on AI systems on the order of $2 trillion (we will show you that math in a second), and those AI platforms in turn are going to eat the world.
Hopefully not literally.
It is always important to realize that correlation is not causality, so the slowing growth for AI accelerator revenues in the latest AMD forecast is not being inversely caused by AMD’s and Nvidia’s success in launching more and more its GPUs into the market. But then again, Carl Jung, protégé of Sigmund Freud, correctly argued for synchronicity, an acausal connecting principle that may exist only in the minds of higher order beings and cautioned that it is important to not take meaning and timing too far.
Sometimes, especially on a Philosophical Friday, it is hard to say much about reality even if it is simple enough to just observe it and react to its actions. Perhaps it is just unnecessary or, better still, insufficient to say anything. . . .
But, that’s how we pay the bills around this joint, and so we will take a look at Su’s evolving TAM for AI acceleration, which was augmented again last week at the Advancing AI event in San Francisco, which will result in the enormous investment that Earth is expected to make in AI over a six year span.
A Rising TAM Lifts All Accelerators
Back in June 2023, at the Datacenter and AI Technology Premiere event where then-impending enhancements of AMD CPU and GPU compute engines were previewed, Su & Co gave a baseline prediction as they were unveiling their aspirations for their third generation “Antares” datacenter GPUs coming later in the year:
If you zoom in on the fine print on this chart, it says “Datacenter AI Accelerator TAM: GPU, FPGA, and others” just so we level set on what we are talking about.
And suffice it to say, FPGAs are not even noise in the data at this point because the GPU accelerator market has become so large for AI workloads, and others is mostly comprised of upstarts and may or may not include the value of the homegrown accelerators made by Google, Amazon Web Services, Microsoft, Meta Platforms, and others.
Only six months later, when the “Antares” MI300A and MI300X accelerators were launched and when the Big Bang of the Gen AI boom was first felt en force, Su massively revised that datacenter AI projection upwards again.
Not only is the baseline revenue level in 2023 higher than expected, but the compound annual growth rate (CAGR) between 2023 and 2027 inclusive of 70 percent was also much higher. And the end result is that by 2027, AMD expected more than $400 billion in accelerator sales
That brings us to the reveal for the MI325X and the MI355X, which happened last week at the Advancing AI event and which saw Su give another forecast, this one running out to 2028 instead of 2027:
The $45 billion in datacenter AI accelerator sales remains the same, but now endpoint revenue is projected to be $500 billion and the CAGR is calculated with 2028, bit 2027, as the endpoint. And that CAGR is lower perhaps as the boom is settling out a bit as the extra year is tacked on. So the growth rate is only 60 percent instead of 70 percent. Or, Su & Co are revising their expectations downward but tacking an extra year on the forecast to draw attention away from that downward revision.
Important Note: When you are given two endpoints and a CAGR, you can’t really know what goes on in the forecast between those two endpoints. It is a bit of a black box. It could go up and down a lot in the middle and the resulting CAGR would not change. This obviously limits the appeal of CAGRs, but this is the best tool we have for forecasting and it implies you are thinking linearly between those two endpoints and that any differences will average out over a long enough time.
So, if you assume linear growth between the two endpoints in these three AMD forecasts for datacenter AI accelerator spending, this is what they look like after you fill in the blanks in the middle:
The estimates for the middle years, shown in bold red italics, are from us and use the CAGR to show the annual growth.
There are two possible scenarios for fleshing out this model.
Scenario 1 keeps the December 2023 forecast for AI accelerator sales from 2023 through 2027 and then adds the new endpoint for 2028. We don’t think this is what AMD was trying to say, and if it was, that would imply, as you see in the chart, a pretty dramatic slowdown in AI accelerator spending in 2028.
Scenario 2 is a revision of the model, and we think this is what AMD meant to do. (Otherwise, the lower CAGR would not work out.) As you can see, $156 billion in AI accelerator sales between 2024 and 2027 inclusive have been removed from the forecast, dropping the cumulative sales by $156 billion in the October 2024 forecast compared to the December 2023 forecast.
AI accelerators represent around half of the cost of the systems that use them, and networking is another 20 percent on top of that, give or take. That estimate is based on having AI servers that have lots of flash and main memory and hefty host CPUs to drive the serial portions of the AI workload. Add another 20 percent or so on top of that to reckon the networking cost for the AI cluster built from AI servers.
If you do that math against $737 billion in AI accelerator spending from 2023 through 2028, that is roughly $1.5 trillion in AI server spending, and with networking, it works out to $1.8 trillion. Toss in systems software and you can call it a cool $2 trillion.
If this is how companies are behaving now and how they will behave in the future, it is no wonder that there is a server and storage recession outside of AI server spending.
Now, here is the final consideration. If the world is going to spend $2 trillion on AI systems over six years from 2023 through 2028, the bean counters of the world are going to expect to get an ROI on that. That is another way of saying that we will want to remove costs in the economy as well as boost revenues in the economy that add up to at least $2 trillion. Probably an order of magnitude multiple of that, in fact. Hopefully, this will be less about eliminating people than doing things differently, but our best hopeful guess is it will probably be about half and half. This might all be about eliminating people, which should not surprise any of us given how people are so needy and so compared to an AI system.
It is important to remember that if no one has a job, then no one pays taxes, and therefore no one gets a universal basic income, which is not worth a damn if you believe in the optionality that a real economy, with people, gives the world. It is funny how those who are espousing UBI so loudly have made sure they got theirs first. We will be working, and working hard, until we are dead. We strongly suspect you will be, too. Hopefully out of our own free will, and in a place of our choosing.