A Deep Dive Into Datacenter And Server Spending Forecasts

Spending on AI systems in 2024 just utterly blew by the expectations of the major market researchers and those who dabble in metrics like we do. There has just been an unprecedented amount of spending, mostly for systems that are accelerated by Nvidia GPUs. It remains to be seen how this spending will hold up, and any forecasts we can talk about today do not necessarily take into account the Second World Trade War that the United States is waging right now.

And so take the following analysis of datacenter server and storage spending and a drilldown on server spending in particular, which is based on data just published by IDC, with a grain of salt. This is meant to complement the drilldown into Gartner’s GenAI spending we did last week. With all of the noise about GenAI, it is helpful to remember that GenAI is not driving even half of the AI use cases out there, even if that day is coming and GenAI could come to dominate as the overall AI landscape.

The IDC datasets were want to examine closely today are for datacenter spending in general among enterprises as well as hyperscalers, cloud builders, and other service providers that build infrastructure for use by others. It has been three and a half years since IDC has publicly released server revenues by type and vendor, and we sorely missed this data and hope to eventually fill in the lost gaps. But, as you can see, the server world of the 2024 that just finished is radically different from the one in early 2021.

Let’s start with overall server and storage spending in the datacenter and then focus down onto servers specifically.

We have used IDC’s Worldwide Quarterly Enterprise Infrastructure Tracker: Buyer and Cloud Deployment tracker in lieu of the missing server tracker to get a sense of datacenter infrastructure spending for the past several years, which as the name suggests keeps track of the combined spending on servers and storage by their capacity deployment model and their customer type. The numbers for the fourth quarter of 2024 have just been assembled.

In the final quarter of last year, spending on systems by shared clouds – what we call cloud builders and what many still call the public cloud, meaning they are shared utilities and not dedicated hosting – rose by 2.24X to $57 billion, which presented 18.9 percent growth sequentially from Q3 2024.

This is a record high spending for infrastructure by the hyperscalers and cloud builders. Spending on gear for dedicated clouds – a more strict hosting model of one customer per server – rose by 21.8 percent year on year to $10 billion, which is also a record high. Combined spending on gear bought in Q4 2024 to be deployed on clouds was a smidgen under doubling to $67 billion.

Obviously, the installation of GPU-accelerated systems at the hyperscalers and cloud builders is driving a lot of the spending on shared cloud infrastructure. How much, this particular tracker does not say. (We can get a sense of that in the other one. Sit tight.)

Non-cloud machines – on premises, single purpose gear still commonly deployed by many enterprises to run back office applications – accounted for $22 billion in revenues, up 25.8 percent and show that the recession for general purpose servers is truly over – or at least it was in Q3 and Q4, it remains to be seen what happens in Q1 and Q2 of 2025.

Here is a handy stable for datacenter server and storage spending from 2020 through 2024:

The growth is accelerating, particularly in 2024 – and a much bigger bump then we saw in 2022 when the GenAI boom first got going. It definitely hit a new level in 2024, and the consensus is that even as spending will cool in the coming years, the floor is going to be higher than we are at now in terms of server and storage spending.

Let’s talk about forecasts for a minute. Here is the one that IDC includes in its public report on the cloud infrastructure tracker:

This chart is interesting, but you can’t really see much here. But if you keep track of the data over time as we do, you can build a much more complete table from which you can see some trends in the past and the new and updated forecasts. Like this:

In this table, we bring everything together since the GenAI boom started in 2022, including a new forecast out to 2029 that IDC has just gazed into its mirrorball to figure out.

IDC used to break down the share of dedicated (hosted) infrastructure that was sold for on premises use and for cloud use, but stopped doing that a little more than a year ago in its public data. It does still break out service providers from server and storage spending by other enterprises, governments, and academic institutions. You can see that in the table above.

If you are thinking that the uptake of GenAI workloads is going to support the growth of spending outside of the hyperscalers, cloud builders, and other service providers – this is what people are talking about – the data from IDC  does not support that premise.

As you can see at the bottom of the table, spending on server and storage infrastructure by enterprises, governments, and academic institutions is increasing at reasonable rates between 2024 and 2029, a compound annual growth rate of 7.6 percent and reaching $108.3 billion in sales by the end of the forecast period. But spending across the service providers is going to grow at a CAGR of 17.5 percent over that period, and  will hit $448 billion by 2029, representing 80.5 percent of spending on servers and storage.

That sure doesn’t sound like IDC believes the 50,000 biggest companies, governments, schools, and research institutions in the world are going to build their own AI datacenters. Much less the next several tens of millions of companies that are even smaller. Look back at 2022: Service providers only accounted for 60.1 percent of server and storage spending. A 20 point swing in seven years is a huge thing. Markets don’t swing that fast usually. In late 2018 and early 2019, server and storage spending was only a few tens of billions of dollars per quarter and pretty evenly split between the tech titans and everyone else.

We keep hearing about datacenter repatriation and sovereign AI, which we think will happen once AI accelerators are more widely available and affordable. But what is clear is that AI is still a rich balance sheet’s game at this point, and companies cannot buy GPUs even if they can afford them. The hyperscalers and clouds are hogging all of the good ones, and Nvidia and AMD are perfectly happy to treat these tech titans with deference because the sale is relatively easy there.

The tech titans are hoping they can make money on AI – by building models or helping those who do or embedding AI functions in their tools and applications – but business are still looking for the return on investment. Once they see that, the next thing they will do is try to have AI cheaper. So we don’t necessarily buy the picture of the future that IDC is painting. Then again, AI is difficult to do well, and is arguably much harder than writing code in COBOL or Java or PHP. If there is an AI skills shortage, then Amazon Web Services, Microsoft, and Google will be able to get customers to try and stay on their AI platforms, which are used to create and deploy AI applications. And customers will pay a premium for that ease of use – as they should.

Let’s talk about servers now. Because thanks to the public release of a portion of the IDC Worldwide Quarterly Server Tracker, now we can see a little bit better just how different the server racket is from how we all knew it back in 2020 and 2021.

The first obvious thing is how the addition of GPUs to a significant portion of the servers that ship has radically boosted revenues in the server market. IDC did not put server shipments in this report, so we cannot calculate average selling prices, but a DGX H100 system using “Hopper” H100 GPUs costs on the order of $375,000 and a rack of “Grace” CG100 Arm CPUs and “Blackwell” B200 GPUs in a DGX GB200 NVL72 system will run on the order of $3.4 million.

By contrast, A decade and a half ago, before the AI revolution started, the average selling price of an X86 server was on the order of $4,000. Yes, there were some big ones running database or data analytics applications, and there were various RISC/Unix and mainframe systems that cost hundreds of thousands to millions of dollars. But there were not very many of them being sold in a quarter. A few hundred thousand, maybe. As GPUs entered the scene for HPC workloads and then AI workloads, those X86 ASPs started rising. And now, om the past few years, the hyperscalers and cloud builders are making their own Arm CPUs and Nvidia is getting a pretty good attach rate for its Grace CPUs in AI systems (and for good reason) and so the non-X86 portion of the market has exploded – even if that Arm CPU only represents less than 10 percent of the cost of a node or a rackscale system.

According to IDC, X86 servers drove $54.8 billion in sales in the fourth quarter of 2024, up 59.9 percent, but as we said, not particularly because of a massive uptake in Intel Xeon and AMD Epyc processors but rather because there is some upgrading going on in the datacenter and these X86 processors do find homes in simpler, less scalable Nvidia GPU machines.

Non-X86 machines, which means a whole lot of Grace in AI systems, homegrown Arm chips at hyperscalers and cloud builders, a smattering of IBM mainframe and Power iron, and a smidgen of Ampere Computing Arm machinery, grew by a factor of 3.62X in the period, from $6.21 billion in Q4 2023 to $22.52 billion in Q4 2024.

Back in the 2000s, when RISC/Unix iron was still used for Internet infrastructure as well as for back office systems and there was a lot more money in IBM mainframes, non-X86 iron represented between 50 percent and 60 percent revenue share, but the Dot Com bust weakened the Unix market and Linux on X86 rose up to be the cheaper alternative for Internet workloads. So that share of non-X86 revenue bled down to around 10 percent by the early 2020s. And boom, just like that, non-X86 iron comprised 15.3 percent of the server market in Q4 2023 and 29.1 percent in Q4 2024. And, it won’t be long before an Arm CPU is in more than half of the server nodes being installed in a quarter.

Incidentally, IDC says that more than half of the revenue in the server market in Q4 was driven by GPU accelerated servers, and then 90 percent of those machines had Nvidia GPUs in them. Our best guess is that it was around 54 percent of sales, which is $41.75 billion in machinery, and given the 2.93X increase in sales year on year, that would put GPU-accelerated server sales at $14.27 billion in Q4 2023.

The other big change in the server rankings is who is selling iron. Dell is still number one, but is just barely beating out Supermicro. Dell has a steady if wiggly enterprise server business thanks to its PowerEdge line, and is getting some low-end HPC business and some high-end AI business. (Like the deal with xAI that it shared with Supermicro for the “Colossus” machine in Memphis.) Hewlett Packard Enterprise, thanks to a good HPC system business and its share of AI system sales as well as the workhorse ProLiant servers, is hanging in there at number three, but IEIT Systems, an upstart Chinese server maker riding up the AI wave using Chinese CPUs, GPUs, and AI accelerators, is rising a little bit faster. Lenovo is hanging in there much as rivals Dell and HPE, with some HPC and AI sales and steady ThinkServer sales.

IBM and Cisco Systems are long out of the top five, and they will probably never return.

The ODM’s that make machines for the hyperscalers and cloud builders have been the big beneficiaries of the first wave of AI and now the second wave with GenAI, as you can see. The ODMs together raked in $36.57 billion in sales, up 2.55X over the prior quarter, and utterly dwarfed the top five OEMs combined. And the rest of the market, which includes specialist manufacturers as well as IBM, Cisco, and Nvidia itself, brought in $18.31 billion, up 65.9 percent. Mostly thanks to Nvidia, we think.