AI To The Rescue For Server And Storage Spending In Q1

Here’s a fun question and don’t cheat by asking ChatGPT. What is more valuable, an ounce of gold or an ounce of an Nvidia “Hopper” H100 GPU accelerator?

The two are closer in price than you might imagine, but it looks like gold is still winning and it also looks like the GPU is more valuable by the ounce than silver, the other commodity precious metal. Nonetheless, the value of these and other GPUs, which are being added to AI training and inference systems as well as to HPC clusters, was the main force driving revenue increases in the server and storage racket.

IDC has just released its consolidated server and storage revenue estimates for Planet Earth and has revised its forecasts for 2023 as well as out to 2027 for sales of server and storage infrastructure by use case and industry sectors. And that got us to thinking about gold and silver bars and the digital gold that companies like Nvidia and AMD – and for its own sake we hope Intel, too – are mining in them there Big Data hills.

The fattest PCI-Express version of the H100 accelerator weighs in at 1,286 grams, which is 45.4 ounces. It probably has a street price somewhere around $20,000, but who can say with so many big buyers with big bucks chasing whatever limited supply Nvidia has. That works out to $441 per ounce, which is about where gold was trading way back in 2005. Gold is trading at around $1,929 per pounce as we go to press, and is therefore about 4.4X as valuable per unit of weight than an H100 PCI-Express card. Silver, by contrast, is shiny like a new server, but it is only trading at around $23 per ounce this afternoon. Like GPU accelerators, gold and silver are commodities that are precious because they are relatively rare, but the difference is that the value of the GPU is not just passively rising and falling based on availability and it can be deployed with software to actually create new value in its own right.

This is the entire thesis behind a massive buildout in AI systems that is propping up the server and storage rackets in Q1. Without it, volumes would be down some and revenues would probably be down a lot more, too. We don’t know for sure because IDC is not breaking out server and storage sales acquired to run AI workloads separately from other workloads. But it did say that cloud infrastructure sales outpaced non-cloud infrastructure revenues in the first quarter, as has been the case for a few years now, and that the cloudy portion of server and storage sales – the strongest and largest portion of the datacenter infrastructure pie at this moment – had unit shipments fall off by a pretty dramatic 11.4 percent, but that average selling prices rose by 29.7 percent thanks to price inflation (server makers feel safe to pass on some costs finally) and “a higher concentration of GPU-accelerated systems being deployed by cloud service providers.”

We think that GPU-boosted servers have been propping up server revenues for a little more than a decade in a material way, and that this has been increasing as the AI revolution has built up its steam. Clearly, in the wake of large language models and deep learning recommendation models going mainstream, demand has exploded, and the cloud builders and hyperscalers are buying as many as they can to try to chase the AI opportunity. It is fair to say that because of supply issues and the high cost of AI training systems, a lot of organizations trying to fit AI into their workloads will have to start in the cloud, which sets up a virtuous cycle that favors the cloud and their rental model over buying systems and installing them on premises – with notable exceptions where large enterprises and startups have the cash to buy their own and not have to pay the cloud premium to get GPU capacity to train their models. This tension will force many to start with pretrained models and tweak them, which is probably a good thing given how long it takes to train a model and the dearth of hardware and people to make it happen.

In the first quarter, which is the most recent one for which data is available from IDC, sales of servers and storage for any and all workloads and to any and all customers added up to $35.3 billion, an increase of 8.2 percent year on year. It is important to remember that this is not a gauge of the revenue that cloud builders and other service providers derive from renting out capacity on the machines they buy.

Breaking it up by delivery model, sales of servers and storage for shared clouds – what we call cloud builders and what many still call the public cloud even though these are not really public utilities at all – rose by 22.5 percent to $15.7 billion.

IDC breaks out another segment of the market into dedicated clouds, which includes servers and storage sold for hosting applications in a more static fashion among service providers as well as gear sold by OEMs under rental agreements (such as Hewlett Packard Enterprise GreenLake, Dell APEX, Lenovo TrueScale, and Cisco+). Sales of iron to support dedicated clouds fell 1.5 percent to $5.8 billion and still represents a relatively small part of the market. But don’t get the wrong idea. In the trailing twelve months, sales of gear into dedicated cloud use cases have risen by 18.7 percent compared to 21 percent for shared cloud infrastructure.

Add them up, and in Q1 2023, sales of servers and storage into all cloud use cases rose by 14.9 percent to $21.5 billion. And sales of non-cloud, legacy platforms – think IBM System z mainframes and Unix and proprietary systems running databases and transaction processing systems – were down nine-tenths of a point to $13.8 billion. In the trailing twelve months, though, sales of these non-cloud systems are up 10.1 percent.

The question now is this: Will increasing interest rates by the world’s central banks as well as the general uncertainty of the national and regional economies cause sales of servers and storage to taper off in 2023, or will they power through on underlying growth in existing applications and add in the emerging AI bubble to do quite well in 2023 and beyond?

If sales of non-cloud gear is a leading indicator, then it doesn’t look good. IDC is projecting that in 2023, server and storage revenues from this class of use case will fall by 6.3 percent to $60.4 billion. We don’t think this is a very good leading indicator, but more a reflection of the upgrade cycles for IBM System z and Power Systems iron as well as for other X86-based systems for doing similar work.

While the overall server and storage market is projected to grow by only 1.1 percent this year, to $156.8 billion, sales of cloudy infrastructure are expected to rise by 7.3 percent to $96.4 billion, and within this, shared cloud infrastructure – the stuff sold to Amazon Web Services, Microsoft Azure, Google Cloud, Meta Platforms, Alibaba, Tencent, and Baidu – is expected to grow by 8.4 percent to $68 billion. Dedicated cloud infrastructure will grow at a more modest 4.8 percent to $28.4 billion.

Sales of non-cloud gear to enterprises, government, and academia seem to be the culprit, if you do some Venn diagraming of the data. Here is a consolidated table we created based on the updated IDC data for 2022 plus forecasts in 2023 and 2027:

Server and storage spending into enterprises, governments, and academic institutions is projected by IDC to fall by 5.1 percent to $62.3 billion this year, which is not good news and it also suggests that when it comes to AI, a lot of these organizations are looking to the cloud to get started. This makes sense given the double uncertainties concerning the economy and the way to weave AI into applications.

The overall forecast for server and storage spending worldwide between 2022 and 2027 remains pretty rosy, according to IDC, with sales expected to have a compound annual growth rate of 6.2 percent and hitting $221.7 billion by the end of the forecast period. Sales of cloudy infrastructure in 2027 will be, as you can see from the table above, a little bit smaller than all infrastructure in 2022, which is remarkable in its own right. But again, a lot of that growth is coming through investments in AI training and inference hardware, whether it is for on premises use or use through a cloud.