No More Roach Motels For Data In The Clouds

The central tenets of cloud computing, which really ought to be called utility computing, is that you only pay for what you use and that you can turn compute, storage, and networking off when you are not actually using, thus freeing up capacity for those who need it.

It is in this way that clouds are supposed to be more efficient than having millions of companies buy their own systems, running them all the time but having very poor utilization of the assets they acquire to run their workloads at their peaks.

The establishment of so-called egress charges – the fees that the clouds charge to move data between their regions or out to other clouds or on-premises equipment – have never violated our sensibilities because if you are using the cloud’s network to move data around, why shouldn’t you pay for it? It is only because clouds did the sensible marketing thing and decided not to charge customers to move their data into a cloud – what are known as ingress charges – that the incongruity and the unfairness was introduced.

Not charging for data ingress but charging for data egress makes perfect sense in a way, but probably not to any antitrust regulators who took a good look at the big clouds. As a marketing tactic, it makes perfect sense to not charge companies to bring their data into your cloud, and it makes perfect sense to punish those companies who use the network to link compute in storage on the indigenous cloud to either other clouds, or other regions where datacenter interconnect (DCI) overhead is large, or to private datacenters or co-location facilities. Amazon Web Services has always been keen on companies going “all-in” on its cloud, and of course it wants to charge customers who move data off of its infrastructure and of course it wants to have those who are moving data off the cloud subsidize the network costs of those who are not.

Cooperative apartments in New York City and other major cities often implement what is called a flip tax, which is assessed on the sellers after a they find a buyer of their apartment, and it is a means of paying for infrastructure for current users by taxing those who have benefitted from that infrastructure and presumably are selling at a higher price than they paid for their apartment. Some flip taxes are on the sale price – a flat tax of sorts – and some flip taxes are only applied to the capital gain on the apartment, should there be one. This is the same kind of idea as an egress charge in the cloud.

Google made a bit of noise this week as it removed egress charges for Google Cloud customers who want to move their data and presumably their applications off Google Cloud and park them elsewhere. The removal of egress fees is not a blanket removal, but one you have to apply for if you are literally moving off Google Cloud. And it only applies to those who are leaving Google Cloud, not to those who are operating applications and storing data in a hybrid fashion across many clouds or a mix of clouds and on-premises gear.

We have never liked the “Roach Motel” nature of cloud storage, and while Google is addressing a problem, it is only addressing part of the problem.

For those of you who are unacquainted, pesticide maker Black Flag had a series of commercials in the early 1980s pitching its Roach Motel – “roaches check in, but they don’t check out!” – and the scented glue trap was successful enough that it is still around today.

In a world where data gravity means data is a lot less portable than compute, and modern applications generate and store a tremendous amount of telemetry and other data that can feed into all kinds of systems to drive efficiencies or revenues, cloud storage is a kind of Roach Motel.

It is clear that any network has to be paid for, but frankly compute and storage isolated from each other is perfectly useless, so maybe the compute and storage pricing should carry the cost of the network and no cloud should charge for an ingress or an egress charge. Or maybe the egress charge and ingress charges should be the same and any data generated on a particular cloud should be free to roam around a region on any cloud with nominal fees for data transfers between regions, other clouds, or on-premises storage.

We are not talking about a lot of money, at least on paper. Somewhere between 5 cents and 9 cents per gigabyte may not sound like a lot, but even at the cheapest rates of 5 cents per gigabyte for high capacity egress, it would still cost $50,000 to move 1 PB of data out of AWS or Azure; at Google Cloud, it would have been $80,000 at current pricing. If there is a lot of back and forth of data exchange, the costs can add up pretty fast, and this is the real issue as far as we are concerned. Moving off a cloud – and needing to get permission to do it for free – is not the issue.

Google has the right idea, but for the wrong reasons.

To be fair, the cloud providers can’t easily bury the costs for the networks that link their systems within datacenters, within regions, and across regions because they cannot always anticipate how companies are going to use the cloud infrastructure or collections of infrastructure across clouds and on-premises. It is not as simple as raising the cost of virtual machines and storage by 15 percent or 20 percent hoping that covers the cost of the networks that create hierarchies of interconnects for compute and storage.

It makes sense to go back to first cloud principles, then. If you use it, you pay for it, and if you shut it off, you don’t.

And thus, we advocate for clouds to actually have data transfer charges within a datacenter, across a region, across regions, and to outside infrastructure. These charges can correspond to their actual costs, and frankly, they should. And if what people have called public clouds – and which are private enterprises designed to make big profits that absolutely do not have any kind of regulation or oversight and which we simply call cloud builders because they are not regulated utilities like those who provide water, electricity, and transportation – were indeed public, there would be rules about how this works.

Common sense should be enough, and if more clouds do the right thing, then all of the other clouds will have no choice but to follow. Just like they all didn’t have ingress charges and they all had egress charges because that is what AWS did when it started off in 2006.