Linux Is The Next Platform, But Who Pays To Maintain It?

Red Hat has once again dropped another huge boulder into the normally serene – or at least relatively calm – open source waters.

Back in December 2020 it killed off the CentOS distribution that lives downstream from Red Hat Enterprise Linux and created the CentOS Stream variant that lives upstream where bugs are not yet all shaken out. And now Red Hat has announced that it would no longer distribute free RHEL source code to those who aren’t paying customers.

That means projects like Oracle Linux, EuroLinux, AlmaLinux, and Rocky Linux no longer will see code improvements from Red Hat, a change in a practice that is more than a decade old. Unsurprisingly, those projects and others reacted with fury to Red Hat’s decision, accusing the open source titan of essentially closing its doors to the open source community and accusing IBM of driving the move.

Big Blue bought Red Hat for $34 billion in 2019, so believing IBM would be behind the decision isn’t surprising, though it does give it all a conspiratorial vibe to it.

In a blog post June 21, Mike McGrath, Red Hat’s vice president of core platforms, noted that more than two years ago, the company made CentOS Stream the “focal point for collaboration” around RHEL, which shrank the feedback loop with partners, customers, and other open source communities and brought greater visibility to RHEL development. Now Red Hat will no longer release RHEL’s code; instead it will be CentOS Stream code, though Red Hat customers and partners still have access to the source code via the company’s Customer Portal.

A key reason for all this? It takes a lot of money to create new features, fix bugs, integrate packages, and provide support. In another blog post responding to the tsunami of criticism, McGrath said the anger from downstream projects is coming from those who don’t want to “pay for the time, effort, and resources going into RHEL” or want to repackage it to make a profit.

“We have to pay the people to do that work — those passionate contributors grinding through those long hours and nights who believe in open source values,” he wrote. “Simply repackaging the code that these individuals produce and reselling it as is, with no value added, makes the production of this open source software unsustainable.”

To be sure, the criticism of Red Hat and IBM has been harsh. Oracle and SUSE – which last year introduced its RHEL-compatible Liberty Linux distribution – both said this week that they will fork their enterprise Linux OSes, with SUSE saying it will throw at least $10 million behind the effort. Oracle Linux will remain RHEL-compatible through release 9.2, but after that there are no guarantees.

Oracle accused IBM of putting its own financial needs before the good – and standard practices – of the Linux community. Edward Screven, chief corporate architect at Oracle, and Wim Coekaerts, head of Oracle Linux development, wrote that CentOS was a popular – and free – RHEL-compatible distro before Red Hat/IBM killed it as a free RHEL alternative, shifting the focus to CentOS Stream, which was announced months after the IBM deal for Red Hat closed.

They also said Rocky Linux and AlmaLinux are alternatives to RHEL and, thus, are the real targets of Red Hat’s action.

“IBM doesn’t want to continue publicly releasing RHEL source code because it has to pay its engineers?” Screven and Coekaerts wrote. “That seems odd, given that Red Hat as a successful independent open source company chose to publicly release RHEL source and pay its engineers for many years before IBM acquired Red Hat. . . . By withholding RHEL source code, IBM has directly attacked. And perhaps that is the real answer to the question of why: eliminate competitors. Fewer competitors means more revenue opportunity for IBM.”

To Greg Kurtzer, who founded Rocky Linux (as well as CentOS and the Singularity Kubernetes container controller project), Red Hat’s actions don’t much change things on the ground. A week after Red Hat’s announcement, the RESF outlined two ways to obtain source code: UBI container images, which are based on RHEL and available online through sources like Docker Hub, and by using pay-per-use public cloud instances, through which anyone can create RHEL images in the cloud.

Kurtzer tells The Next Platform that Red Hat tried to disrupt downstream variants of RHEL – even going so far as to banning customers from distributing the code – it won’t work. He also says that 95 percent or more of RHEL source code comes from other upstream sources anyway. The immediate effect is limited.

“It feels like it was much, much ado about nothing because it took us just a little while to just figure out,” he says. “We’re still in the green light, according to the GPL and according to Copyleft and GNU. We just have to go to a different place to get the sources. It’s a little bit more of a pain. We’ll figure out how to automate it, but our users are completely non-disrupted. Nothing’s really changed for our community, so everything just continues on.”

That said, the ripple effects of the decision will continue to be felt. While Red Hat may not have thrown a huge wrench into the open source engine, the company violated the spirit of open source development and breached the community’s trust. Much of RHEL consists of code from hundreds of other projects and thousands of non-Red Hat contributors, all of whom believed coding would be freely available.

Now Oracle and SUSE are both promising to fork their enterprise Linux distributions and Kurtzer says he’s heard others might as well.

“Even though Red Hat’s move wasn’t a big catastrophic interference, it created concern and created turmoil and drama in the community,” he says. “For that reason, there are people now concerned. ‘Can Red Hat do more?’ is the question that I keep hearing. What happens if the Red Hat’s next shot is more hurtful? What happens if the next one is something that actually takes us and breaks Rocky, Oracle, Liberty Linux? What happens if what they do next is more catastrophic?”

The problem is with commercial companies having control of community projects. Kurtzer in 2020 founded CIQ to build an HPC platform that enterprises can also adopt for such workloads as AI, machine learning, and analytics. Central to the effort was creating a Linux OS for the platform, in this case Rocky Linux, as an alternative to CentOS, RHEL, Oracle Linux, and others. The initial idea was to use Rocky Linux for CIQ, but the open source community pushed back against the idea of another company-driven open source project. Thus the birth of the RESF later that year.

Three years later, Red Hat’s action puts a spotlight on the need to protect these projects from the financial needs of commercial IT firms. Kurtzer notes Red Hat’s control of CentOS – now CentOS Stream – and Fedora, adding that companies like like MongoDB, Elasticsearch, and VMware – with the Tanzu Community Edition – also made decisions about open source technologies based on business needs.

“I don’t mean to speak badly about any of these companies but it’s really hard to manage an open source community and project from behind a locked door of a company,” he says. “It’s just a matter of time before that company is going to make decisions on behalf of that community that benefit the company, not the community. It’s a hard balance. The best way of doing this is to ensure that open source projects stand separate from commercial entities, and there needs to be a mechanism for doing that.”

Red Hat’s move and the decisions by Oracle, SUSE, and possibly others to fork their distros threatens the compatibility and stability needed to maintain the enterprise Linux ecosystem, he says. There needs to be an enterprise Linux standard free of corporate control.

“The future is going to be around the standard. That’s how we’re going to end up solving these problems moving forward,” Kurtzer says. “The community really needs freedom of choice, but they also need freedom of choice around a compatible standard. That to me is the most critical piece. If you don’t have that, you have single vendors creating vendor lock-in and control. Customers want that freedom. They want to know that there’s competition in the market. Competition sucks for the vendors, it’s great for the customers and it’s great for the community. We need more competition. We need more choice.”

With Red Hat’s move and Oracle and SUSE rattling their forking swords, time becomes important to developing that enterprise Linux standard. Kurtzer says RESF is working on an announcement about what it will do to help create a standard.

“We are in a little bit of a mad dash right now to try to align,” he says. “It’s tricky. We’re seeing vendors that really want to get a message out there, whether they’re trying to do something good for the community [or] whether they’re trying to lead due to business interests. It is important to get in front of this. We are trying to push something out very, very quickly now, but we’re also trying to build a standard that everybody’s going to appreciate. I don’t want to do something that would take value away from Oracle or take value away from SUSE or anybody else. I’m not suggesting that the RESF itself will be hosting this at all. I don’t know who’s going to be hosting this, but I can say that from a goals perspective, a mission perspective, this is where we’ve always been thinking.”

“This should not be a single company and it shouldn’t be multiple single companies. It should be a coalition of companies all coming together and working together. If we do that right, I think we solve the problems for the both immediate- and long-term future and will bring stability to the ecosystem. If we don’t do this right, I don’t think we’re any worse off than we are right now.”