Disaggregation Busts Open Switch Operating Systems

In case you haven’t noticed, the switch at the heart of the datacenter is being torn apart and commoditized at all levels. The so-called open networking movement, being fomented by hyperscalers and the large enterprises that want to be like them, want to tear networking software free from hardware. But the shredding is not going to stop there. Now, the networking software stack is being pried apart and opened up, too.

This is the contention of Dell, which likes commodity hardware with a plethora of software to run atop it and which has been an early and enthusiastic supporter of the open networking movement for the past three years because it wants to do to Cisco Systems and other very profitable switch appliance makers what the X86 server collective did to proprietary and Unix systems – relegate them to the margins and eat their revenues. But, this is not something that will happen overnight with switching, any more than it did with systems.

“We are well down the path,” Jeff Baher, senior director of technical marketing for Dell’s networking division, tells The Next Platform. “We are at the beginning of a ten year to fifteen year transition that is similar to the transition from mainframes to client/server, where we saw the disaggregation of elements of the mainframe – CPUs, operating systems, middleware, virtualization, and the applications that ride on top of them. We see networking going very much that way, and we think that networking has been held back because of the strong linkage between the hardware and the software. For more complex networking devices, such as the high end routers that we do not develop, they have a large amount of proprietary hardware and it is hard to break the linkages between that hardware and the software that is driving that. But there are other switch environments based on fixed form factors and merchant silicon, and the software requirements for switching are pretty well known, and as a result, switching is a better candidate for this level of disruption.”

You Have To Break Up To Make Up

Dell bought Force 10 Networks, one of the upstart switch makers alongside of Juniper Networks, Arista Networks, Extreme Networks, and a few others, back in July 2011 when it had 1,300 customers and close to $200 million in revenues on a trailing twelve month basis. These days, Dell has over 22,000 customers who have bought its switching gear in one form or another, which is a reasonably large installed base. Baher says that over 400 of Dell’s customers have started down the open networking road using various combinations of switching gear based on merchant silicon and a variety of open software from Cumulus Networks, Big Switch Networks, Pluribus Networks, and IPinfusion.

“These are the high rollers, the largest of the large, who have the kind of scale where in some cases they can develop and manufacture their own silicon or switching systems and be able to write their own applications. That is obviously not for everyone,” concedes Baher. But, the customers on the next level down in the datacenter scale hierarchy from the biggest hyperscalers and cloud builders who have created their own switches and operating systems – big financial institutions are the best example – want some of the same flexibility that these behemoths have in their stacks. “These customers can’t wait for the next iOS or NX-OS release from Cisco or Junos release from Juniper.”

Therefore, Dell is busting up what used to be called its FTOS network operating system into something that looks and smells like a Linux distribution. The new OS10 will be the operating system that Dell will use on its datacenter switches, replacing the monolithic OS9 (formerly FTOS), and on its campus switches, replacing another monolithic networking operating system called OS6.

The OS10 code is a brand new operating system, created from the ground up on a Debian Linux kernel and made to look and feel as much like a Linux server platform as possible, except that it runs on merchant networking ASICs instead of X86, ARM, or PowerPC processors. Baher says that Dell has architected OS10 so the underlying Linux kernel can be swapped out as needed or as customer preference dictates, and it started with Debian Linux because it has to start somewhere. (With Ubuntu Server being a Debian distribution and being the most popular Linux for OpenStack clouds, this is an obvious place to start.) Like many current network operating systems, the prior OS9/FTOS was based on the NetBSD variant of Unix, which was the popular starting point a decade ago; some use a variant if VxWorks as their foundation.

With current Linux-based network operating systems – and there are a bunch of them such as the Extensible Operating System from Arista, OpenNetLinux from Big Switch, PicaOS from Pica8, Cumulus Linux from Cumulus Networks, and more recently OpenSwitch from Hewlett Packard Enterprise – there is a tendency to modify the Linux kernel, but Dell wanted to keep it as close to the server version of the kernel as possible so that a switch can be managed just like any other kind of server. (This is what Facebook and Google do, and presumably Amazon Web Services, too.)

And thus the foundation for OS10, which is called OS10 Base, is just a trimmed down Linux kernel that has been hardened for networking with Control Plane Services (CPS) APIs above it and platform abstraction services through the Switch Abstraction Interface (SAI) layer that Dell and Microsoft created together and open sourced through the Open Compute Project last year. This OS10 Base network operating system is freely distributed by Dell and has its roots in the Linux and other open source networking communities, but Dell is stopping short of open sourcing the OS10 base stack. The CPS control plane layer could be contributed back to the community, says Baher, and SAI is an open specification with code contributed by Microsoft and Dell already, and we suspect that if a big customer makes a fuss and wants all of the source code to OS10, Dell will comply for a fee. The OS10 Base distribution includes three years of support from Dell when it is loaded on its own switches.

On top of this NOS foundation is what Dell is calling the OS10 Premium package, which provides the kind of Layer 2 and Layer 3 switching and routing protocols that are bundled inside of FTOS/OS9 for datacenter switches and OS6 for campus switches as well as management tools and policy controls that determine how Ethernet traffic is shaped as it flows through the switch. The Quagga and BIRD routing stacks are among the applications that can be added to the OS10 Premium Package, as can the NetSNMP management tools and Nagios monitoring tools; other commercial networking packages can be plugged into OS10 Premium as well, and Dell is working on a bunch of these now, including load balancers, firewalls, and BGP gateways as well as Ansible, Chef, and Puppet. A mix of new code and existing FTOS code that was ported over from NetBSD Unix made up the Dell elements of OS10 Premium. There is also an interesting opportunity to bundle some of the network function virtualization (NFV) elements of a Linux/OpenStack cloud onto a switch running OS10, but the switch will need more than a switch ASIC to do such work – likely a Xeon or ARM server chip.

The important thing is that the resulting stack will look and feel like FTOS to Dell’s users, even if its components have been broken up into piece parts, ported to a Linux distro, and running alongside a lot of other software on the switch.

While Dell is not providing pricing for the OS10 Premium bundle, Baher says that it will cost about the same as the current OS9 does on its existing S-Series top of rack switches and Z series core and aggregation switches. The software will be available on the S-Series S3048, S4048, and S6000 switches first, and the Z9100 and Z9500 big bad switches will get support by the end of 2016. The OS10 network operating system is designed to be agnostic to the network processor (sometimes called an NPU, usually just called an ASIC) thanks to the adherence to the SAI API abstraction layer, but SAI only supports certain ASICs at the moment. So OS10 will in theory be able to support ASICs from Broadcom, Marvell, Cavium, and others, so long as they have an ONIE-compatible NOS boot loader. But for now, OS10 Base supports Broadcom Trident-II and Trident-II+ chipsets with support for the Tomahawk chips designed for hyperscalers coming later this year.

OS10 Base is in beta now, and will be production grade by March. At that time, OS10 Premium will go into beta, and sometime in the summer, after a code shakedown with earlier customers, it will follow into production.

Dell’s partners in network operating systems are probably wondering how they are going to compete against Dell in the software space. The answer is simple: by the merits of their code. That will be the same answer in storage after Dell completes its $67 billion takeover of EMC and partners such as Nutanix wonder where they stand. Ditto for server virtualization and cloud once Dell has control of VMware. Dell wants to sell stuff to customers, with Microsoft being a very attractive account for infrastructure.