When Diverse Network ASICs Meet A Unifying Operating System

It has been two decades since Juniper Networks, then the big upstart rival to Cisco Systems and others as the dot-com boom was rising towards its crescendo several years hence, took FreeBSD Unix and turned it into a network operating system that spanned both routers and switches. It has been a decade since Cisco itself took the operating system at the heart of its MDS Fibre Channel switches (itself based on Wind River’s real-time Linux) for storage area networks and transformed it into NX-OS for its Nexus line of switches. And it has also been a decade since switch upstart Arista Networks launched its Extensible Operating System, or EOS, which is derived from Linux.

Every decade or so, datacenter networking gets a new operating system that shakes things up, and ArcOS could be the next big thing. That is certainly what the founders of Arrcus, the company behind this closed-source operating system for switches and routers that emerged last summer, believe.

Since the beginning, switches and routers were like mainframes, minicomputers, and even Unix systems in that the maker of the switches – often also the maker of the switch or router ASIC, too – was also the designer of the operating system that ran on the devices. The big change that happened two decades ago is that switch vendors decided to have one operating system span all of their devices – something IBM figured out with the System/360 mainframe in 1964 and then promptly forgot as its platform proliferated in the following decades. And as network operating system makers Cumulus Linux and Big Switch Networks started to get some traction a decade ago as companies wanted to have more control over the network software stack, like the hyperscalers and cloud builders who have created their own NOSes because of the frustration of incompatibilities across devices, many of the upstart switch makers like Dell (Force10 Networks), Hewlett Packard Enterprise (3Com), and Mellanox Technologies open sourced their NOSes to try to catch the open network wave.

Open networking has been by and large fomented by hardware that can run the homegrown NOSes created by the hyperscalers and the cloud builders, and by increasingly powerful switch ASICs from merchant silicon providers that are adding more and more routing functionality to their devices. It is a virtuous cycle that has been building towards a software platform that can scale across all of this disparate hardware and get customers out of vendor lock in.

It’s natural enough to ask why now is the time for a new network operating system to emerge in the datacenter. So that is precisely what we did with Devesh Garg, chief executive officer and co-founder at Arrcus and one of the speakers at out Next I/O Platform event in San Jose on September 24, which you can still register for here if you hurry. Garg worked at National Semiconductor, Broadcom, Tilera, EZchip Technologies, and even spent nearly nine years at Bessemer Venture Partners funding other companies, including PA Semi. Having been in the industry for some time, Garg gave a philosophical answer, which is good.

“When I was a kid, I read a ton, and for some reason I always gravitated towards history,” Garg explains. “It started out with military history, and then I got into various flavors. That interest in history coupled with all of the time I spent in the systems and semiconductor world, added to the perspectives I got from my stint with Bessemer, made me realize that in every industry, when you look at it from a technology perspective, closed, proprietary, vertical integration leads to best-in-class horizontal segmentation.”

As an example, Garg recalls his time at National, which was founded in 1959 and which became part of Texas Instruments in 2011. In the beginning and for many decades, National owned the chip designers, the libraries, the intellectual property, the fabs, the assembly and test houses, as well as all the operational aspects of running a chip business and started with sand and ideas at one end and out came chips at the other end. If you fast forward to today, very few semiconductor companies own their fabs. Chips companies excepting Intel and Samsung, are mostly engineers that design chips and adjunct people to sell and support them. They use Cadence and Synopsis tools, they use third party memories, PHYs, and SERDES, and they have third party assembly and test houses put their chips together after they come out of third party fabs like TSMC and GlobalFoundries.

“We have history on our side, which basically says that that the marketplace will gravitate towards best in class solutions,” Garg continues. “And best is not just defined in one dimension. It is a combination of technology, usability, business flexibility, and cost. So however one chooses to define “best,” people will ultimately gravitate towards that. And so I thought that the timing was right and the forcing functions that were being created by the user demands in a hyper-connected world coupled with this pent up capability from all the advancements on the silicon and hardware side, wanted a release. We just felt that software was the missing piece.”

Many have thought this before, and the handful of open network operating systems have not really done all that well. Part of the reason, we think, is that the natural customers to make and adopt an open source NOS have created their own and think of them as extremely proprietary and strategic. Google, Facebook, and AWS have no intention of opening up their NOSes, and even though Microsoft, ironically enough, has helped established its SONiC variant of Linux as an open source NOS, no one can say that it is taking the datacenter by storm. The same holds true largely for Cumulus Networks, IP Infusion, Big Switch, and others who have tried to punch through the software moat around networking. Cisco’s IOS and NX-OS still prevail, with JunOS dwindling and EOS on the rise.

To be fair, Arista Networks has done a very good job in the past decade, but it is hard to say how much of its success is due to aggressive and diverse hardware and how much is due to EOS spanning all of that hardware. It is probably the mix of both that has made all of the difference. But EOS only runs on Arista’s iron, and that could be a limiting factor if ArcOS is picked up on a wider array of switching and routing gear.

Garg gives a tip of the hat to what he calls the Generation 1 of cross-platform network operating systems.

“The Generation 1 vendors did serve a purpose in that they helped in the systemization and productization of the system components,” Garg says. So ONIE drivers, zero touch provisioning, board support packages – all of these things have become much more stable and usable in today’s environment because of the work that’s happened over the last ten years. But the piece that was missing was a control plane operating system that was a viable alternative to you know Arista’s EOS and Cisco’s IOS and NX-OS. And that’s why I think the time is right because there’s so many other things that are lining up in our favor.”

The cross-platform nature of ArcOS, coupled with its ability to run in any function on the network, could turn out to be the key differentiator. A lot of these other NOSes were point solutions that could only be deployed in certain parts of the network, and that just creates animosity with the incumbent vendors that dominate the rest of the networking stack. Given the mission-critical nature of networking in the modern datacenter, it costs a great deal to qualify a new network operating system, and it can take a lot of time. If ArcOS can run across more platforms, qualify faster, and do more jobs in the network, then, says Garg, it has a good chance of shaking up switching and routing. “That totally changes the business conversation and the TCO advantages that we can bring to a customer across the entirety of their network.”