Innovation is core to the Internet and competition is good for consumers. While this is true, Facebook is innovating, but not competing, even if Cisco claims they are.
Since the beginning of networking, there have been hacks. No device or feature fits every customer. Whether these hacks were in the form of scripts, hardware modifications, software features or complete systems, they were born out of need. Facebook has a need.
I have been in the industry long enough to have seen companies like Blizzard, Akamai, Hotmail and many others conquer scale issues. The initial solutions were ugly (motherboards screwed directly to metal shelves..) but they were brought on by a need that was not being fulfilled by the computer industry.
We are in the same place today, but in the networking space. Companies like Google, Facebook and Amazon are hampered by the products that are being offered by the big players, Cisco, Juniper, etc. It’s not just price, it’s also features, functionality and support.
So with that background, it is easy to see why Facebook, Google and smaller companies like NGI “SDN for Real” would build (or design) their own routing/switching hardware. These companies already did it for their server infrastructure, and a switch, at it’s core, is just a specialized computer. It wasn’t long ago that many of the main Internet routers were repurposed mini computers with multiple interface cards, such as the ANS RISC System/6000-based T-3 routers.
Another driver helping Google, Facebook and others build switches is the availability of Open Source Networking Software. While routing stacks such as gated, zebra, etc have existed for a long time (again, back to the repurposed mini computers) the availability of “whitebox” or open switching hardware and features like DPDK from Intel are a more recent occurrence. This open hardware allows people to utilize Open Source Networking Software such as Quagga.
From the support side, one of the benefits of Open Source Software is the ability for companies to modify, patch and tailor the software to their needs. This is very different from the previous support model where companies would have to go to their vendor, ask for a fix/feature and wait. Today companies can change it themselves, hire someone to change it or wait for the feature/fix to come in a later release.
Most of “White Box” devices are some form of BCM reference platform (XGS), can be bought quite cheaply from number of ODMs. The real value is in the SW – HAL and routing stack. Pay attention – no FB/Google/Amazon use off the shelf routing stacks, nor open sourcing theirs…
It will stay this way for the foreseeable future – good routing engineers are extremely scarce same goes to BCM know how…
The Facebook systems are built by Quanta and utilize Broadcom chipsets. There are enough software vendors out there, along with the reference OSes (i.e. Fastpath, which can be extended by Aricent) that there is no need for any one to do a home grown routing stack. Google and Facebook are pretty open about what they do and how they do it, Amazon is the most closed because they have the most to lose if their competitors are able to take advantage of their know how.
On the development side, there are quite a few people (at least who I know) who are well versed in how the Broadcom/Marvell SDKs work. Even I have personally done some designs and a bit of coding, the heavy lifting is in designing the chipset.