Netflix’s New Spinnaker Open Source Tool Makes It Easy to Use Amazon’s Cloud—And Google’s
NETFLIX STREAMS TV shows and movies to more than 60 million people worldwide. It’s one of the most popular Internet video operations on earth, delivering about 10 billion hours of the stuff each month. And for the most part, it delivers all that video from hundreds of computers that belong to someone else. It runs the Netflix video empire atop Amazon’s cloud computing service—a service that lets anyone rent nearly unlimited amounts of computing power over the Internet.
The world of cloud computing is a complicated one, both technically and politically, and today, Netflix showed just how complex—and how intriguing—this new world order can be.
‘Over time, this gives them a really great migration strategy, the ability to mix and max clouds.’
ROB MEE, PIVOTAL
Over the past year, the company has built a new tool for quickly and continuously deploying its latest software code to machines running in the Amazon cloud, and this morning, it open sourced that tool, known as Spinnaker, sharing it with the world at large, so that anyone else can use it. Netflix has done something similar in the past. But Spinnaker is a little different. Netflix built the tool in tandem with Google, one of Amazon’s biggest competitors in the cloud computing market. And Spinnaker is specifically designed to deploy software to not only the Amazon cloud, but, yes, to Google’s cloud as well. Google spent a year working with Netflix to ensure this was the case.
Netflix’s Andrew Glover, who oversaw the development of Spinnaker, says that the company has no intention of moving its online empire off of the Amazon cloud and on to Google’s—even in part. Inside Netflix, engineers only use Spinnaker in delivering code to Amazon. But it’s telling that Netflix has worked closely with Google in creating Spinnaker—and that it’s publicly joining hands with Google in open sourcing it. It highlights the seemingly strange but enormously effective way that open source software helps drive the world of cloud computing. And though Netflix says it’s completely committed to running its empire on Amazon, the partnership also shows that cloud computing provides a certain freedom to move operations from place to place, and from vendor to vendor. Today, Amazon dominates the cloud computing market, pulling in an enormous $6 billion a year from cloud computing, but there’s always room for competition. No online business is stuck on one cloud, including Netflix.
Google joined forces with Netflix on Spinnaker because it wants businesses to use the tool with the Google cloud. And Netflix joined forces with Google because it wants to ensure that it can take advantage of any improvements Google makes to the tool. According to Glover, Netflix has also worked with engineers at cloud computing company Pivotal to make sure that Spinnaker can also deploy code to Pivotal’s Cloud Foundry software. And Netflix plans to work with Microsoft engineers so that the tool will dovetail with the Microsoft Azure cloud service. All of this, Glover says, will help ensure that Netflix can benefit from the work of the wider community.
MORE CLOUD COMPUTING
Dell. EMC. HP. Cisco. These Tech Giants Are the Walking Dead
Dell. EMC. HP. Cisco. These Tech Giants Are the Walking Dead
Companies Should Be as Mad About Amazon’s Ban as Shoppers
Companies Should Be as Mad About Amazon’s Ban as Shoppers
Amazon Reveals Just How Huge the Cloud Is for Its Business
Amazon Reveals Just How Huge the Cloud Is for Its Business
It’s worth remembering, however, that Amazon also offers a service called Amazon Prime Video, now a significant competitor to Netflix. Yes, that’s right: Netflix runs its empire on machine that belong to one of its biggest competitors. The modern Internet is a place where this kind of thing happens quite often, particularly in the world of cloud computing. Sure, an arrangement like this comes with risks. But it also comes with ways of mitigating those risks. Glover is still adamant that Netflix isn’t planning to use Spinnaker to spread its empire across services other than Amazon. But at the very least, Spinnaker shows that doing so is a possibility.
Would Amazon somehow mistreat Netflix for competitive reasons? There are so many reasons it wouldn’t. This would hurt Amazon’s reputation with other customers, for one. And in the long run, Netflix, one of its biggest customers, would leave its cloud entirely. But consider how Amazon has treated competing products that show up on its online store. The world’s largest online retailer recently nixed Apple TV and Google Chromecast hardware from being sold on its site. Amazon, you see, sells its own Fire TV video hardware—hardware that helps deliver the Amazon video service that directly competes with Netflix.
“This certainly makes sure that Amazon treats them well,” Rob Mee, the CEO of Pivotal, says of Spinnaker and Netflix. “Over time, this gives them a really great migration strategy, the ability to mix and max clouds, and arbitrage one against the other.”
For his part, Mee said days before the announcement that he wasn’t aware that his company was working with Netflix on Spinnaker. And Microsoft declined to comment on the project. It’s unclear just how deeply Pivotal and Microsoft are involved, but Netflix name-checks them in its blog post. This could be a way of showing Amazon that Netflix has many other cloud options. Or it could be Netflix’s way of playing down the importance of the partnership with Google specifically.
After all, in addition to giving itself options beyond Amazon, Netflix must keep Amazon on its side. Amazon is by far the world’s largest cloud computing company because it was first to market, but also because the scope of its services are so wide. Plus, um, a vast majority of Netflix is running on Amazon. Netflix can’t afford to bite the hand that holds it up—or at least not bite it too hard.
Whatever the politics swarming around today’s announcement, it lays bare the dynamics of today’s cloud-dominated world of enterprise tech. Thanks to cloud computing, companies and coders can build and run software without setting up their own computer servers. And because so many vendors offer cloud computing services—from Amazon to Google and Microsoft to smaller operations like Digital Ocean—companies and coders needn’t be overly dependent on any one cloud. They can mix and match, depending on the quality of the technology, the prices, and anything else that matters.
There are cases where switching from cloud to cloud is difficult. But Spinnaker can help change this. And like so many tools in the world of cloud computing, it’s open source. That means anyone can use it. But it also means anyone can work to improve it. Anyone, for instance, can make sure its works with other cloud services, including Microsoft Azure or Digital Ocean, or with private data center software from the likes of Pivotal. That’s good for Netflix—and for everyone else.
Spinnaker is what’s called a continuous delivery system. It’s one of many tools that let companies instantly deploy their latest code on a vast network of machines, allowing that code to be tested or put into action. These delivery systems, including tools from companies such as Chef and Thoughtworks, help delivery code to physical computers as well as to cloud services from the likes of Amazon and Google.
What’s interesting about Spinnaker is that it’s used by Netflix, one of the most significant online operations on earth. That means it’s built for serious—and rapid—work. According to Netflix, it lets engineers create complex “pipelines” for delivering various types of completed software onto machines in the cloud. For instance, you can deploy a new test version of some software to one set of servers while the old version continues to run on another set—what’s known as a Blue-Green or Red-Black strategy. If one new version doesn’t work as planned, the company says, you can revert to the old version.
Plus, Spinnaker seeks to simplify and streamline the way code is deployed and managed across multiple cloud services. “We worked with [Netflix] to build an interface that allows the Spinnaker tools to plug in with any other additional cloud providers,” says Google’s Rick Buskens, who was part of the project.
Alex Ethier, who oversees the continuous delivery tools offered by Chef, says he’ll reserve judgment on Spinnaker until he can closely review the system. But as described by Google and Netflix, he says, it seems to tackle an important problem. “The space is ripe for tools that make it easier to practice [continuous delivery] at velocity,” he says. ”
According to Google’s Buskens, Spinnaker lets you readily deliver different parts of your code to different services. You can deploy your new code to Google, while your old code runs on Amazon. Or you can mix and match. “Spinnaker is designed to work with multiple cloud platforms simultaneously,” says Buskens, who has worked on some of the fundamental infrastructure than underpins Google’s own operation, including a tool called Borg.
