2 ms·

I wouldn't do my own datacenter as in build out a building, but I would certainly lease a cage and build out my own infrastructure at probably half that count.

I wouldn't do my own datacenter as in build out a building, but I would certainly lease a cage and build out my own infrastructure at probably half that count. As far as I know, Netflix has POP's for content delivery in many carrier facilities so it's not a pure cloud player by any means. For the cost of 1 year for a single cloud server you can buy a 52 core dell with 256gb ram and a few SSD's and have price/performance ratios that heavily scale in your favor and with a scheduler like marathon, k8s, nomad you can easily achieve elasticity in a private colo to a large degree without also marrying yourself to cloud specifics and nothing would stop you from peering with major cloud providers for seasonal elastic needs.

blackaspen 9y ago
That's a good distinction. CoLo providers are pretty reasonably priced and most of the larger ones are good at what they do. There's a HUGE scale difference between a CoLo and your own -- as soon as you start having to negotiate diesel supply contracts yourself things get crazy fast.
- supernovae 9y ago
  I would never roll my own unless I was of facebook/google size. Colo space is abundant and affordable.
willshepherdson 9y ago
I think this is a crucial distinction. Netflix has a relatively unique architecture for their content delivery system compared to other online video providers, which is worth considering in context of this article. With Netflix's Open Connect (https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/) system, most residential Internet Service Providers have appliances at the regional point of interconnection. These appliances are inside of or directly connected to the ISP's network in the region. Back in 2016 they had appliances deployed to over 1,000 locations. Content is pre-positioned during off-peak hours to the appliances. Previously Netflix has been open about how they use AWS for "generic, scalable computing" including "all of the logic of the application interface, the content discovery and selection experience, recommendation algorithms, transcoding, etc." At the time, they claimed that one of the advantages of doing this was the ease of use and growing commoditization of the “cloud” market, so it makes sense that they are evaluating alternative cloud providers for these commodity services.
scarface74 9y ago
I've used most of Hashicorp's tooling - Consul, Nomad, Vault, etc. - and I consider myself to be a Hashicorp fanboy. But if you're spinning up a bunch of VMs "in the cloud" and that's all you're doing, you're not taking advantage of the platform. The idea behind using AWS is not just to avoid your own capital infrastructure, it's also to allow someone else to do the "undifferentiated heavy lifting". In our case: Consul -> ParameterStore Vault -> KMS/Cognito/IAM Nomad -> Elastic Container Services/Fargate (serverless Docker)/Lambda SQL Server -> RDS -> Aurora Memcached -> ElastiCache (Memcached protocol) Consul+Fabio (service discovery/load balancing) -> Elastic Load Balancers/Route 53/Autoscaling VSTS (MS's hosted CI/CD platform) -> AWS Code Build/Code Deploy/Code Pipeline "not marrying yourself to a one cloud provider" is equivalent to "coding using a repository pattern to access the database" in both cases people claim to not wanting to tie themselves to one vendor and then they realize that they aren't taking advantage of what the vendor has to offer. In my experience, no one wants to go through the pain of moving to another vendor just to save a little money.
btian 9y ago
Netflix doesn't pay retail for AWS or G Cloud

blackaspen 9y ago

That's a good distinction. CoLo providers are pretty reasonably priced and most of the larger ones are good at what they do. There's a HUGE scale difference between a CoLo and your own -- as soon as you start having to negotiate diesel supply contracts yourself things get crazy fast.

supernovae 9y ago
I would never roll my own unless I was of facebook/google size. Colo space is abundant and affordable.

willshepherdson 9y ago

I think this is a crucial distinction. Netflix has a relatively unique architecture for their content delivery system compared to other online video providers, which is worth considering in context of this article. With Netflix's Open Connect (https://openconnect.netflix.com/en/ https://openconnect.netflix.com/en/) system, most residential Internet Service Providers have appliances at the regional point of interconnection. These appliances are inside of or directly connected to the ISP's network in the region. Back in 2016 they had appliances deployed to over 1,000 locations. Content is pre-positioned during off-peak hours to the appliances. Previously Netflix has been open about how they use AWS for "generic, scalable computing" including "all of the logic of the application interface, the content discovery and selection experience, recommendation algorithms, transcoding, etc." At the time, they claimed that one of the advantages of doing this was the ease of use and growing commoditization of the “cloud” market, so it makes sense that they are evaluating alternative cloud providers for these commodity services.

scarface74 9y ago

I've used most of Hashicorp's tooling - Consul, Nomad, Vault, etc. - and I consider myself to be a Hashicorp fanboy. But if you're spinning up a bunch of VMs "in the cloud" and that's all you're doing, you're not taking advantage of the platform. The idea behind using AWS is not just to avoid your own capital infrastructure, it's also to allow someone else to do the "undifferentiated heavy lifting". In our case: Consul -> ParameterStore Vault -> KMS/Cognito/IAM Nomad -> Elastic Container Services/Fargate (serverless Docker)/Lambda SQL Server -> RDS -> Aurora Memcached -> ElastiCache (Memcached protocol) Consul+Fabio (service discovery/load balancing) -> Elastic Load Balancers/Route 53/Autoscaling VSTS (MS's hosted CI/CD platform) -> AWS Code Build/Code Deploy/Code Pipeline "not marrying yourself to a one cloud provider" is equivalent to "coding using a repository pattern to access the database" in both cases people claim to not wanting to tie themselves to one vendor and then they realize that they aren't taking advantage of what the vendor has to offer. In my experience, no one wants to go through the pain of moving to another vendor just to save a little money.

btian 9y ago

Netflix doesn't pay retail for AWS or G Cloud