6 ms·

Reading both this entry and the one that explained why they went with AWS, I'm left confused about why they ever went to AWS in the first place.

• jerf 16y ago
  Me too. This feels like "oooh, shiny!" from high levels, not engineering needs. You shouldn't outsource your core competency, and this feels like what they are doing. I say "feel" advisedly. I don't have inner knowledge. It's just that the explanation up to this point doesn't seem all that compelling; for what seems like rather dubious benefits it seems like they've taken on an awful lot of risks they have little ability to manage themselves.
  • irons 16y ago
    This came up in the first entry. Netflix would apparently dispute your take on their core competency. http://techblog.netflix.com/2010/12/four-reasons-we-choose-amazons-cloud-as.html http://techblog.netflix.com/2010/12/four-reasons-we-choose-a... The problems [Amazon] are trying to solve are incredibly difficult ones, but they aren’t specific to our business. Every successful internet company has to figure out great storage solutions, hardware failover, networking infrastructure, etc. Their third argument for virtualized infrastructure was "We're not very good at predicting customer growth or device engagement", which strikes me as a compelling problem to solve. Having to change their approach to managing service dependencies and failures doesn't seem like too high a price to pay.
    • jerf 16y ago
      Yes, I read the first entry. The use case that particularly came to mind when I said it seemed like they were outsourcing their core competency was actually the location of data centers and other such physical constraints, like peering arrangements. They're stuck with Amazon's decisions on that front. Granted they're probably large enough to get Amazon to do whatever it is they need to do, heck they can probably all but simply purchase a new data center and announce to Amazon that Amazon now has a new data center to manage :), but it still seems like an odd choice of control to release.
      • mechanical_fish 16y ago
        the location of data centers and other such physical constraints, like peering arrangements This is not Netflix's core competency either. This is the core competency of a CDN. If you have problems like this, you should almost certainly hire one, and architect accordingly. Netflix has long since done so: http://blog.streamingmedia.com/the_business_of_online_vi/2010/03/akamai-now-the-primary-cdn-for-netflix-but-at-a-very-low-price.html http://blog.streamingmedia.com/the_business_of_online_vi/201...
        • patrickgzill 16y ago
          Actually Netflix now uses Level3's (level3.com) CDN: http://www.level3.com/index.cfm?pageID=491&PR=958 http://www.level3.com/index.cfm?pageID=491&PR=958
          • elq 16y ago
            Notice how they say - "primary content delivery network", primary not sole. netflix actually uses many cdns, including akamai and l3.
          • lambda 16y ago
            No, they still use Akamai (dig cdn-0.nflximg.com to verify), but they are planning on moving to Level3 (assuming the Comcast/Level3 fight doesn't sour that deal too badly).
• jolan 16y ago
  I think they alluded to it in the first post. When they rolled out Xbox 360 streaming, they severely underestimated demand. It was nearly impossible to get a full quality stream for 2-3 months after the roll out. Now with AWS, they can spin up additional instances at will.
  • wmf 16y ago
    Note that Netflix does not use AWS for streaming.
    • jollojou 16y ago
      Can you please provide a reference?
• recampbell 16y ago
  Having built a non-trivial production system, I have a love-hate relationship with AWS. I love the possibilities and potential, but I'm deeply disappointed when it fails to live up to the hype. It's just a matter of alternatives. There's frankly no other IaaS out there that can match the features of EC2 (eg, EBS, elastic IP). These guys are unstoppable, they come out with more features every week.
  • wmf 16y ago
    Yeah, EC2 is the best cloud but it's a shame that it's still missing features that ESX 2.0 had (and Amazon has shown no interest in fixing it).
    • jeffbarr 16y ago
      Wes, what features are you looking for?
      • gcr 16y ago
        Do you work with AWS?
        • deleted 16y ago
          [deleted]
      • wmf 16y ago
        It's not me in particular, but if you read the EC2 forum going back to 2006 people have been asking for full virtualization, multiple IPs, multicast, non-NAT, reverse ARP, shared EBS, etc.
      • agmiklas 16y ago
        Not the original poster, but here's some stuff I really wish AWS had: a) Ability to map an elastic IP to an ELB. The CNAME thing it uses now has way too many drawbacks [1]. b) Ability to make an RDS instance a slave to or a master for a normal MySQL instance. This would make it possible to use RDS as a backup to our ordinary DB infrastructure using normal MySQL replication (and eventually, vice versa) c) Retention periods for EBS snapshots. You can get this yourself by writing a few simple scripts, but it would be really nice if snapshots could simply be labelled "delete after 90 days". d) A cross-availability zone, synchronously replicated EBS volume. This is probably fairly specific to our use case, but it would be neat if AWS natively provided something like DRBD. [1] http://blog.pagerduty.com/2010/08/31/load-balancers-need-static-ips/ http://blog.pagerduty.com/2010/08/31/load-balancers-need-sta...
• byteclub 16y ago
  Sounds like the initial investment into reengineering their software to handle distributed environments well turned out to be quite large, which is why it does sound a bit like they brought the troubles upon themselves. But presumably they now have 2 advantages: ability to scale horizontally, and having their software adapted to distributed environments (which means that technically they are not limited to AWS, in case problems arise with Amazon's service). Sounds like the foundation of their house is stronger now, which was probably the whole point of the exercise.
• tybris 16y ago
  In short: Because restaurants don't farm their own crops or raise their own livestock. They cook.
  • ergo98 16y ago
    Analogies merely confuse (they always have that superficial "aha!" moment, yet never bear scrutiny). If it must be, however, in this case it's more synonymous with a restaurant that farmed out their kitchen and their wait staff, but they carefully crafted the menu, chose the silverware, and did all of the marketing.
    • RobertKohr 16y ago
      That is an interesting business proposal.
      • patio11 16y ago
        It seems to be the franchise model: you buy the land and hire the grunts, we'll give you the Big Book O' Brand and handle the marketing.