6 ms·
All of these clouds are spawned out of already-large tech companies, many of whom have built a massive technical infrastructure to support their original busine
by scarface74 7y ago
All of these clouds are spawned out of already-large tech companies, many of whom have built a massive technical infrastructure to support their original businesses, then turn those resources into services to rent out in the form of cloud
This is a myth that’s been disputed several times by high ranking Amazon officials. Amazon never turned their internal infrastructure into AWS. It was always a completely separate initiative. Amazon was not on AWS for years.
Google also only uses GCP for a few internal services.
- serioussecurity 7y agoYes and no. They run on the same hardware and share a lot of the internal tools. Which is a very big lift.
- eeZah7Ux 7y agoFalse.
- rossjudson 7y agoWrong; it's true. I work on Google Cloud Storage. There are multiple layers of sharing at varying depths. The externally visible APIs to cloud storage are the tip of the iceberg, and the iceberg is very, very deep.
- eeZah7Ux 7y agoThe part about AWS is false.
- skybrian 7y agoThat's the software level though. Further down the stack, things like hardware designs, hardware purchasing, and maintenance can be shared.
- Retric 7y agoAssuming the basic assumptions hold. Google for example doesn’t really care much if individual searches are slightly less accurate so low data center temperatures should be less important for them than say a financial institution. https://www.cnet.com/news/google-computer-memory-flakier-than-expected/ https://www.cnet.com/news/google-computer-memory-flakier-tha... Facebook’s hardware designs might generalize well, but perhaps not.
- joshuamorton 7y agoI...I don't think the article you're quoting supports your conclusions. It concludes that running (slightly) too hot doesn't meaningfully impact error rates, and that even if you run at the recommended temperature, memory errors will happen anyway. So the conclusion is that, whether you're a financial institution or Google, you need ECC, at which point running too hot doesn't have any appreciable effect anyway.
- Retric 7y ago“Previous research, such as some data from a 300-computer cluster, showed that memory modules had correctable error rates of 200 to 5,000 failures per billion hours of operation. Google, though, found the rate much higher: 25,000 to 75,000 failures per billion hours.” First it’s old data, but google having observed higher error rates suggests different choices. Location, power supplies, motherboard design etc can all play a role. Further, they should be optimizing for slightly different things. Finally, they got data for temperatures from running their actual production system at these temperatures while assuming it would cause even more problems.
- joshuamorton 7y ago> but google having observed higher error rates suggests different choices. Alternatively, it suggests better data from a larger, real-er world study. >Finally, they got data for temperatures from running their actual production system at these temperatures while assuming it would cause even more problems. I'm not sure what you're saying here. To be able to detect the effect of temp, they ran at both higher and lower temps and compared.
- jvolkman 7y agoMany of GCP's services did actually grow out of Google's internal tools. For example, BigQuery is a public version of Google's internal Dremel. And Cloud Spanner is of course a public version of Spanner.
- kube-system 7y agoAnd Kubernetes came from Borg.
- ovi256 7y agoKubernetes is an open-source project inspired by Borg, not a proprietary product like the examples given by GP.
- kube-system 7y agoIt supports the original claim that is being disputed: > technical infrastructure to support their original businesses[0], then turn those resources into services[1] to rent out in the form of cloud[2] [0]: i.e. Borg [1]: i.e. Kubernetes [2]: i.e. GKE
- pradn 7y agoYes, Kubernetes's design was inspired by Borg, but the code is not Borg's code. BigQuery and Cloud Spanner use most of the code of the internal versions. That's the difference.
- snazz 7y agoAnd Cloud Source Repositories is similar to the internal code search as well as https://cs.android.com https://cs.android.com and https://source.chromium.org https://source.chromium.org. Edit: and https://cs.opensource.google https://cs.opensource.google. It’s this product: https://developers.google.com/code-search https://developers.google.com/code-search
- kortilla 7y agoBut this is still backwards. Internal google services are being cloned into watered down versions for the public. Google isn’t moving to GCP internally.
- alberth 7y agoSource: https://www.quora.com/How-and-why-did-Amazon-get-into-the-cloud-computing-business-Rumor-has-it-that-they-wanted-to-lease-out-their-excess-capacity-outside-of-the-holiday-season-November–January-Is-that-true?share=1 https://www.quora.com/How-and-why-did-Amazon-get-into-the-cl...
- deleted 7y ago[deleted]
- crazygringo 7y agoTechnical infrastructure = ability to build and run cost-efficient data centers at scale. I'm not sure anybody's ever seriously claimed the software infrastructure was the same. The "origin myth" of AWS I've always heard is that many of their servers were going unused outside the Christmas season, so AWS was a way to monetize them the rest of the year. At a hardware level. So all these tech companies have been able to leverage their technical infrastructure -- the hardware one.
- somedudetbh 7y agoThere are many conflicting stories about the origin of AWS, but the whole "our infra is underutilized except during Christmas, let's rent it out!" is one that I never hear repeated by anyone with any claim to knowing what was actually happening. I think it's an "Omidyar Pez Dispenser"-tier story: fully bullshit. It kind of doesn't even withstand a moment's scrutiny. What was Amazon going to do during Christmas? Like, if they lease all this spare capacity during non-peak, what happens to those customers during peak? There are spot instances now, but the first two AWS products, S3 and EC2, didn't come with some sort of, "Except during peak" contract. So Amazon needs exactly as much hardware to meet demand during Peak as it did before, and now it's got a bunch of customer workloads it also needs to serve during peak. I don't see how this would've solved any problems. Most stories about the origin of AWS have a few things in common: (1) There was limited support for the idea of AWS at senior executive levels (2) The early AWS products were not used by for any significant Retail workloads within Amazon for quite a long time (as in, several years) (3) As recently as a few years ago, there were major organizational pushes to move legacy Retail workloads onto AWS (4) As recently as last year, there were still so many teams on Oracle that everyone with an Oracle dependency was required to have an OKR to get off it. I include this because IIRC this is public-- after Oracle was making hay with the "Amazon sells Dynamo/RDS but runs on Oracle" campaign, Amazon publicly released information about their Oracle Annihilation roadmap. So it supports the theory that it's taken Amazon quite a long time to shift certain Retail workloads to AWS, even in cases where an AWS product that's suited to the workload has existed for quite a long time. I've never worked for Amazon, so I don't know, and it's impossible for all the stories I've heard from ex-Amazon people to all be true, but I think there's a pretty good argument against the "we should put this spare off-peak capacity to use" AWS origin myth.