17 ms·
2020 Cloud Report: AWS vs. GCP vs. Azure
- brenden2 7y agoI wish they had benchmarked the arm instances on AWS, I'd like to see a 3rd party validate AWS's claims about performance.
- tybit 7y agoScylla DB did a write up on it https://www.scylladb.com/2019/12/05/is-arm-ready-for-server-dominance/ https://www.scylladb.com/2019/12/05/is-arm-ready-for-server-...
- kryptiskt 7y agoI saw this just now on Twitter (performance/cost building FreeBSD): https://twitter.com/cperciva/status/1206688489518985216 https://twitter.com/cperciva/status/1206688489518985216
- QUFB 7y agoSorry for the non-technical comment here, but why do I have to provide my email address to read the full report PDF? I suppose the marketing department at Cockroach Labs proscribed this workflow. Is this a viable strategy? Might it be better to provide the report without the annoying engineers with the email requirement? How many readers of the report are interested in Cockroach Labs, rather than the benchmarks?
- jessaustin 7y agoI guess maybe they only care to deal with people who want this report badly enough to jump through some hoops?
- jasode 7y ago>, but why do I have to provide my email address to read the full report PDF? ... because the people who made the PDF content want to make a trade: your email address for their PDF whitepaper. The industry jargon for this technique is "inbound marketing".[1] >Is this a viable strategy? Yes, it often is. But I do get that it annoys a lot of people. That's why 99.9% of the time, I also don't bother giving my email address just to read pdf whitepapers. It's a voluntary trade -- and like you, I usually choose not to do it. [1] https://www.google.com/search?q=%22inbound+marketing%22 https://www.google.com/search?q=%22inbound+marketing%22
- adambyrtek 7y agoYou can always use a disposable email address like 10 Minute Mail: https://10minutemail.com/ https://10minutemail.com/
- universenz 7y agoNotice how the responses are honest ones, where people would rather skip the report than supply their real email address? The target demographic are tech savvy, so 90% of the leads are going to be 10MM email addresses. The popularity of this thread illustrate that there is a need/use for this content - so Cockroach nailed the bait, but have massively overshot the type of trap required to convert these savvy mice.
- chirau 7y agoThat 10minutemail workaround is easy to block, just require a business domain and a use a verification link.
- edoceo 7y agoRight! 10mm and many others are on our blacklist when we collect email (2-opt-in newsletter and sign-up)
- Supermancho 7y agoyou can get email/verifications from many temp email services.
- adambyrtek 7y agoLike many similar problems, it's a cat and mouse game. For example, it's equally easy to detect adblockers, but they are still really effective.
- chirau 7y agoCockroach is a DB company. Customer acquisition cost is in the thousands if not tens of thousands or hundreds of thousands. The steps I mentioned above are what, maybe a few hundred dollars. Trust me, their lead generation pipeline most likely already subscribes to this and utilizes it with minimal concern. There are plenty of CRM tools that can flag out BS domains and/or low value not worth it domains. If you are a person who gets by using 10minuteemail, you are not their target market. So will their target demographic. The extra hop is very necessary for prevalidation. And they will be just fine with it. You on the other hand, being the cheapskate that you are, have to commit an extra 5 minutes every time you generate these BS emails. Obviously you don't value your time as much. And that is why, again, you are not the target market because you extract no business value from accessing the knowledge they spent time and money aggregating for you. Your getting the PDF is inconsequential to both them and yourself. So go ahead, no one really cares. EDIT: Lol, you was not referring to you specifically, just the folks who think they are being smart or outsmarting corps by using these services like 10minutemail or whatever it's called
- jsploit 7y agoDirect PDF link: https://cdn2.hubspot.net/hubfs/1753393/guides/Cloud_Report_2020-1.pdf https://cdn2.hubspot.net/hubfs/1753393/guides/Cloud_Report_2...
- dtrailin 7y agoThese reports behind an email wall frustrate me to no end. I don't want to receive your spam to read a single article!
- kyrra 7y agoIts rather expensive (and time consuming) to produce reports like this. They are doing it to generate leads.
- etxm 7y agoNarrator: No they aren’t. They’re going to generate unsubs and spam reports.
- johannes1234321 7y agoThose can be filtered out quickly. Remaining are potentially valuable leads.
- freehunter 7y agoIf there is a cost associated with producing content and that cost can’t be recouped by the presentation of that content (speaker fees at conferences, advertisements, sponsorships, etc) then the content should be sold for a fair market rate. That’s basically the entire point of GDPR. Stop making people trade personal information for “free” content.
- manigandham 7y agoNo that's not the point of GDPR, it's about choice and control over data. You're free to not share your email if you don't want to read the report. The rest of us can make our own decision.
- mobileexpert 7y agoLooking at the Azure networking numbers I wonder if they got their configs correctly optimized. Azure networking was always tricky to get best performance when I used it heavily at my last co.
- jread 7y agoI suspect they are bottlenecked on CPU - most of these instance types are capable of higher throughput than they've captured, but you'd need more than 1 Iperf process to measure it on Azure.
- Havoc 7y agoYeah was about to say - I'm sure I've seen numbers 5x that even on small azure instances.
- daxfohl 7y agoNo, I contacted them directly after seeing this report and confirmed they did not even use accelerated networking (which is a free, but opt-in service on Azure), much less things like placement proximity groups (which only recently went public). So machines were communicating across availability zones (separated physically by miles and logically by additional networking layers) and a bunch of networking was happening in CPU rather than offloaded to FPGA. Disclosure: I work for Azure networking.
- jdsully 7y agoWhy is accelerated networking opt in? It seems like making it default would improve these comparisons a lot and would help a lot of your customers who likely haven't found this setting either.
- daxfohl 7y agoI don't know offhand, it's a different team that runs it. I'd guess it's something related to backwards compatibility and the general "roll big changes out slowly so it doesn't break" credo. And it may be tied to a certain kind of HW that supports it, which doesn't exist everywhere, since cloud providers can't just throw out all their HW each time a new feature comes in (as much as we wish we could). But that's just a guess.
- unreal37 7y agoThey calculated the costs based on 3 years running at the hourly rate. That's kinda weird. How about including multi-year discounts? These are available to everyone. [1] https://azure.microsoft.com/en-ca/pricing/reserved-vm-instances/ https://azure.microsoft.com/en-ca/pricing/reserved-vm-instan... [2] https://aws.amazon.com/ec2/pricing/reserved-instances/ https://aws.amazon.com/ec2/pricing/reserved-instances/ [3] https://cloud.google.com/compute/docs/instances/signing-up-committed-use-discounts https://cloud.google.com/compute/docs/instances/signing-up-c...
- CoolGuySteve 7y agoYeah, and without comparing spot/interruptible prices, these results are meaningless for me.
- Scaevolus 7y agoYou wouldn't typically run your database VMs on spot/interruptible instances.
- alexnewman 7y agowhy not? Just kidding, but it's getting more common :)
- alexnewman 7y agoIn distributed databases with appropriate grace under churn... Sorry I wasn't clear in my earlier post
- scarface74 7y agoYou wouldn’t typically run your databases in a cloud environment on your own VMs - you would use a managed service.
- timc3 7y agoThat depends on several factors. The database you might want to run may not be available for instance.
- AustinDizzy 7y agoThis is a good report, but I'd also be interested to see focus on GPU performance on both classic GPU intensive workloads and machine learning workloads.
- ishjoh 7y agoWell done Cockroach labs team. This data is very valuable and much appreciated. A small nitpick if you're reading. Fig 19: Sotrage: Azure Write Throughput the y axis has an incorrect label of "1,00" when it should be "1,000"
- orangechairs 7y agoCockroach here: Thank you! We'll get this fixed.
- taspeotis 7y agoA most minor point (and not to detract from the great effort you to went to produce the report): it uses yellow for AWS (good, their logo is yellow/orange) but red for Azure (bad, Azure is ... azure.)
- otacketoc 7y agoThey should also have included oracle which will soon surpass google (making it 5th most popular) cloud
- lucasmullens 7y agoWhat rankings are you referring to?
- otacketoc 7y agoRevenue
- deleted 7y ago[deleted]
- outworlder 7y ago> They should also have included oracle which will soon surpass google (making it 5th most popular) cloud I seriously doubt it. Only companies already heavily invested in Oracle are taking their offering seriously. They are pretty late to the party.
- BossingAround 7y agoSurely, if we were including other clouds, we'd include IBM before Oracle... :))
- outworlder 7y agoI would add Alibaba Cloud before them too :)
- tcbasche 7y agoSure, and I'll just patch together a few of my home PCs and we can call that a cloud too
- green-eclipse 7y agoOracle's cloud is an also-ran. Just follow the investment the cloud giants are making. Oracle is not even in the same league. From: https://www.platformonomics.com/2019/02/follow-the-capex-cloud-table-stakes-2018-edition/ https://www.platformonomics.com/2019/02/follow-the-capex-clo... "Google’s incremental CAPEX spend in 2018 is about what cloud pretender Oracle has spent on CAPEX throughout their entire history."
- account73466 7y agoI remember that GCP claimed they will follow Moore's Law and drop the price for the same instance type over time. It was just BS marketing, unfortunately I believed them.
- guuguuguu 7y agoSad story
- mathattack 7y agoI’ve observed price cuts over time for Azure and AWS but very hard to get good published data.
- user5994461 7y agoAWS also stopped to drop instance price periodically. Sadly Moore law has stopped some time ago affecting everyone.
- sudosysgen 7y agoAs we're starting to see, it was mostly Intel not keeping up, actually. Moore's law is mostly holding up till today, and probably for a couple more years before we really have to switch paradigms.
- user5994461 7y agoThe definition of doubling performance every few years stopped holding up a while ago. The formal definition of doubling the amount of transistors somewhat holds up, see the huge increase in core counts in recent years, but that doesn't benefit most applications much. Incidentally cloud providers are billing by core count so adding more cores doesn't help the bill.
- imtringued 7y agoMost applications that run on a server create a process or thread per request/session and scale reasonably well.
- CzarnyZiutek 7y agogoogle sponsored article.
- ignoramous 7y ago> ...in 2018, AWS outperformed GCP by 40%, which we attributed to AWS’s Nitro System present in c5 and m5 series... In 2020, we see a return to similar overall performance in each cloud. How did GCP and Azure catch-up to AWS Nitro in one year which IIRC is a coming together of minimalistic micro-vms, hardware-accelerated network and IO cards, hardware offload for encryption and other maintenance tasks... a work that was 5 years in the making [0]? [0] https://www.youtube-nocookie.com/embed/rUY-00yFlE4?t=1m45s https://www.youtube-nocookie.com/embed/rUY-00yFlE4?t=1m45s
- zeroxfe 7y agoIt's more likely that all of them started working on it at about the same time and AWS came out first.
- cdoxsey 7y agoMeltdown and Spectre did a real number on the c5 and m5 instance types. Amazon's original fix reduced performance substantially, which lot's of people noticed, but no one knew why till the vulnerabilities were announced. Meanwhile Google built a clever workaround which minimized the pain. It was really a mic-drop moment from Google and I wish someone would dig in deep and tell the whole story. At this point I'm sure AWS has merged Google's patches, but it showed how much Google was investing in their GCP offerings.
- DDerTyp 7y agoInteresting! Do you know of any resource where I can read more about this?
- campers 7y agoThere was a few interesting bits in a post the other days about their new compute VM class, #4 in particular https://cloud.google.com/blog/products/compute/understanding-dynamic-resource-management-in-e2-vms https://cloud.google.com/blog/products/compute/understanding... Its pretty light on details but was news to me.
- m0zg 7y agoI wonder why they stuck to the "standard" machine types on GCP. Unlike with AWS you get to vary the config there somewhat, so if your workload benefits from more CPU you can add jus the CPU. Same for RAM. That will affect the per-dollar figures because you can tailor your instance to your workload pretty exactly.
- carlivar 7y agoThey should add Alicloud.
- dzhiurgis 7y ago> cloud > compares virtual machines I mean cloud is different for everyone, but for me a VM is far from cloud. It's the least differentiable item in each of their suite.
- killjoywashere 7y agoGCP seems to have gotten the "most improved" award, but is Azure winning anything?
- scarface74 7y agoSupport and confidence of Big Enterprise....
- rrwright 7y agoPentagon DoD contracts — Jedi.
- tozeur 7y agoPerformance is only half the battle. Companies need high SLAs, compliance, easy integration, familiarity, and a whole lot more.
- phonon 7y agoDid this use the latest 36% higher performance EBS systems on AWS? https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-ec2-nitro-system-based-instances-now-support-36-faster-amazon-ebs-optimized-instance-performance/ https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-ec...
- samuelfekete 7y agoAfter clicking on this link, I now get ads about this report.