9 ms·
ARM vs. Intel on Amazon’s Cloud: A URL Parsing Benchmark
- andrewstuart 4y agoRunning on your own computer is much faster than cloud.
- gifnamething 4y agoSo how do you plan to get your own Graviton 3 computer?
- itsthecourier 4y agoSo around 15% cheaper and 15% faster, good stuff
- jamesfinlayson 4y agoYeah I can't remember what AWS's marketing says exactly but they definitely say cheaper and faster (for Lambda as least).
- Rafuino 4y agoWhen you control the pricing and want to drive people to your own hardware, of course it's gonna be cheaper. Gotta exercise that market power over your suppliers
- Scalene2 4y agoI've been impressed not only with cloud arm performance but also usability. I had a hobby project that was using minecraft, javascript, bash, and such. IIRC absolutely nothing needed to be changed going from x86 to Arm. I got better performance (for my odd application) and better price.
- dboreham 4y agoIn my experience it often does go smoothly, but otoh sometimes not. The typical reason for that is when a container image dependency is not published for ARM. And then the docker tooling does not make it particularly easy for you to substitute your own ARM binary container image for a public hub-published x64-only one. And then sometimes the image won't build at all on ARM for <reasons> of the kind that waste days of your time and turn out to require building an entire Linux distribution to fix...
- mike503 4y agoYeah like New Relic not packaging its PHP agent (which they make sound trivial and I'm pretty sure it is) https://github.com/newrelic/newrelic-php-agent/issues/323 https://github.com/newrelic/newrelic-php-agent/issues/323 I didn't bother taking the time to compile from scratch and figure out a way to inject it into my container image. It was a smaller client of mine. But it seems real simple to setup a second pipeline in their CI system and move on.
- joncfoo 4y agoAt $previous_job we shifted a large workload from Intel to Graviton which was projected to save ~$1.7m annually while keeping roughly equivalent performance (after some tuning).
- wahern 4y ago> which was projected to save ~$1.7m annually Did it?
- notyourwork 4y agoI've seen first hand validation on massive workloads moving to Graviton based instances. This includes low latency high TPS java services and offline big data compute on EMR. All combined the hype is quite real. Heck, even moving an intel based service to newer nitro based EC2 instances resulted in a drastic performance improvement. Moved from m5.24xlarge --> m6g.8xlarge with better service performance and improved latency characteristics. Intel is in trouble in my opinion.
- vladvasiliu 4y ago> Moved from m5.24xlarge --> m6g.8xlarge with better service performance and improved latency characteristics. Intel is in trouble in my opinion. I wonder if this is actually an Intel issue or if there are some other optimizations at play, such as in the virtualization layer. Because at one point I wanted to try Jetbrains' new "gateway" product, which basically runs a remote IDE and only shows the GUI locally. I was curious on one hand, but I also wanted a machine with a bit more oomph for my occasional compilation needs (rust on Linux, fwiw). I was really unimpressed, the c6i was comparable to my local slim laptop running an 11th gen i7u part. My similar slim AMD 5650U laptop is actually faster. IIRC, the c6i.metal wasn't particularly faster on this kind of single threaded work.
- di4na 4y agoThe difference is in the pricing and the fact the core are "whole" On intel aws, you pay per HyperThread. On Graviton, you pay per core. But on this kind of workload and with modern schedulers, HT bump is rather limited. So in practice you are paying twice the price for the same number of cores. This is the biggest contributing factor to that difference and i keep being surprised noone mention it.
- nerdbaggy 4y agoThe Graviton are also real cores in the VM, not like the Intel/AMD that have hyper threading.
- jeffbee 4y agoThere is not some force of nature that makes 1 thread per core "real". On pointer-chasing workloads a hyperthread is as real as anything. HT is basically another way to exploit instruction-level parallelism that your compiler left on the table. You gotta pick what suits your program.
- Dalewyn 4y agoOr to put it another way: A hyperthread logical processor is a way of obtaining better efficiency by using surplus resources left on the table by the "real" logical processor that otherwise would remain idle (read: waste). Considering hyperthreading as a way of obtaining an extra "real core" is a gross misunderstanding of what hyperthreading is meant to achieve.
- paulmd 4y ago> Considering hyperthreading as a way of obtaining an extra "real core" is a gross misunderstanding of what hyperthreading is meant to achieve. It's as real as a Bulldozer CMT thread, and that was widely considered a "real core". An integer ALU being pinned to a particular thread doesn't make it "real" especially when it comes at the expense of shared frontend resources like a decoder that has to alternate between servicing each thread on alternate cycles (which, as Agner Fog's Microarchitecture notes, massively bottlenecks both threads). And obviously if one thread has ILP that can be exploited, and the other "core" isn't using its ALU, it would be better to share that! And when you allow that, what you get is SMT. At the end of the day that's all CMT is - SMT with inefficiently allocated (pinned) resources, and it had even less dedicated resources on the frontend as well. And people will absolutely die on the hill that bulldozer was a "real core". There are probably some scheduling advantages to doing CMT instead of SMT, but also performance costs as well. So what is a "real core" anyway in this context? Is physical (unsharable, unchangeable) division of resources "inherently better" (or even inherently different) from logical/software-defined division of resources? Then you've got this whole thing from IBM recently... and leaving aside the fact it's cache, the question IBM is fundamentally asking us is, why not just allocate more resources to "cores" that need them? Why not execution units as well, why wouldn't that be better? Why is hardware defined core better than software defined core? https://www.anandtech.com/show/16924/did-ibm-just-preview-the-future-of-caches https://www.anandtech.com/show/16924/did-ibm-just-preview-th... And when you look to how you would implement that for non-cache resources, isn't the simplistic answer something very similar to SMT? Not sure POWER9 is all that far off base with runtime-configurerd SMT4/SMT8 and a big fat core, maybe that's how Intel can make better use of some of the gigacores they've built. Sure it can run one thread really fast but why not 8 threads on the same resources? Or you can go the other way and have one thread issue onto multiple cores, as long as there's ILP to cover it and the performance impact of crossing cores is not large, does the difference really matter? And yeah maybe that's still one core... but then so is a bulldozer CMT module too lol. The whole "what is a core exactly" is kind of trite, it doesn't really matter.
- jeffbee 4y agoGood luck actually getting a c7g.
- benmmurphy 4y agoWhen I was trying to launch c7gs I was having trouble but then I just added a capacity reservation and was able to launch fine. This was in us-west-2 about 6 months ago. Not sure if there are still problems in that region or if this hack still works.
- ary 4y agoWhen I see the word "benchmark" and don't see a methodology I get a little wary. In this case the author ran a custom benchmark from one of their projects. https://github.com/ada-url/ada/blob/main/benchmarks/wpt_bench.cpp https://github.com/ada-url/ada/blob/main/benchmarks/wpt_benc... To be clear I'm not questioning the benchmark's accuracy or author's bona fides, but that post was a little short for my taste.
- a012 4y agoTo me it's clear in the post: they benchmarked how fast is their piece of code on c6i and c7g instances. It's not to prove Graviton 3 is faster than Intel in general, just one in many cases.
- nerdbaggy 4y agoIntel had published results and “surprisingly” they are the opposite https://www.intel.com/content/www/us/en/products/performance/cloud-facts.html https://www.intel.com/content/www/us/en/products/performance...
- pclmulqdq 4y agoOne person is compiling with -march=native (Intel) and using hand-tuned AVX-512 where appropriate. The other is running an application, and wants good performance for the money without too much vendor lock-in.
- anonymousDan 4y agoIn other words, if you care about performance go with Intel? As for vendor lock-in, how is relying on AWS specific hw much better (see other comments below)?
- jsty 4y agoIf you care about performance to the extent of inlining hand-tuned assembly, you almost certainly care enough to do your own perf benchmarking on your own workloads.
- pclmulqdq 4y agoPresumably, you aren't doing anything special in those ARM builds, so you can use an x86 build just as easily (or a non-Amazon ARM chip) depending on cost.
- intelVISA 4y agoif(AMD){ slowPath(); } "Wow Intel's so much faster..!"
- jeffbee 4y agoBy the way for anyone curious how the numbers in the article compare to a modern desktop computer, a 13th-generation Intel Core does 169ns/url, twice as fast as either of these server rigs. If I cap mine to 3GHz to reflect more realistic server scenarios, 310ns/url, a bit quicker than the Graviton 3. Even the "efficiency cores" on a desktop are faster than these server parts, at 286ns/url.
- 7e 4y agoCloud servers have many, many more cores, cache and memory channels than desktop chips. You can’t clock them that high, nor would hyperscalers want to. They would cost more in power and cooling. Cloud is about multi factor optimization.
- fulafel 4y agoHow about running as many threads as your cpu has SMT threads? Those are what AWS sells as vCPUs.
- ksec 4y agoJust to note the Graviton 3 is on 5nm. The 13th Gen is on Intel 7. The Core is also Real Core compared to a VPC which is a thread.
- garblegarble 4y agoAnother datapoint on a modern desktop CPU, this time ARM (M1 Max laptop, so the same TSMC 5nm process as Graviton 3): 175ns/url
- klelatti 4y agoWell the Graviton 3 runs at 2.6 GHz so not sure that really shows 13th-gen Core in a great light.
- Cyykratahk 4y agoI tried this benchmark on both my desktop (Ryzen 5800X3D) and phone (Sony Xperia 1 IV) Windows => time/url=364.626ns WSL2 => time/url=234.915ns Android => time/url=252.276ns This benchmark feels flawed to me, but I'm not qualified to speculate why.
- drej 4y agoI remember just adding one letter to the instance type in EMR to switch from Intel to ARM... and saw a 20% speedup as well as additional 20% hourly cost saving, all in all 1/3 cost decrease from a single character. Some migrations are easier than others.
- time0ut 4y agoWe saw similar gains going from Intel to Graviton 2 on EMR: 15% to 40% speedups depending on the job. Saw similar gains in EMR serverless switching from x86 to arm as well. The only issue I've had is actually getting enough on-demand instances in certain regions during peak times. I can't wait for Graviton 3 to be available on EMR.
- drcongo 4y agoAs someone who knows very little about CPUs but is aware of the energy savings that can be had on ARM, along with reports like this about speed and my personal experience of my M1 Pro MBP which feels like I won't need a new computer for a decade, can anyone explain to me why we're not seeing much wider adoption of ARM on servers? I'm primarily thinking of the lack of ARM runners on GitHub and the almost unicorn-level rarity of ARM options on VPS hosts.
- prmoustache 4y agoMost VPS hosting services don't build their own hardware the way some of the giants can and rely on available offering from server manufacturers. Some even use consumer grade regular tower pc. The offering is growing but is still not as large as intel/amd one.
- hebrox 4y agoAlso lack of ARM runners on GitLab CI
- dboreham 4y agoI think this tells us that price/performance is not a critical factor to these vendors, most of the time. Perhaps because they have high margins, or because the CPU is not a significant part of their input costs?
- DeathArrow 4y agoI would like to see more extended benchmarks.
- Chessn00b 4y agoIsn't the claim that ARM has nothing comparable to AVX512 technically incorrect? AFAIK SVE and SVE2 are definitly "comparable" even tough the actual vector size depends on the actual hardware of the processor.
- Symmetry 4y agoIn current commercially available cores SVE and SVE2 are comparable to AVX but not AVX512 specifically, because every SVE design out there that isn't in a Fujitsu supercomputer is limited to 256 bits. There might also be differences on scatter/gather that are important in some places.
- daneel_w 4y agoJust chiming in: also switched a lot of stuff at work from c5/c6/r5/r6 Intel to c6g/r6g Graviton. We are since seeing notably lower average load - read: more available capacity despite same number of instances - while also having a lower monthly cost.
- password4321 4y agoIs this affected at all by "other people's code" running on the same hardware simultaneously? I am not an expert on the various instance types and/or bursting, but I assume that ARM hardware is not as underprovisioned just by lack of fellow users.
- benmmurphy 4y agoOur CI jobs run on AWS building for arm64 and amd64 and arm64 runs slower. This might be because AWS uses graviton2 for cloud build and not graviton3.