8 ms·
Graviton 3: First Impressions
- invalidname 4y agoWhile the article is interesting I would be more interested in details about carbon footprint and cost reduction. Also how would this impact more typical node, Java loads?
- Hizonner 4y agoYou know, if you wanted to improve carbon footprint, a better place to look might be at software bloat. The sheer number of times things get encoded and decoded to text is mind boggling. Especially in "typical node, Java loads".
- tyingq 4y agoLogging and Cybersecurity are bloaty areas as well. I've seen plenty of AWS cost breakdowns where the cybersec functions were the highest percentage of spend. Or desktops where carbon black, or windows defender were using most of the CPU or IO cycles. And networks where syslog traffic was the biggest percentage of traffic.
- orangepurple 4y agoNorton, Symantec, and McAfee contribute greatly to global warming in the financial services sector. At least half of CPU cycles on employee laptops are devoted to them.
- Cthulhu_ 4y agoBut do they actually work? For years I've been of the opinion that most anti-virus solutions don't actually stop virusses, instead they give you a false sense of security and their messaging is intentionally alarmist to make individuals and organizations pay their subscription fees. In my limited and sheltered experience, the only viruses I've gotten in the past decade or so was from dodgy pirated stuff or big "download" button ads on download sites.
- MrBuddyCasino 4y agoAt best they don’t work, in reality they are an attack vector themselves and a performance nightmare. They should (mostly) not exist.
- MaxBarraclough 4y agoPresumably then they're knocking hours off the laptops' battery lives?
- Dunedan 4y agoAs AWS doesn't price services based on carbon footprint, you can't infer the carbon footprint from the cost. I agree however that certain AWS services are disproportional expensive.
- maxerickson 4y agoPresumably the price provides some sort of bounds. (Unless they are doing something like putting profits towards some sort of carbon maximization scheme)
- tyingq 4y agoWell and a fair amount of cybersec oriented services are a pattern of "sniff and copy every bit of data and do something with it" or "trawl all state". Which is inherently heavy.
- ip26 4y agoThis has been looked at; the electricity savings of more efficient programming are too small to be worth it with a fixed compute capacity. However, if you can reduce the amount of machines or machine-time you require, that can become material.
- jeffbee 4y agoVirtually 100% of cloud operating expenses are electricity, so you can pretty much assume that if it costs less it has a lower carbon footprint.
- _joel 4y ago+ Rent, support staff, development costs, regulation and compliance, network, maintenance (cooling, fire suppression + lots more), marketing. Speaking as someone who did sys admin for a small independent cloud provider, it definitely isn't virtually 100% of operating costs
- jeffbee 4y agoNo offense intended to your personal experience, but I don't think "small independent cloud" is terribly important in the global analysis. This paper concludes that TDP and TCO have become the same thing, i.e. power is heat, power is opex. https://www.gwern.net/docs/ai/scaling/hardware/2021-jouppi.pdf https://www.gwern.net/docs/ai/scaling/hardware/2021-jouppi.p...
- ranman 4y agoOne key call out re: power from that paper - "Most OpEx cost is for provisioning power and not for electricity use [2], so saving power already provisioned for doesn’t improve TCO as much as one might hope."
- ranman 4y agoI'd peg it closer to 75%. There's some significant opex on the endpoint connectivity side as well. That's why a lot of these providers are buying stakes in some of the transoceanic cable projects (and some terrestrial ones as well). Or they're agreeing not to charge each other (kind of sort of- see open bandwidth alliance).
- mr_toad 4y agoDepreciation is an operating expense. Not a cash expense, but if you ignore depreciation you’ll end up like Detroit.
- shepherdjerred 4y agoAWS is pushing to move its internal services (most which are in Java) to graviton, so I would expect it to be excellent for “normal” workloads/languages
- Hizonner 4y agoIt feels like we've gone badly wrong somewhere when processors have to spend so many of their resources guessing about the program. I am not saying I have a solution, just that feeling.
- tyingq 4y agoThere's the Mill CPU, which sounds terrific on paper. Hard to gauge when it might turn into something commercially usable though.
- 0xCMP 4y agoMill would definitely make things more interesting. They were supposed to have their simulator online a while ago, but sounds like they needed to redo the work on the compiler (from what I understood). Once that comes out it sounds like the next step is getting the simulator online for people to play with.
- rwmj 4y agoUli Drepper has this tool which you can use to annotate source code with explanations of which optimisations are applied. In this case it would rely on GCC recognizing branches which are hard to predict (eg. a branch in an inner loop which is data-dependent), and I'm not sure GCC is able to do that. https://github.com/drepper/optmark https://github.com/drepper/optmark
- cesaref 4y agoI thought this was the reasoning behind Itanium, the idea that scheduling could be worked out in advance by the compiler (probably profile guided from tests or something like that) which would reduce the latency and silicon cost of implementations. However, it wasn't exactly a raging success, with I think the predicted amazing compiler tech not materialising, but maybe it is the right answer, but the implementation was wrong? I'm no CPU expert...
- Hizonner 4y agoI'm not sure what happened with Itanium. I do think a big part of the problem is that people want to distribute binaries that will run on a lot of CPUs that are physically really different inside. But nowadays there's JIT compilation even for JavaScript, so you could distribute something like LLVM, or even (ecch) JavaScript itself, and have the "compiler scheduling" happen at installation time or even at program start.
- rwmj 4y ago> GCC will flat out refuse to emit SVE instructions (at least in our limited experience), even if you use assembly, This seems ... wrong? I haven't tried it but according to the link below SVE2 intrinsics are supported in GCC 10 (and Clang 9): https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/making-the-most-of-the-arm-architecture-in-gcc-10 https://community.arm.com/arm-community-blogs/b/tools-softwa...
- adrian_b 4y agoYes, gcc 10.1 has introduced support for the SVE2 intrinsics (ACLE). Moreover, starting with the 8.1 version, gcc began to use SVE in certain cases when it succeeded to auto-vectorize loops (if the correct -march option had been used). Nevertheless, many Linux distributions are still shipped with older gcc versions, so SVE/SVE2 does not work with the available compiler or cross-compiler. You must upgrade gcc to 10.1 or a newer version.
- Dunedan 4y ago> The final result is a chip that lets AWS sell each Graviton 3 core at a lower price, while still delivering a significant performance boost over their previous Graviton 2 chip. That's not correct. AWS sells Graviton 3 based EC2 instances at a higher price than Graviton 2 based instances! For example a c6g.large instance (powered by Graviton 2) costs $0.068/hour in us-east-1, while a c7g.large instance (powered by Graviton 3) costs $0.0725/hour [1]. Both instances have the same core count and memory, although c7g instances have slightly better network throughput. I believe that is pretty unusual as, if my memory serves me right, newer instances family generations are usually cheaper than the previous generation. [1]: https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/
- myroon5 4y agoDefinitely unusual, as the graph here shows: https://github.com/patmyron/cloud/ https://github.com/patmyron/cloud/
- pattato 4y agoDid you create these?
- WASDx 4y agoCould it be due to increasing global energy prices?
- usefulcat 4y agoI don't follow. You seem to be implying that Amazon would like to reduce their electricity usage. If so, shouldn't they be charging less for the more efficient instance type?
- nine_k 4y agoNo, they charge for compute, which the new CPU provides more of, even though it consumes the same amount of electricity as a unit.
- tomrod 4y agoVery interesting! I'm not terribly well versed in ARM vs x86 so its helpful to see these kinds of benchmarks and reports. One bit of feedback for the author: the sliding scale is helpful, but the y axes are different between the visualizations so you cannot see the apples to apples comparison needed. Suggest regenerating those.
- DeathArrow 4y agoSuch a shame we can't play with a socketed CPU like this and a motherboard with EFI support as a workstation.
- wmf 4y agoRelated Graviton 3 benchmarks: https://www.phoronix.com/scan.php?page=article&item=graviton3-amd-intel&num=1 https://www.phoronix.com/scan.php?page=article&item=graviton...
- bullen 4y agoWhat about memory contention when many cores try to write/read the same memory? There is no point to add more cores if they can't cooperate. How come I'm the only one pointning this out? I think 4 cores will max out the memory contention, so keep on piling these 128 core heaters on. But they will not outlive a simple Raspberry 4!?
- Sirened 4y agoIt's likely that it's going to need a post on its own since it's an extremely complicated topic. Someone else wrote an awesome post about this for the Neoverse 2 chips [1] and they found that with LSE atomics, the N2 performs as well or better than Icelake. Given gravitron3 has a much wider fabric, I would assume this lead only improves. [1] https://travisdowns.github.io/blog/2020/07/06/concurrency-costs.html https://travisdowns.github.io/blog/2020/07/06/concurrency-co...
- bullen 4y agoAh, yes I remember this post, but it reads pretty cryptic to me. I would like to know what the slowdowns actually become in practice, does it add latency to the execution of other threads and how will the machine as a whole behave? I know M4 had much better multicore shared memory perf. than M3, but now both of those are old and I don't have users to test anything now.
- electricshampo1 4y agoThe whole chip in general will be used in aggregate by independent vms/containers etc that do NOT read and write to the same memory. Some kernel datastructures within a given vm are still shared, ditto for within a single process, but good design minimizes that (per cpu/thread data structures, sharded locks, etc etc).
- MaxBarraclough 4y agoI don't think they were referring to contention across VM boundaries.
- 4y ago
- ykevinator2 4y agoNo burstable graviton3's yet :-(
- jadbox 4y agoI'd love to see benchmarks for webservers like Node or Py.
- WhitneyLand 4y agoHow much can SVE instructions help with machine learning? I’ve wondered why Apple Silicon made the trade off decision to not include SVE support yet, given that support for lower precision FP vectorization seems like it could have made their NVidia perf gap smaller.
- Erlangen 4y agoI don't understand these graphs titled "Branch Predictor Pattern Recognition". What do they mean? Could someone here explain it a bit in detail? Thanks ahead.
- ip26 4y agoIt’s showing you how complicated of a branch sequence it can train on. Simple sequences are in the lower left, complex in the top right, and the ideal result would be a flat surface at the bottom. The earlier the surface rises into a mesa, the easier the branch predictor was overwhelmed.
- jazzythom 4y agoNeato
- nnx 4y agoI wonder how Graviton3 compare with M1 Max/Ultra, for the same number of CPU cores. The GPU and "ML" cores would totally destroy it for some use cases tho.