6 ms·
Graviton2 and Graviton3
- pm90 5y agoArm seems poised to replace x86 in servers. If I were Intel this would make me really nervous.
- taf2 5y agoPossibly on the desktop too... I imagine we'll see many m1 like windows pc options in the near future...
- ta988 5y agoThey are coming. Clearly not as impressive as the latest m1 max and pro, but getting there.
- betaby 5y agoVery unlikely. See for example Linus reasoning https://www.realworldtech.com/forum/?threadid=183440&curpostid=183486 https://www.realworldtech.com/forum/?threadid=183440&curpost...
- nichch 5y agoUnlikely for now. The ball has just started rolling with the M1.
- faeriechangling 5y agoLinus's reasoning is sound, but the issue is that ARM development platforms are becoming a thing and to be honest I see x86 as being in the early stages of a death spiral and so does Intel the way they're focusing on the fabrication side of their business. If anything programmers are adopting ARM based computers faster than the rest of the market. As pretty much every developer tool gets ported for Apple silicon every company is going to shrug and go "May as well release an ARM Windows/ARM Linux build as well".
- vineyardmike 5y agoI totally agree with everything you said except that devs are switching faster. I think first to switch was low end chrome books and surface go type devices. M1 is pulling the devs and professionals in, and gaming will be the last holdout (due to optimized IP that may be abandonware and never updated).
- ta988 5y agoThe good thing I see at work is that we all make everything work for x86 and arm. So we can deploy on any kind of cloud platform cpu and not worry about that anymore.
- dilyevsky 5y ago1. Fewer and fewer people run their stack on the laptop. There is tooling today to run even unit tests remotely pretty painlessly like bazel (and the like) and docker 2. With languages like java, go, python, node it doesn’t even matter 3. Devs are migrating to arm en masse (M1s)
- ta988 5y agoAgree with 1. I'm part of 3. Regarding 2, it does matter for anything that has bits of optimized c code that was only done for x86. I have a lot of Node and Python things that don't run on my M1 natively (they even crash on qemu x86 vms whatever the kind of cpu features I emulate).
- dilyevsky 5y agoRight which highlights why C is an outdated language that didn’t fully live up to its original promise. Linus’s entire argument seems to hinge on the premise that most userland devs still use it as their primary language. Eventually those python and node bugs will get fixed and people will largely not care
- jeffreyrogers 5y agoLinus's argument is that devs will use the same processors in production that they develop on. But everyone already has to develop for ARM because mobile runs ARM. And now the M1 Macs do too (and these AWS servers). So if you're forced to use ARM because of mobile and now there are good options for desktop and servers to use ARM as well, I don't see why people wouldn't switch to them. Basically Linus's own logic seems to contradict his claim.
- solatic 5y agoWe've been migrating our production to Graviton2 (now Graviton3). Our developers run x86 Macs. Everything runs on the JVM, Python, Node, Go, so nobody feels like there's a difference. The ARM transition has been transparent for us. Linus' reasoning makes sense, but the real world disagrees with him (at least in our case).
- vlovich123 5y agoI think the flaw in Linus’ argument is that this happened in the 90s-2010s for x86. A foundational time, especially for his worldview, but I don’t know that the pattern repeats (some of his viewpoint is colored by his time at Transmeta). The development world today looks very different. Back then, language support for other architectures was more bespoke and CPU vendors had to add support for their chips. Today, there are plenty of very rich, platform-agnostic (both CPU and OS) libraries. Additionally, mobile development has sufficiently matured ARM development that I don’t think that argument holds. If it did, then developers wouldn’t be able to develop on their x86 MacBooks and deploy to their mobile Apple devices (yes it’s ARM now but it hasn’t been for the majority). I think the plain x86-box -> server story is pretty solid for but the cloud has changed that. Everyone is now starting out in the cloud with CPU-agnostic languages where switching architectures usually is as simple as changing 1 line in a config. In some cases it matters but the vast majority of SW dev shops don’t feel this like you used to in the 90s and 00s. Plus M1s now provide developers with local ARM development.
- _ph_ 5y agoThat is, as far as the reasoning applies, why I consider the M1 Macs so pivotal. The MB Pro already was a very popular machine for developers. Now it not only got much faster and better, it also offers access to great ARM development machines. Be it for the largest market, smartphones, or for the cloud solutions based on ARM machines as the Graviton.
- jeffreyrogers 5y ago5 or 6 years ago Marc Andreesen was saying this would happen eventually. I was skeptical when I first heard the claim, but it's seeming more and more likely.
- freemint 5y agoDon't forget Ampere's A1 i found them really, really impressive for SAT solving and that you can get them at 1ct/core/hour at Orcale makes them really financially attractive.
- erulabs 5y ago> most workloads could actually run more efficiently if they had more memory bandwidth and lower latency access to memory Turns out memory access speed is more or less the entire game for everything except scientific computing or insanely optimized code. In the real world, CPU frequency seems to matter much less than DRAM timings, for example, in everything but extremely well engineered games. It'll be interesting to learn (if we ever do) how much of the "real-world" 25% performance gain is solely due to DDR5. I remember getting my AMD K8 Opteron around 2003 or 2004 with the first on-die memory controller. Absolutely demolished Intel chips at the time in non-synthetic benchmarks.
- shoo 5y ago> everything except scientific computing or insanely optimized code for insanely unoptimized code, such as accidentally ending up writing something compute intensive in pure python, its very plausible for it to be compute constrained -- but less because of the hardware and more because 99 %or 99.9% of the operations you're asking the cpu to perform are effectively waste.
- erulabs 5y agoCertainly, but many "wasteful" operations are reading/writing main memory unnecessarily - and since a single trip out to memory and back can take many many CPU cycles, typically optimizing memory access time does more for "bad code" than optimizing the number of CPU cycles per second. But obviously, you're right too - faster is faster after all :)
- throwaway894345 5y agoI'm skeptical of this. I would expect many of those wasted instructions to be expensive memory instructions. I'm pretty sure regular Python code allocates a lot and the interpreter spends lots of time chasing after memory references.
- hinkley 5y agoIn the 90’s there were people trying to solve this problem by putting a small CPU on chip with the memory and running some operations there. I routinely wonder why memory hasn’t gotten smarter over time.
- phamilton 5y agoA recurring theme is "build a processor that performs well on real workloads". It occurs to me that AWS might have far more insight into "real workloads" than any CPU designer out there. Do they track things like L1 cache misses across all of EC2?
- virtuallynathan 5y agoHard to track for other people’s VMs, but they probably have (or can sample) that data for every AWS-operated service (dynamo, S3, redshift, etc..)
- sroussey 5y agoYeah, this is key. They definitely optimize for their own services. And they don’t run S3 and redshift on the same cpu/server at the same time.
- vineyardmike 5y agoThey may not run those particular services on the same hosts but they heavily use Lambda (and docker) which can share hosts and be tossed around the data centers to saturate cores.
- vineyardmike 5y agoThere is a strong internal mandate for internal services to switch over to gravitron. So they either likely have this data, or are just trying to free up more x64 cores for external customers.
- uplifter 5y agoReality varies. Its a truism in optimization that the only valid benchmark is the task you are trying to accomplish. These chips have been optimized for an average of the tasks run on AWS (which is entirely sensible for them), but that doesn't mean they'll be the best for your specific job.
- trhway 5y ago
- wmf 5y agoI can't escape the feeling that AWS is taking credit for industry trends (DDR5) and Arm's decisions (Neoverse).
- magila 5y agoYeah, the tone of these talks is kind of weird. They talk about how "we decided to do foo" when the reality is "we updated to the latest tech from our upstream providers which got us foo".
- pm90 5y agoLike how Apple takes credit for packaging new technology in an easy to use product? What’s wrong with that? They’re not exactly hiding it.
- aledalgrande 5y agoIsn't making the CPU wider one of the things Apple also did with M1? Doesn't feel like they are the first.
- wmf 5y agoApple designed the M1. AWS is (probably) using off-the-shelf Neoverse V1 cores that they did not design. [Imagine "you made this, I made this" meme here]
- vineyardmike 5y agoThey have a huge design team making custom silicon. They deserve a bit more credit even if they’re leaning on ARM IP.
- ashtonkem 5y agoThe rest of the server has to be designed too, since they can’t just buy from Dell or some other OEM and put Graviton into it. At their scale this means management software and hardware too, which is a right old pain in the butt to design and deploy.
- veselin 5y agoWell, I disagree with pretty much everything in the claims. First, most real unoptimised code faces many issues before memory bandwidth. During my PhD, the optimisation guys doing spiral.net sat nextdoor and they produced beautiful plots of what limits performance for a bunch of tasks and how each optimisation they do removes an upper bound line until last they get to some bandwidth limitation. Real code will likely have false IPC dependencies, memory latency problems due to pointer chaising or branch mispredictions well before memory bandwidth. Then the database workload is something I would consider insanely optimized. Most engines are in fierce performance competition. And normally they hit the memory bandwidth in the end. This probably answers why the author is not comparing to EPYC instances that have the memory bandwidth to compete with Graviton. Then the claims that they choose not to implement SMT or to use DDR5 are both coming from their upstream providers.
- wyldfire 5y agoWouldn't SMT be a feature that you are free to use when designing your own cores? I'm assuming Amazon has an architectural license (Annapurna acquisition probably had them, this team is likely the Graviton design team at AWS). So who is the upstream provider? ARM? And if they designed the CPU wouldn't they decide which memory controller is appropriate? Seems like AWS should get as much credit for their CPUs as Apple gets for theirs. Bottom line for Graviton is that a lot of AWS customers rely on open source software that already works well on ARM. And the AWS customer themselves often write their code in a language that will work just as well on ARM. So AWS can offer it's customers tremendous value with minimal transition pain. But sure, if you have a CPU-bound workload, it'll do better on EPYC or Xeon than Graviton.
- adfgdtyhaet 5y agoI don't understand this at all. The first and second transcripts seemingly contradict each other. The first one says: >Cores got so big and complex that it was hard to keep everything utilized. But then the second one is about how they improved performance by making the cores bigger and more complex. Why is it possible to feed their wide core but not the competition's? Why is it that idle transistors are bad in the competition but Graviton benefits from specialized vector instructions that are only useful in some workloads? >With Graviton2, one of the things we prioritized was large core local caches. In fact, the core local L1 caches on Graviton2 are twice as large as the current generation x86 processors. This doesn't make sense. All modern x86 machines have both an L1 and L2 cache that are local to each core, with only the L3 cache being shared. My laptop has a total of 288 kiB of cache dedicated to each core. The fact that the competition uses 32 kiB L1 caches has nothing to do with a difference in philosophy. Everyone realizes that caching is important and everyone uses the biggest caches they can get away with. The reason x86 designers chose a smaller cache is because, in their designs, increasing the cache size would reduce performance. Large caches are slower than small ones. Increasing the hit rate is not necessarily worth it if it makes every cache access more expensive. >most workloads could actually run more efficiently if they had more memory bandwidth and lower latency access to memory In other news, water is wet. Memory has been the bottleneck for as long as I've been alive.