10 ms·
Graviton 3, Apple M2 and Qualcomm 8cx 3rd gen: a URL parsing benchmark
- mhh__ 4y agoWithout more context about what the code actually does this doesn't tell me all that much, other than what I could guess from the intended usecases of the chips. The strength of Apple silicon is that it can crush benchmarks and transfer that power very well to real world concurrent workloads too e.g. is this basically just measuring the L1 latency? If not are the compilers generating the right instructions etc. (One would assume they are but I have had issues with getting good arm codegen previously, only to find that the compiler couldn't work out what ISA to target other than a conservative guess)
- renewiltord 4y agoGit repo available from post.
- mhh__ 4y agoI ain't readin' all that (OK maybe I will but lemire does do this quite a lot, his blog is 40% gems 60% slightly sloppy borderline factoids that only make sense if you think in exactly the same way he does)
- KerrAvon 4y agoYes. Since it's Ada, I'm suspicious of codegen tuning being a major factor here.
- wtallis 4y agoIs there something specific about this library that makes you suspicious, or are you assuming from the name that this is using the Ada programming language?
- zimpenfish 4y ago"Ada is a fast and spec-compliant URL parser written in C++." Wouldn't modern C++ compilers have decent codegen tuning for all these platforms?
- dan-robertson 4y agoIt does seem the benchmark has its data in cache, based on the timings. If the benchmark were only measuring L1 latency, what would that imply about the ‘scaling by inverse clock speed’ bit? My guess is as follows. Chips with higher clock rates will be penalised: (a) it is harder to decrease latencies (memory, pipeline length, etc) in absolute terms than run at a higher clock speed to maybe do non-memory things faster; and (b) if you’re waiting 5ns to read some data, that hurts you more after the scaling if your clock speed is higher. The fact that the M1 wins after the scaling despite the higher clock rate suggests to me that either they have a big advantage on memory latency or there’s some non-memory-latency advantage in scheduling or branch prediction that leads to more useful instructions being retired per cycle. But maybe I’m interpreting it the wrong way.
- Kwpolska 4y agoA comparison with x86_64 CPUs (e.g. those seen in comparable MacBooks and AWS machines) would be useful. Also, I'm not sure if "correcting" the numbers for 3 GHz is reasonable and reflects real-life performance. Perhaps some throttling could be applied to test the CPUs using a common frequency?
- deltaci 4y agoIt's a benchmark of GitHub Actions(Azure) vs a really old Macbook Pro 15, not exactly what you are looking for, but it tells the vibe already. https://buildjet.com/for-github-actions/blog/a-performance-review-of-github-actions-the-cost-of-slow-hardware https://buildjet.com/for-github-actions/blog/a-performance-r...
- 015a 4y agoThis is a big, general problem with CI providers I don't hear talked about enough: because they charge per-minute, they are actively incentivized to run on old hardware, slowing builds and milking more from customers in the process. Doubly-so when your CI is hosted by a major cloud provider who would otherwise have to scrap these old machines. I wish this were only a theoretical concern, a theoretical incentive, but its not. Github Actions is slow, and Gitlab suffers from a similar problem; their hosted SaaS runners are on GCP n1-standard-1 machines. The oldest machine type in GCP's fleet, the n1-standard-1 is powered by a variety of dusty, old CPUs Google Cloud has no other use for, from Sandy Bridge to Skylake. That's a 12 year old CPU.
- AnthonyMouse 4y agoThere are workloads where newer CPUs are dramatically faster (e.g. AVX-512), but in general the difference isn't huge. Most of what the newer CPUs get you is more cores and higher power efficiency, which you don't care about when you're paying per-vCPU. Which vCPU is faster, a ten year old Xeon E5-2643 v2 at 3.5GHz or a two year old Xeon Platinum 8352V at 2.1GHz? It depends on the workload. Which has more memory bandwidth per core? But the cloud provider prefers the latter because it has 500% more cores for 50% more power. Which is why the latter still goes for >$2000 and the former is <$15.
- jeffbee 4y agoTo me it would be somewhat more interesting to compare head-to-head mobile CPUs instead of comparing laptops and servers. In this particular microbenchmark, mobile 12th and 13th-generation Core performance cores, and even the efficiency cores on the 13th generation, are faster than the M2.
- Dalewyn 4y agoIntel 12th and 13th gen both use the same efficiency cores.
- jeffbee 4y agoWell, on the ones I happen to have on hand the 12th gen hits 3300MHz and the 13th gen goes all the way to 4200MHz.
- Dalewyn 4y agoYeah well, the efficiency cores on a 12900K will be faster than those on an N100, so what is your point? We're discussing overall compute power differences between CPU architectures, minute differences in performance between identical-architecture CPU cores stemming from higher clock speeds is outside the scope of this discussion.
- refulgentis 4y agoNot for me, I understand and have had this reaction and it sort of undervalues the value of related info. I learned a bit from him and your response for example
- scns 4y agoEven when they are faster, the M2 is on the same die as the RAM and the bandwidth and latency are way better. That matters for compilation or am i mistaken?
- jeffbee 4y ago
- smoldesu 4y ago> Note that you cannot blindly correct for frequency in this manner because it is not physically possible to just change the frequency as I did They're also not the same core architecture? Comparing ARM chips that conform to the same spec won't necessarily scale the same across frequencies. Even if all of these CPUs did have scaling clock speeds, their core logic is not the same. Hell, even the Firestorm and Icestorm cores on the M1 SOC shouldn't be considered directly comparable if you scale the clock speeds.
- deleted 4y ago[deleted]
- wmf 4y agoThat's the point. He knows they're different architectures (although X1 and V1 are related) so normalizing frequency exposes the architectural differences.
- psanford 4y agoIt looks like there's been some good progress on getting Linux running natively on the Windows Dev Kit 2023 hardware[0]. There was a previous discussion here about this hardware back in 2022-11[1]. [0]: https://github.com/linux-surface/surface-pro-x/issues/43 https://github.com/linux-surface/surface-pro-x/issues/43 [1]: https://news.ycombinator.com/item?id=33418044 https://news.ycombinator.com/item?id=33418044
- Thaxll 4y agoI think the performance of my oracle free instance ( arm cpu ) is 10x worse than those results.
- dylan604 4y agomy oracle free instance uses mariadb instead of mysql, but i'm guessing you meant that as the free instance provided by oracle instead of an instance not using anything from oracle. =)
- Thaxll 4y agoYes I'm talking about: https://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier_topic-Always_Free_Resources.htm https://docs.oracle.com/en-us/iaas/Content/FreeTier/freetier... Ampere A1 Compute instances
- monocasa 4y agoPart of what they don't mention is that Graviton 3 and the Snapdragon 8cx Gen 3 have pretty much the same processor core. The Neoverse V1 is only a slightly modified Cortex X1. Hence the same results when you account for clock frequency.
- joseph_grobbles 4y ago[dead]
- deleted 4y ago[deleted]
- bushbaba 4y agoSeems weird to compare the c7g.large vs m2 and not the largest VM sizes.
- monocasa 4y agoIt's a single threaded test that doesn't use a lot of memory.
- seiferteric 4y agoNot that it's for sure, but M3 is probably coming out late this year/early next year and will be on 3nm, so once again having a huge node advantage. Just seems like Apple will have the latest node before everyone else for the foreseeable future.
- webaholic 4y agoApple pays a premium to TSMC to reserve the early runs on the next gen nodes. They can do this because they can charge their users a premium for Apple devices. I am not sure the rest of the players have that much pricing power or margins.
- smoldesu 4y agoNvidia does, which is why they bought out the 4nm node (and beat Apple in GPU compute by a country mile).
- monocasa 4y agoApple A16 is on N4 too.
- MBCook 4y agoPlus they can guarantee big volumes. Even if only the pros get the new chip this year as rumored that’s still a very large order to make. Next year it’s likely all iPhones (plus possibly iPads) will also be on the new process. It looks like Apple sells around 200 million iPhones a year, and the two pro models are somewhat more popular combined than the non-pros. So even if we assume 2/3rds of Apple sales are older models, the pros would still need around 40 million chips on the new process in the first year. For comparison it looks like AMD sells about 80 million chips a year, across all CPU models.
- kramerger 4y agoActually, both Apple and AMD have reduced their orders with TSMC due to expected drop in sales. https://wccftech.com/tsmc-faces-order-cutback-from-major-5nm-and-7nm-customers-report/ https://wccftech.com/tsmc-faces-order-cutback-from-major-5nm...
- cpascal 4y agoThere's probably some virtualization overhead on the c7g.large, right?
- avrionov 4y agoFor comparison I got ns/url=194.976 on Apple MacBook Pro M1.
- MBCook 4y agoThat makes sense. The M2 is extremely similar to the M1 for a single core.
- ksec 4y agoIf I remember correctly, The 8cx 3rd Gen is based on Cortex X1, the same as Snapdragon 888. The Graviton 3 is based on Neoverse V1 which in itself is tweaked version of Neoverse N1 with relaxed pref / watt / die area and much improved SIMD work load. And N1 is based on Cortex X1. The current Snapdragon is on Cortex X3. With Cortex X4 coming soon.
- thejosh 4y agoWe've been having good success with Graviton 3 for our small data-processing workloads. It seems with Apple now focusing on AARCH64 the architecture being taken seriously by developers (because ya know, phones, raspberry pis etc weren't enough :)) and there is good support for most software I use day-to-day. Now it seems Amazon is pushing HARD on Graviton, as the release of 3 has seen many services now being featured running on it. They made great process with Graviton 3, so it's not hard to see why. Exciting times :)
- ausudhz 4y agoIt's also due to cost. At scale is cheaper to produce your own professors than buy it from a vendor. Not to mention performance and energy consumption benefits. The phone had it to begin with due to the low consumption and easy instructions set. Then the Moore's law did the rest and now they're powerful enough for data centers
- tuetuopay 4y agoFWIW: - on an AMD Ryzen 5 Pro 4650U (laptop) I get 270 ns/url - on an Ampere Altra Max M128 I get 340ns/url So yeah, this show how great Apple's M2 are
- moralestapia 4y agoApple M2 is 190 ns/url for those interested
- peoplefromibiza 4y agotried on ~2021 i9-11950H (11th Gen Intel, laptop), got 166 ns/url