5 ms·

I'm still extremely suspicious of all these benchmarks, because they don't seem to correlate with my real-world experience. All these benchmarks claim to be co

I'm still extremely suspicious of all these benchmarks, because they don't seem to correlate with my real-world experience. All these benchmarks claim to be comparing real-world applications, but I don't trust them because of what they always show with regards to mobile i9 and desktop Ryzen. I've got a Intel Macbook Pro, and a Ryzen desktop. When I'm using them for development, my Ryzen is absolutely wiping the floor with the Macbook, Android builds complete in a fraction of the time, and I barely ever have to wait for anything to complete. The story is much the same with almost any development tool or process I use. Yet benchmarks like these show comparatively minor performance differences between the two. I just get the impression these benchmarks are being cherrypicked, or are "nerfed" in some way on multicore systems.

ogre_codes 6y ago
This bit from the article might bring things into focus: > "I add the following comment as of Dec 14: we care how fast our dev computer runs. the multi-core advantage of 3900X is useless for most of the benchmarks here that don't represent the real-world production performance at all but devs usually work on their laptops, desktops, etc most of the time so I think such real-world development performance benchmarks make it worthwhile though. of course, Ryzen 3900X will perform way better than M1 and Intel when it comes to production, mostly achieved by parallelizing."
- eropple 6y ago
  That explanation doesn't make a lot of sense, though. Like, I mostly do web-dev type stuff these days and my 3900X is alarmingly busy. Background compiles, entirely too many Chromium (or Electron) threads, all my other random running applications--the thing is significantly snappier even than the 2600X I replaced it with, and it's gotten to the point where using a laptop shy of a desktop replacement is pretty aggravating. (I'm not interested in going back to MacOS so I'm not really looking at the M1 as anything other than intellectually interesting at the moment, but if I were it certainly at least seems like it might compete there.)
  - ogre_codes 6y ago
    I agree. Though much of the writing on this benchmark/ blog seems confusing to me and the charts are all over the place in terms of style as well.
gzer0 6y ago
Would you also be able to run some of these? They're publicly available; the more datasets we have the better picture we get.
- merdadicapra 6y ago
  I've tried But the instructions are not clear. Take for example the `als` benchmark for java renaissance. It has a lot of options, one of them used to not force GC. Was it set? What version of Java? I'm running openjdk 11. When I launched it it gave me this warning 20/12/14 18:55:20 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Would a native hadoop library make a difference? And how much? There are other warnings complaining about lacking some additional library and switching to default, but what really surprised me is that on this particular laptop that mounts an Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz with 8 cores, a CPU at least 5 years old, I could beat the i9 benchmark. The `als` benchmark for the i9 completed in 3581.838 ms, on my (very old) laptop the running time was 3533.964 ms. Raw data from here (https://docs.google.com/spreadsheets/d/1g4U7LAImfEcXRihJbySZcRr32tn6WSWAtslfXltds58/edit?usp=sharing https://docs.google.com/spreadsheets/d/1g4U7LAImfEcXRihJbySZ...) Tab "Java Renaissance (Chart)" Col O, Row 4
- sdflhasjd 6y ago
  So, I picked out a couple that I could run without installing too much extra: Ryzen 3900X: antd-admin webpack build: ~23 seconds. Not surprising, it's just single core. golang.org/x/benchmarks, BenchmarkBuild-8: 4.61 seconds, so far off the value of 19.65s in the article that I feel I'm getting some suspicions confirmed...
  - outworlder 6y ago
    > that I feel I'm getting some suspicions confirmed... Be careful. Benchmarking is hard. Before assuming malice, try to control for other variables.
  - merb 6y ago
    well he used bad ram and didn't specify his ssd. basically compilation is cpu+ssd+memory so if one of these suck you get what he did. a good cpu with garbage components. not sure why somebody buys an X5xx mainboard and uses 3200mhz memory (which he hopefully ran in dual channel...) also my macbook pro is slower.
andrepew 6y ago
In laptops, the performance of the CPU in sustained use is heavily tied to the cooling solution. Almost all laptops will hit a performance ceiling due to thermal throttling. A desktop is at a massive advantage here. There are also potentially OS differences or other config differences. You'd need to normalize everything to get an accurate picture when comparing the two CPUs.
- dragontamer 6y ago
  But desktop cores use vanishingly small amounts of power for single-threaded tasks. The only time you really run into thermal-limits on desktops (or laptops) is when you're hitting SIMD across many cores: 16-thread or 32-thread tasks with AVX and the like.
  - andrepew 6y ago
    They're talking about a difference in real world dev use. My Macbook Pro thermal throttles just running Chrome sometimes (which is always running while I develop). If I started a single threaded compile alongside that, it most definitely would run slower.
  - astrange 6y ago
    Multi-threaded tasks rarely exist in the wild. Compiling a project runs multiple processes at once, which is strictly better because it doesn’t need synchronization.
floatboth 6y ago
They're mostly some weird "java bench" "python bench" thingies rather than multicore builds of large C++/Rust/etc projects
cesarb 6y ago
Are your real-world applications running on the same operating system? According to a thread on RWT (https://www.realworldtech.com/forum/?threadid=197081&curpostid=197081 https://www.realworldtech.com/forum/?threadid=197081&curpost...), macOS is much slower than Linux on the same hardware for some tasks like building software. If your Ryzen desktop is running Linux and your Macbook Pro is running macOS, that would be enough to explain at least part of the speed difference.
- manquer 6y ago
  However it may not be fair comparison as apple has no incentive to support optimization outside their stack. A skewed better performance in OSX is not just about the processor but about their stack preferring one arch. Benchmark for OS+CPU stack should be their best combination not where one vendor has advantage.
- nextos 6y ago
  Yes, for that reason it'd be amazing if one could run Linux on M1 hardware. Despite some old Intel models were great for this purpose and Linus has shown some interest, I'm not holding my breath. It'd be nice if we saw some good competing CPUs. Nvidia is in my opinion is well positioned midterm due to the acquisition of ARM. But I'd prefer AMD due to openness and staying within the PC standard.
- psadauskas 6y ago
  In my experience of the last several years, the MacOS filesystem layer for lots of tiny files seems to be much slower than Linux. In our app, `rm -rf node_modules` (75,000 files, 1.1GB) takes about 10x longer on MacOS than it does on Linux (btrfs, zfs, ext4 all similar). `yarn install` has a similar difference between the two. I'd imagine most compiling tasks involving lots of tiny files would be impacted by the filesystem performance as well, hamstringing the performance of the M1.
csomar 6y ago
Your Ryzen probably comes with a Cooler (wraith prism). The CPU slows down considerably when it gets hot. Also, if you built the thing yourself, you might have added a super fast RAM, Motherboard, Nvme, and a capable graphic card. Comparing Desktop to Laptop is Apple to Oranges. Otherwise the desktop market will no longer exist.
- gameswithgo 6y ago
  that doesn't address any of his comparisons. the person doing these benchmarks also had a cooler, fast ram, motherboard has no impact, mac air has FASTER ram and a good ssd also.
  - csomar 6y ago
    How do you know? Parent didn't specify how fast his RAM is. The new macbook air is definitively fast but you could get something custom built that's faster (4400Mhz)
mayli 6y ago
Yeah, those benchmarks seems to be highly biased over non-multicore applications. The geekbench showed pretty much the same, M1 beats 3900X on single core performance, but only have 0.5 on overall multi-core score. https://browser.geekbench.com/v5/cpu/compare/5326018?baseline=5326348 https://browser.geekbench.com/v5/cpu/compare/5326018?baselin... This isn't a benchmark that "focused on the real-world programming", it's more of a benchmark on single core performance.
BatteryMountain 6y ago
Same for me. I have a Ryzen 5 and it feels about 3x faster than anything else in the house, and it's not even the most expensive chip in the house. Next year definitely getting another Amd chip to upgrade my now old ryzen 5. Not touching Intel again in a long time.
- Geminidog 6y ago
  Except the parent poster is comparing Intel macbooks with a ryzen. He's not comparing M1 arm chips like the post. He's mistaken. The amazing thing here if the claims are true is that a simple macbook air has performance on par with a full blown Ryzen desktop.
- josephg 6y ago
  Me too. I did a test rust compile to quantify the difference. A project that takes 2 minutes on my old 2016 MacBook Pro takes 20 seconds on my new ryzen 5800 desktop machine. And everything launches instantly. It’s an absolute breath of fresh air and I love it.
  - merb 6y ago
    keep in mind that you should not use your 2016 MacBook with an external monitor (without clasmshell)
zamalek 6y ago
> I'm still extremely suspicious of all these benchmarks, because they don't seem to correlate with my real-world experience. Couple that with using the latest and greatest from Apple, alongside the previous Ryzen generation when substantial improvements are happening each generation. > You may wonder why I used 3900X instead of Ryzen 5000-series CPUs: Because I don't have it. In addition, 3200MT/s RAM is not adequate for Ryzen because that sets the IF to 1600MHz which leaves a huge amount of performance on the floor, assuming OP enabled XMP at all. Furthermore, those results are suspect because of OP is (as you pointed out) incorrectly testing single-core performance, Intel should be besting AMD for that generation (only the latest generation is superior across all dimensions). This is fan-made marketing material. Performance-to-watt would have shown the true advantage of M1, not this cherry-picked nonsense.
markstos 6y ago
Share benchmarks of your setup then.
Geminidog 6y ago
>I've got a Intel Macbook Pro, and a Ryzen desktop. Read the title. It's Apple M1, these are apple arm chips NOT intel chips. Your "real-world experience" doesn't correlate because you're comparing Apple Intel macbooks with Intel desktops. You need to compare the new Apple M1 macbooks with ARM chips to feel the difference.
- cozzyd 6y ago
  You misunderstood parent. The benchmark shows i9 vs. Ryzen (in addition to the M1), which does not correlate with parent's experience.

ogre_codes 6y ago

This bit from the article might bring things into focus: > "I add the following comment as of Dec 14: we care how fast our dev computer runs. the multi-core advantage of 3900X is useless for most of the benchmarks here that don't represent the real-world production performance at all but devs usually work on their laptops, desktops, etc most of the time so I think such real-world development performance benchmarks make it worthwhile though. of course, Ryzen 3900X will perform way better than M1 and Intel when it comes to production, mostly achieved by parallelizing."

eropple 6y ago
That explanation doesn't make a lot of sense, though. Like, I mostly do web-dev type stuff these days and my 3900X is alarmingly busy. Background compiles, entirely too many Chromium (or Electron) threads, all my other random running applications--the thing is significantly snappier even than the 2600X I replaced it with, and it's gotten to the point where using a laptop shy of a desktop replacement is pretty aggravating. (I'm not interested in going back to MacOS so I'm not really looking at the M1 as anything other than intellectually interesting at the moment, but if I were it certainly at least seems like it might compete there.)
- ogre_codes 6y ago
  I agree. Though much of the writing on this benchmark/ blog seems confusing to me and the charts are all over the place in terms of style as well.

gzer0 6y ago

Would you also be able to run some of these? They're publicly available; the more datasets we have the better picture we get.

merdadicapra 6y ago
I've tried But the instructions are not clear. Take for example the `als` benchmark for java renaissance. It has a lot of options, one of them used to not force GC. Was it set? What version of Java? I'm running openjdk 11. When I launched it it gave me this warning 20/12/14 18:55:20 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Would a native hadoop library make a difference? And how much? There are other warnings complaining about lacking some additional library and switching to default, but what really surprised me is that on this particular laptop that mounts an Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz with 8 cores, a CPU at least 5 years old, I could beat the i9 benchmark. The `als` benchmark for the i9 completed in 3581.838 ms, on my (very old) laptop the running time was 3533.964 ms. Raw data from here (https://docs.google.com/spreadsheets/d/1g4U7LAImfEcXRihJbySZcRr32tn6WSWAtslfXltds58/edit?usp=sharing https://docs.google.com/spreadsheets/d/1g4U7LAImfEcXRihJbySZ...) Tab "Java Renaissance (Chart)" Col O, Row 4
sdflhasjd 6y ago
So, I picked out a couple that I could run without installing too much extra: Ryzen 3900X: antd-admin webpack build: ~23 seconds. Not surprising, it's just single core. golang.org/x/benchmarks, BenchmarkBuild-8: 4.61 seconds, so far off the value of 19.65s in the article that I feel I'm getting some suspicions confirmed...
- outworlder 6y ago
  > that I feel I'm getting some suspicions confirmed... Be careful. Benchmarking is hard. Before assuming malice, try to control for other variables.
- merb 6y ago
  well he used bad ram and didn't specify his ssd. basically compilation is cpu+ssd+memory so if one of these suck you get what he did. a good cpu with garbage components. not sure why somebody buys an X5xx mainboard and uses 3200mhz memory (which he hopefully ran in dual channel...) also my macbook pro is slower.

andrepew 6y ago

In laptops, the performance of the CPU in sustained use is heavily tied to the cooling solution. Almost all laptops will hit a performance ceiling due to thermal throttling. A desktop is at a massive advantage here. There are also potentially OS differences or other config differences. You'd need to normalize everything to get an accurate picture when comparing the two CPUs.

dragontamer 6y ago
But desktop cores use vanishingly small amounts of power for single-threaded tasks. The only time you really run into thermal-limits on desktops (or laptops) is when you're hitting SIMD across many cores: 16-thread or 32-thread tasks with AVX and the like.
- andrepew 6y ago
  They're talking about a difference in real world dev use. My Macbook Pro thermal throttles just running Chrome sometimes (which is always running while I develop). If I started a single threaded compile alongside that, it most definitely would run slower.
- astrange 6y ago
  Multi-threaded tasks rarely exist in the wild. Compiling a project runs multiple processes at once, which is strictly better because it doesn’t need synchronization.

floatboth 6y ago

They're mostly some weird "java bench" "python bench" thingies rather than multicore builds of large C++/Rust/etc projects

cesarb 6y ago

Are your real-world applications running on the same operating system? According to a thread on RWT (https://www.realworldtech.com/forum/?threadid=197081&curpostid=197081 https://www.realworldtech.com/forum/?threadid=197081&curpost...), macOS is much slower than Linux on the same hardware for some tasks like building software. If your Ryzen desktop is running Linux and your Macbook Pro is running macOS, that would be enough to explain at least part of the speed difference.

manquer 6y ago
However it may not be fair comparison as apple has no incentive to support optimization outside their stack. A skewed better performance in OSX is not just about the processor but about their stack preferring one arch. Benchmark for OS+CPU stack should be their best combination not where one vendor has advantage.
nextos 6y ago
Yes, for that reason it'd be amazing if one could run Linux on M1 hardware. Despite some old Intel models were great for this purpose and Linus has shown some interest, I'm not holding my breath. It'd be nice if we saw some good competing CPUs. Nvidia is in my opinion is well positioned midterm due to the acquisition of ARM. But I'd prefer AMD due to openness and staying within the PC standard.
psadauskas 6y ago
In my experience of the last several years, the MacOS filesystem layer for lots of tiny files seems to be much slower than Linux. In our app, `rm -rf node_modules` (75,000 files, 1.1GB) takes about 10x longer on MacOS than it does on Linux (btrfs, zfs, ext4 all similar). `yarn install` has a similar difference between the two. I'd imagine most compiling tasks involving lots of tiny files would be impacted by the filesystem performance as well, hamstringing the performance of the M1.

csomar 6y ago

Your Ryzen probably comes with a Cooler (wraith prism). The CPU slows down considerably when it gets hot. Also, if you built the thing yourself, you might have added a super fast RAM, Motherboard, Nvme, and a capable graphic card. Comparing Desktop to Laptop is Apple to Oranges. Otherwise the desktop market will no longer exist.

gameswithgo 6y ago
that doesn't address any of his comparisons. the person doing these benchmarks also had a cooler, fast ram, motherboard has no impact, mac air has FASTER ram and a good ssd also.
- csomar 6y ago
  How do you know? Parent didn't specify how fast his RAM is. The new macbook air is definitively fast but you could get something custom built that's faster (4400Mhz)

mayli 6y ago

Yeah, those benchmarks seems to be highly biased over non-multicore applications. The geekbench showed pretty much the same, M1 beats 3900X on single core performance, but only have 0.5 on overall multi-core score. https://browser.geekbench.com/v5/cpu/compare/5326018?baseline=5326348 https://browser.geekbench.com/v5/cpu/compare/5326018?baselin... This isn't a benchmark that "focused on the real-world programming", it's more of a benchmark on single core performance.

BatteryMountain 6y ago

Same for me. I have a Ryzen 5 and it feels about 3x faster than anything else in the house, and it's not even the most expensive chip in the house. Next year definitely getting another Amd chip to upgrade my now old ryzen 5. Not touching Intel again in a long time.

Geminidog 6y ago
Except the parent poster is comparing Intel macbooks with a ryzen. He's not comparing M1 arm chips like the post. He's mistaken. The amazing thing here if the claims are true is that a simple macbook air has performance on par with a full blown Ryzen desktop.
josephg 6y ago
Me too. I did a test rust compile to quantify the difference. A project that takes 2 minutes on my old 2016 MacBook Pro takes 20 seconds on my new ryzen 5800 desktop machine. And everything launches instantly. It’s an absolute breath of fresh air and I love it.
- merb 6y ago
  keep in mind that you should not use your 2016 MacBook with an external monitor (without clasmshell)

zamalek 6y ago

> I'm still extremely suspicious of all these benchmarks, because they don't seem to correlate with my real-world experience. Couple that with using the latest and greatest from Apple, alongside the previous Ryzen generation when substantial improvements are happening each generation. > You may wonder why I used 3900X instead of Ryzen 5000-series CPUs: Because I don't have it. In addition, 3200MT/s RAM is not adequate for Ryzen because that sets the IF to 1600MHz which leaves a huge amount of performance on the floor, assuming OP enabled XMP at all. Furthermore, those results are suspect because of OP is (as you pointed out) incorrectly testing single-core performance, Intel should be besting AMD for that generation (only the latest generation is superior across all dimensions). This is fan-made marketing material. Performance-to-watt would have shown the true advantage of M1, not this cherry-picked nonsense.

markstos 6y ago

Share benchmarks of your setup then.

Geminidog 6y ago

>I've got a Intel Macbook Pro, and a Ryzen desktop. Read the title. It's Apple M1, these are apple arm chips NOT intel chips. Your "real-world experience" doesn't correlate because you're comparing Apple Intel macbooks with Intel desktops. You need to compare the new Apple M1 macbooks with ARM chips to feel the difference.

cozzyd 6y ago
You misunderstood parent. The benchmark shows i9 vs. Ryzen (in addition to the M1), which does not correlate with parent's experience.