4 ms·

The methology used in these tests is utter nonsense. Yes, there is something to be learned by his results but it has a the very least 50% to do with DAW intern

by n9 3y ago

The methology used in these tests is utter nonsense. Yes, there is something to be learned by his results but it has a the very least 50% to do with DAW internals and CPU vs the CPUs themselves. He used large numbers of tracks to test which isn't a valid measure of CPU optimization across different DAW platforms. There is widely varying CPU overhead issues among the different DAWs. Running 300 tracks in Ableton is going to cause a bunch of different bottlenecks and inefficiencies to be pushed... not just CPU. Clickbait. I'm not saying that the M3Pro is better or worse or trying to defend Apple, I'm just pointing out as someone with a lot of experience in DAW-land that this testing methology is bunk as a CPU test and as a DAW performance test.

david_allison 3y ago
It's a music site. Showing that the efficiency cores aren't fully utilized by the DAWs is appropriate for the audience. It shows to less specialized users that they'll need to examine their tools and their use of efficiency cores to determine if an upgrade is worthwhile
- wtallis 3y ago
  I'm not sure if it's quite accurate to say that the efficiency cores aren't fully utilized; it looks more like some of these apps are second guessing the OS and going out of their way to avoid using the efficiency cores (probably by only spawning one thread per P core, since macOS doesn't let apps directly control thread affinity). Most of the time, that is probably the wrong approach for an app to take, but maybe DAWs have latency requirements that can't be met when E cores are in the mix. But that's something the DAW vendors should have to document and justify, because it's just as likely that they're making assumptions about E cores that don't hold up over time across the full range of chips.
  - david_allison 3y ago
    > I'm not sure if it's quite accurate to say that the efficiency cores aren't fully utilized The graphs in the source video[0] display 0% utilization on E-cores for some apps. Given that it only occurs for some DAWs, it's very likely the wrong approach (given that 2/3 of the apps using E-cores have better performance characteristics than the others) https://youtu.be/FSqX4bt9to4?t=125 https://youtu.be/FSqX4bt9to4?t=125
- deleted 3y ago
  [deleted]
- n9 3y ago
  testing how a DAW handles 300 identical low-CPU consumptive tracks is not at all appropriate for a music CPU workload assessment. No one except Charlie Clauser or Hollywood cinema post production uses projects with 300+ tracks -- it is a really idiosyncratic test that exposes the optimization liabilities of the DAW vs the CPU. A Better test would be to set up an intensive real world 20 track project, assess utilization and overhead, and then find a real-world way to increment that stress upward to the breaking point. Most computer music forum wonks use some kind of test based on softsynths or reverbs or a combination and stack them high on a smaller number of tracks. I mean... 300 tracks? You're basically testing the DAW's thread management at that point, not the CPU as it would be used.
  - deleted 3y ago
    [deleted]
  - squeaky-clean 3y ago
    300 tracks is more realistic than 20, at least for electronic music. Here is a Skrillex single with 180 tracks. https://youtu.be/2dYFJdQf7rs?si=rsBc8gCtz5DQYGI9 https://youtu.be/2dYFJdQf7rs?si=rsBc8gCtz5DQYGI9
  - wtallis 3y ago
    > I mean... 300 tracks? You're basically testing the DAW's thread management at that point, not the CPU as it would be used. Any realistic test is going to have more tracks than there are CPU cores, so thread management would seem to be an unavoidable aspect of the test. The test results were in the 60–100 track range, not 300, so not really as excessive as you are claiming. And while testing with too many tracks could certainly be problematic for introducing excessive context switching overhead, the problematic behavior that was actually revealed is much more serious and less subtle and applies to any scenario with more than 5–8 threads.
squeaky-clean 3y ago
I'm an Ableton user so it's important to me to know how these laptops actually perform when using Ableton and not in some theoretical benchmark.

by n9 3y ago

david_allison 3y ago

It's a music site. Showing that the efficiency cores aren't fully utilized by the DAWs is appropriate for the audience. It shows to less specialized users that they'll need to examine their tools and their use of efficiency cores to determine if an upgrade is worthwhile

wtallis 3y ago
I'm not sure if it's quite accurate to say that the efficiency cores aren't fully utilized; it looks more like some of these apps are second guessing the OS and going out of their way to avoid using the efficiency cores (probably by only spawning one thread per P core, since macOS doesn't let apps directly control thread affinity). Most of the time, that is probably the wrong approach for an app to take, but maybe DAWs have latency requirements that can't be met when E cores are in the mix. But that's something the DAW vendors should have to document and justify, because it's just as likely that they're making assumptions about E cores that don't hold up over time across the full range of chips.
- david_allison 3y ago
  > I'm not sure if it's quite accurate to say that the efficiency cores aren't fully utilized The graphs in the source video[0] display 0% utilization on E-cores for some apps. Given that it only occurs for some DAWs, it's very likely the wrong approach (given that 2/3 of the apps using E-cores have better performance characteristics than the others) https://youtu.be/FSqX4bt9to4?t=125 https://youtu.be/FSqX4bt9to4?t=125
deleted 3y ago
[deleted]
n9 3y ago
testing how a DAW handles 300 identical low-CPU consumptive tracks is not at all appropriate for a music CPU workload assessment. No one except Charlie Clauser or Hollywood cinema post production uses projects with 300+ tracks -- it is a really idiosyncratic test that exposes the optimization liabilities of the DAW vs the CPU. A Better test would be to set up an intensive real world 20 track project, assess utilization and overhead, and then find a real-world way to increment that stress upward to the breaking point. Most computer music forum wonks use some kind of test based on softsynths or reverbs or a combination and stack them high on a smaller number of tracks. I mean... 300 tracks? You're basically testing the DAW's thread management at that point, not the CPU as it would be used.
- deleted 3y ago
  [deleted]
- squeaky-clean 3y ago
  300 tracks is more realistic than 20, at least for electronic music. Here is a Skrillex single with 180 tracks. https://youtu.be/2dYFJdQf7rs?si=rsBc8gCtz5DQYGI9 https://youtu.be/2dYFJdQf7rs?si=rsBc8gCtz5DQYGI9
- wtallis 3y ago
  > I mean... 300 tracks? You're basically testing the DAW's thread management at that point, not the CPU as it would be used. Any realistic test is going to have more tracks than there are CPU cores, so thread management would seem to be an unavoidable aspect of the test. The test results were in the 60–100 track range, not 300, so not really as excessive as you are claiming. And while testing with too many tracks could certainly be problematic for introducing excessive context switching overhead, the problematic behavior that was actually revealed is much more serious and less subtle and applies to any scenario with more than 5–8 threads.

squeaky-clean 3y ago

I'm an Ableton user so it's important to me to know how these laptops actually perform when using Ableton and not in some theoretical benchmark.