9 ms·

Apple M3 Pro Chip Has 25% Less Memory Bandwidth Than M1/M2 Pro

104 points by TaurenHunter 3y ago

gavinhoward 3y ago
Oh good. I was planning to get a Mac for testing purposes, but I wanted something weak so that I could make sure my software is fast on slow devices! /s
deeth_starr_v 3y ago
A sly comment I saw elsewhere was this gives them a nice bump they can use when the M4 is released and they go back to 4x4 Edit: my personal feeling is this is just a cost savings they are doing like they did with the base SSDs
- alwillis 3y ago
  From someone who was briefed by Apple [1], this is Apple segmenting these processors. The M1 Pro and the M2 Pro were closer to their respective Max variants. With the M3 series, the M3 Pro is more of a mainstream processor vs. being closer to the M3 Max. [1]: https://podcasts.apple.com/us/podcast/upgrade/id918152703?i=1000633269988 https://podcasts.apple.com/us/podcast/upgrade/id918152703?i=...
  - alberth 3y ago
    What’s strange about this strategy is it’s less cost effective because Apple is now designing a completely different Pro & Max chiplet, as opposed to just scaling up the base chiplet as done in the M1 & M2. This is a lot of extra complexity they are inflicting on themselves if accurate.
    - chii 3y ago
      > extra complexity if this extra complexity is mitigated by hiring a larger, separate team (rather than strain existing engineers), then they might win overall because of the customer segmentation.
    - 3c6bYDXLMj 3y ago
      I trust that Apple knows significantly more about the efficiency of their chip design process.
      - deleted 3y ago
        [deleted]
    - Aaargh20318 3y ago
      > What’s strange about this strategy is it’s less cost effective because Apple is now designing a completely different Pro & Max chiplet That may be true for the M3, but they undoubtedly are already working on the M4 and M5; they may have bigger differences planned for future releases that require the Pro and Max designs to be separate. Could be that they are just doing the initial work to split the design into two branches for M3 so they can make more changes from M4 onwards.
      - cyanydeez 3y ago
        [flagged]
- alberth 3y ago
  Probably not because they will compare the M4 to the M2 (because M3 owners are less likely to upgrade).
  - KeplerBoy 3y ago
    Oh come on. They will compare to whatever makes their product look good.
    - reroute22 3y ago
      They will compare to the generation of products that yields the largest expected number of potential buyers. The largest expected number of potential buyers equals "the probability of upgrading when properly advertised to" * (multiplied by) "the total number of macOS users currently owning the hardware of the generation in question". Where "the probability of upgrading when properly advertised to" depends on how big the jump is, not only in performance, but cumulatively with every other change made to the product. It also depends on how old it is as people are more likely to make another expensive purchase if the previous one was long time ago, just distributing financial burden over time. And where "the total number of macOS users currently owning the hardware of the generation in question" is merely measured as Apple has direct access to those statistics at any time. For instance, perhaps the probability that an M2 user will upgrade is only 10%, given how recently M2 released, and how small the difference between M2 based products (as a whole, not just performance of the SoC) and M3 based products is. And perhaps there is 10 million of M2 users. Expected number of potential buyers is 10M * 0.1 = 1M. Then perhaps the probability that an M1 user will upgrade is 30%. And perhaps there is 20 million of M1 users. Expected number of potential buyers is 20M * 0.3 = 6M. Then perhaps the probability that an Intel user will upgrade is 40%. And perhaps there is only 10M of Intel users. Expected number of potential buyers is 10M * 0.4 = 4M. Therefore: most focus is given to comparing to M1. Second tier amount of focus (only mention in speech occasionally, and show on charts) is given to comparisons with Intel. And third tier amount of focus is given to comparisons with M2 (never mention in speech, but do show on charts). Give 0 focus to anything else to keep the total amount of information presented manageable and avoid getting negative results by confusing and overloading people. There is nothing malicious here. Just practicality which in this case, dare I say, benefits not only Apple themselves, but their users as well: those who are most likely to feel the need to upgrade as is will receive the most direct and clear information on what they will get out of it.
- bjoli 3y ago
  I was downvoted to oblivion for suggesting that was the reason the last couple of years of intel MacBooks had botched thermal solutions. They were really indefensibly bad, but then again: the jump to m1 was enormous. I have a hard time feeling I was the only one conspiratorial enough to suggest something like that.
  - znpy 3y ago
    Knowing how much apple exploits its users a d knowing how apple fanatics are willing to buy whatever and in retrospect… it makes perfect sense. I believe you.
    - 3c6bYDXLMj 3y ago
      Comments from someone that speaks as if they always expect the worst of Apple, telling everyone that they expect the worst of Apple, are a dime a dozen and do nothing to add to the conversation.
    - qeternity 3y ago
      Solid take, couldn't possibly be that you simply don't understand why people buy Apple. Everyone else is wrong.
  - alpaca128 3y ago
    Three reasons that make me disagree: Firstly M1/M2 Macbooks don't have great cooling either. The Air models use basically nothing but a thin sheet of metal, maybe they'd run longer without throttling otherwise. Secondly Intel Macbooks had subpar cooling (and power delivery) much longer than the last couple of years. A video on YouTube shows more than doubled performance on a watercooled 2015 Macbook. And let's not forget that other manufacturers are sometimes even worse than Apple: Microsoft once sold a Surface Pro where the upgraded i7 version was slower than the model with i5 CPU because the cooling was so terrible! No, I don't see how that's more than a conspiracy theory. I am using an M2 Air right now and it has 2-3x better performance than my older i7 Thinkpad while being completely silent and running 3x longer on battery. Apple didn't have the need to pull any tricks.
fomine3 3y ago
It's not good change for AI workload that requires memory capacity and bandwidth (rather than cache). Generative AI boom (on local machine) was started just about a year ago, so maybe it was too late to take effect for chip design.
- zamadatix 3y ago
  If you're serious about doing generative ai locally the Max makes more sense anyways. Significantly higher capacities, more total compute. For typical generative use cases the Pro is still decently faster memory than a traditional system but decently slower memory than a good GPU I.e. not really much different than before.
tedunangst 3y ago
Presumably the people who require this extra bandwidth will just pick up older models off ebay?
jpcfl 3y ago
Raw statistics like this without additional context are practically meaningless.
- reroute22 3y ago
  The answer to these sort of questions (OP) is always the same: it was done because it made all the sense in the world to do. Most likely, 150 GB/s -> 200 GB/s results in a fairly small improvement (when paired with a processor of M2 Pro / M3 Pro overall capability) only in a fairly small and specific subset of GPU applications. In particular, it's pretty much the matter of fact that that extra bandwidth achieves nothing in CPU-bound applications. It's also a matter of fact that only some of the GPU applications benefit, I just can not attest to exactly how big (or rather, small) and important that subset is. With every new process node nowadays the following happens: 1. The cost per unit of area increases substantially. Decades ago the cost per unit of area was practically staying the same, resulting in 2x smaller node being 2x cheaper for the same design as the same design would take up 2x less area. Not anymore. The cost per unit of area is higher, therefore, if a portion of the design doesn't shrink much, it's actually more expensive on newer node. It takes a large shrink for the design to get cheaper or even merely stay the same in per-unit manufacturing cost. 2. IO shrinks very little. 4nm -> 3nm resulted in only 10% IO shrink. 1.25x SRAM, and 1.7x logic. 3. DRAM bandwidth is just a product of bus width * DRAM frequency, where bus width is really just a number-of-DRAM-controllers * 32. 4. DRAM controllers is IO. It barely shrinks in area going 4nm -> 3nm. But 3nm is more expensive per unit of area to manufacture. Therefore, DRAM controllers of the same design and the same count and the same bandwidth cost more money now on 3nm. Most likely, that very marginal and situational performance benefit in a subset of GPU applications that M2 Pro saw going from 150 GB/s to 200 GB/s was still large enough to justify the relatively low (on 5nm) cost of 8 DRAM controllers (in traditional 32-bit-bus-per-controller terms). On M3 Pro that performance gain probably just dropped below the threshold and became unjustifiable against the increased cost of DRAM controllers, and the number of DRAM controllers was reduced to 6.
PeterStuer 3y ago
Its all about market segmentation. Most 'Pro' buyer application mixes will not feel any significant effect from this restriction. However, those that do will now have to go for the 'Max' and can afford the premium.
exabrial 3y ago
> the M3 Pro system on a chip (SoC) features 150GB/s memory bandwidth, compared to 200GB/s on the earlier M1 Pro and M2 Pro. As for the M3 Max, Apple says it is capable of "up to 400GB/s." Yeah, that's a downgrade.
- _5ygc 3y ago
  Genuine question, from someone that won’t upgrade from their M1 Air: what difference does it make for most users? Where would that extra 50GB/s be felt?
  - exabrial 3y ago
    The M1 doesn't really benchmark in pure CPUs tests much higher that the Intel chips it replaced... it's was an incremental upgrade, not a generational upgrade; in other word, pure CPU benchmark increases would have been realized when/if the nextgen Intel chips were used. Instead, most of the felt responsiveness on the M1 comes from the insane memory bandwidth. Everything from launching apps to task swapping to garbage collection events in various languages gets a boost from the lower latency and higher bandwidth.
    - xctr94 3y ago
      Okay, I really don’t know why the M1 feels so amazingly responsive compared to my i7. But is 12.5% loss in memory bandwidth that important for these kinds of tasks? It seems really hard to saturate, say, 100GB per second!
    - reroute22 3y ago
      This is all blatantly false and the exact opposite is true. > The M1 doesn't really benchmark in pure CPUs tests much higher that the Intel chips it replaced It does. See Cinebench 2024 ST at https://www.cpu-monkey.com/en/compare_cpu-apple_m1_pro_10_cpu_16_gpu-vs-intel_core_i9_9880h https://www.cpu-monkey.com/en/compare_cpu-apple_m1_pro_10_cp... : M1 Pro is 1.6x faster than Intel Core i9-9880H installed in the latest (2019) 16" Intel MBP (see the table at https://en.wikipedia.org/wiki/MacBook_Pro_(Intel-based)#Technical_specifications_4 https://en.wikipedia.org/wiki/MacBook_Pro_(Intel-based)#Tech...). > Instead, most of the felt responsiveness on the M1 comes from the insane memory bandwidth. False. Extra bandwidth is there for and meaningfully improves the performance of the GPU, not the CPU. CPU can not even access all of that bandwidth in Apple M-series designs in the first place: "While 243GB/s is massive, and overshadows any other design in the industry, it’s still quite far from the 409GB/s the chip is capable of. More importantly for the M1 Max, it’s only slightly higher than the 204GB/s limit of the M1 Pro, so from a CPU-only workload perspective, it doesn’t appear to make sense to get the Max if one is focused just on CPU bandwidth." (source: https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc...) Furthermore, even within the bandwidth actually available, it's just a matter of fact that extra bandwidth barely if ever really makes a dent in CPU limited (as opposed to GPU limited) applications. See https://youtu.be/omumzW1AtGE?t=500 https://youtu.be/omumzW1AtGE?t=500 as an example of one of the studies on the overall subject. > a boost from the lower latency False. M-series systems have slightly _higher_ system memory to/from CPU latency, and drastically higher system memory to/from GPU latency (especially compared to other iGPU solutions). See https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... for the CPU side: 111 ns for M1 Max CPU to DRAM. See https://www.anandtech.com/show/16680/tiger-lake-h-performance-review/3 https://www.anandtech.com/show/16680/tiger-lake-h-performanc... for the same data for Intel i9-11980HK as a ballpark comparison point: 101 ns. Apple's results are basically inline with everything else in the industry, and are in fact slightly worse on latency. For GPU side, see https://chipsandcheese.com/2023/10/31/a-brief-look-at-apples-m2-pro-igpu/ https://chipsandcheese.com/2023/10/31/a-brief-look-at-apples... : "DRAM access takes more than 342 ns at the 128 MB test size. Going further sends latency beyond 400 ns, perhaps due to TLB misses. M2 Pro thus has higher DRAM access latency than AMD’s Phoenix, which has similar memory access latency to recent discrete GPUs." You will see: DRAM latency from M2 Pro's iGPU is up to 475ns, while iGPU on AMD 7840HS peaks (on the bad side) at 270ns. M2 (non-Pro) does better than M2 Pro peaking at 330ns or so, still roughly 20% worse than AMD. I'm sorry, but the exact negation of everything you said is in fact true.

104 points by TaurenHunter 3y ago

Oh good. I was planning to get a Mac for testing purposes, but I wanted something weak so that I could make sure my software is fast on slow devices! /s

deeth_starr_v 3y ago

A sly comment I saw elsewhere was this gives them a nice bump they can use when the M4 is released and they go back to 4x4 Edit: my personal feeling is this is just a cost savings they are doing like they did with the base SSDs

alwillis 3y ago
From someone who was briefed by Apple [1], this is Apple segmenting these processors. The M1 Pro and the M2 Pro were closer to their respective Max variants. With the M3 series, the M3 Pro is more of a mainstream processor vs. being closer to the M3 Max. [1]: https://podcasts.apple.com/us/podcast/upgrade/id918152703?i=1000633269988 https://podcasts.apple.com/us/podcast/upgrade/id918152703?i=...
- alberth 3y ago
  What’s strange about this strategy is it’s less cost effective because Apple is now designing a completely different Pro & Max chiplet, as opposed to just scaling up the base chiplet as done in the M1 & M2. This is a lot of extra complexity they are inflicting on themselves if accurate.
  - chii 3y ago
    > extra complexity if this extra complexity is mitigated by hiring a larger, separate team (rather than strain existing engineers), then they might win overall because of the customer segmentation.
  - 3c6bYDXLMj 3y ago
    I trust that Apple knows significantly more about the efficiency of their chip design process.
    - deleted 3y ago
      [deleted]
  - Aaargh20318 3y ago
    > What’s strange about this strategy is it’s less cost effective because Apple is now designing a completely different Pro & Max chiplet That may be true for the M3, but they undoubtedly are already working on the M4 and M5; they may have bigger differences planned for future releases that require the Pro and Max designs to be separate. Could be that they are just doing the initial work to split the design into two branches for M3 so they can make more changes from M4 onwards.
    - cyanydeez 3y ago
      [flagged]
alberth 3y ago
Probably not because they will compare the M4 to the M2 (because M3 owners are less likely to upgrade).
- KeplerBoy 3y ago
  Oh come on. They will compare to whatever makes their product look good.
  - reroute22 3y ago
    They will compare to the generation of products that yields the largest expected number of potential buyers. The largest expected number of potential buyers equals "the probability of upgrading when properly advertised to" * (multiplied by) "the total number of macOS users currently owning the hardware of the generation in question". Where "the probability of upgrading when properly advertised to" depends on how big the jump is, not only in performance, but cumulatively with every other change made to the product. It also depends on how old it is as people are more likely to make another expensive purchase if the previous one was long time ago, just distributing financial burden over time. And where "the total number of macOS users currently owning the hardware of the generation in question" is merely measured as Apple has direct access to those statistics at any time. For instance, perhaps the probability that an M2 user will upgrade is only 10%, given how recently M2 released, and how small the difference between M2 based products (as a whole, not just performance of the SoC) and M3 based products is. And perhaps there is 10 million of M2 users. Expected number of potential buyers is 10M * 0.1 = 1M. Then perhaps the probability that an M1 user will upgrade is 30%. And perhaps there is 20 million of M1 users. Expected number of potential buyers is 20M * 0.3 = 6M. Then perhaps the probability that an Intel user will upgrade is 40%. And perhaps there is only 10M of Intel users. Expected number of potential buyers is 10M * 0.4 = 4M. Therefore: most focus is given to comparing to M1. Second tier amount of focus (only mention in speech occasionally, and show on charts) is given to comparisons with Intel. And third tier amount of focus is given to comparisons with M2 (never mention in speech, but do show on charts). Give 0 focus to anything else to keep the total amount of information presented manageable and avoid getting negative results by confusing and overloading people. There is nothing malicious here. Just practicality which in this case, dare I say, benefits not only Apple themselves, but their users as well: those who are most likely to feel the need to upgrade as is will receive the most direct and clear information on what they will get out of it.
bjoli 3y ago
I was downvoted to oblivion for suggesting that was the reason the last couple of years of intel MacBooks had botched thermal solutions. They were really indefensibly bad, but then again: the jump to m1 was enormous. I have a hard time feeling I was the only one conspiratorial enough to suggest something like that.
- znpy 3y ago
  Knowing how much apple exploits its users a d knowing how apple fanatics are willing to buy whatever and in retrospect… it makes perfect sense. I believe you.
  - 3c6bYDXLMj 3y ago
    Comments from someone that speaks as if they always expect the worst of Apple, telling everyone that they expect the worst of Apple, are a dime a dozen and do nothing to add to the conversation.
  - qeternity 3y ago
    Solid take, couldn't possibly be that you simply don't understand why people buy Apple. Everyone else is wrong.
- alpaca128 3y ago
  Three reasons that make me disagree: Firstly M1/M2 Macbooks don't have great cooling either. The Air models use basically nothing but a thin sheet of metal, maybe they'd run longer without throttling otherwise. Secondly Intel Macbooks had subpar cooling (and power delivery) much longer than the last couple of years. A video on YouTube shows more than doubled performance on a watercooled 2015 Macbook. And let's not forget that other manufacturers are sometimes even worse than Apple: Microsoft once sold a Surface Pro where the upgraded i7 version was slower than the model with i5 CPU because the cooling was so terrible! No, I don't see how that's more than a conspiracy theory. I am using an M2 Air right now and it has 2-3x better performance than my older i7 Thinkpad while being completely silent and running 3x longer on battery. Apple didn't have the need to pull any tricks.

fomine3 3y ago

It's not good change for AI workload that requires memory capacity and bandwidth (rather than cache). Generative AI boom (on local machine) was started just about a year ago, so maybe it was too late to take effect for chip design.

zamadatix 3y ago
If you're serious about doing generative ai locally the Max makes more sense anyways. Significantly higher capacities, more total compute. For typical generative use cases the Pro is still decently faster memory than a traditional system but decently slower memory than a good GPU I.e. not really much different than before.

tedunangst 3y ago

Presumably the people who require this extra bandwidth will just pick up older models off ebay?

jpcfl 3y ago

Raw statistics like this without additional context are practically meaningless.

reroute22 3y ago
The answer to these sort of questions (OP) is always the same: it was done because it made all the sense in the world to do. Most likely, 150 GB/s -> 200 GB/s results in a fairly small improvement (when paired with a processor of M2 Pro / M3 Pro overall capability) only in a fairly small and specific subset of GPU applications. In particular, it's pretty much the matter of fact that that extra bandwidth achieves nothing in CPU-bound applications. It's also a matter of fact that only some of the GPU applications benefit, I just can not attest to exactly how big (or rather, small) and important that subset is. With every new process node nowadays the following happens: 1. The cost per unit of area increases substantially. Decades ago the cost per unit of area was practically staying the same, resulting in 2x smaller node being 2x cheaper for the same design as the same design would take up 2x less area. Not anymore. The cost per unit of area is higher, therefore, if a portion of the design doesn't shrink much, it's actually more expensive on newer node. It takes a large shrink for the design to get cheaper or even merely stay the same in per-unit manufacturing cost. 2. IO shrinks very little. 4nm -> 3nm resulted in only 10% IO shrink. 1.25x SRAM, and 1.7x logic. 3. DRAM bandwidth is just a product of bus width * DRAM frequency, where bus width is really just a number-of-DRAM-controllers * 32. 4. DRAM controllers is IO. It barely shrinks in area going 4nm -> 3nm. But 3nm is more expensive per unit of area to manufacture. Therefore, DRAM controllers of the same design and the same count and the same bandwidth cost more money now on 3nm. Most likely, that very marginal and situational performance benefit in a subset of GPU applications that M2 Pro saw going from 150 GB/s to 200 GB/s was still large enough to justify the relatively low (on 5nm) cost of 8 DRAM controllers (in traditional 32-bit-bus-per-controller terms). On M3 Pro that performance gain probably just dropped below the threshold and became unjustifiable against the increased cost of DRAM controllers, and the number of DRAM controllers was reduced to 6.

PeterStuer 3y ago

Its all about market segmentation. Most 'Pro' buyer application mixes will not feel any significant effect from this restriction. However, those that do will now have to go for the 'Max' and can afford the premium.

exabrial 3y ago

> the M3 Pro system on a chip (SoC) features 150GB/s memory bandwidth, compared to 200GB/s on the earlier M1 Pro and M2 Pro. As for the M3 Max, Apple says it is capable of "up to 400GB/s." Yeah, that's a downgrade.

_5ygc 3y ago
Genuine question, from someone that won’t upgrade from their M1 Air: what difference does it make for most users? Where would that extra 50GB/s be felt?
- exabrial 3y ago
  The M1 doesn't really benchmark in pure CPUs tests much higher that the Intel chips it replaced... it's was an incremental upgrade, not a generational upgrade; in other word, pure CPU benchmark increases would have been realized when/if the nextgen Intel chips were used. Instead, most of the felt responsiveness on the M1 comes from the insane memory bandwidth. Everything from launching apps to task swapping to garbage collection events in various languages gets a boost from the lower latency and higher bandwidth.
  - xctr94 3y ago
    Okay, I really don’t know why the M1 feels so amazingly responsive compared to my i7. But is 12.5% loss in memory bandwidth that important for these kinds of tasks? It seems really hard to saturate, say, 100GB per second!
  - reroute22 3y ago
    This is all blatantly false and the exact opposite is true. > The M1 doesn't really benchmark in pure CPUs tests much higher that the Intel chips it replaced It does. See Cinebench 2024 ST at https://www.cpu-monkey.com/en/compare_cpu-apple_m1_pro_10_cpu_16_gpu-vs-intel_core_i9_9880h https://www.cpu-monkey.com/en/compare_cpu-apple_m1_pro_10_cp... : M1 Pro is 1.6x faster than Intel Core i9-9880H installed in the latest (2019) 16" Intel MBP (see the table at https://en.wikipedia.org/wiki/MacBook_Pro_(Intel-based)#Technical_specifications_4 https://en.wikipedia.org/wiki/MacBook_Pro_(Intel-based)#Tech...). > Instead, most of the felt responsiveness on the M1 comes from the insane memory bandwidth. False. Extra bandwidth is there for and meaningfully improves the performance of the GPU, not the CPU. CPU can not even access all of that bandwidth in Apple M-series designs in the first place: "While 243GB/s is massive, and overshadows any other design in the industry, it’s still quite far from the 409GB/s the chip is capable of. More importantly for the M1 Max, it’s only slightly higher than the 204GB/s limit of the M1 Pro, so from a CPU-only workload perspective, it doesn’t appear to make sense to get the Max if one is focused just on CPU bandwidth." (source: https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc...) Furthermore, even within the bandwidth actually available, it's just a matter of fact that extra bandwidth barely if ever really makes a dent in CPU limited (as opposed to GPU limited) applications. See https://youtu.be/omumzW1AtGE?t=500 https://youtu.be/omumzW1AtGE?t=500 as an example of one of the studies on the overall subject. > a boost from the lower latency False. M-series systems have slightly _higher_ system memory to/from CPU latency, and drastically higher system memory to/from GPU latency (especially compared to other iGPU solutions). See https://www.anandtech.com/show/17024/apple-m1-max-performance-review/2 https://www.anandtech.com/show/17024/apple-m1-max-performanc... for the CPU side: 111 ns for M1 Max CPU to DRAM. See https://www.anandtech.com/show/16680/tiger-lake-h-performance-review/3 https://www.anandtech.com/show/16680/tiger-lake-h-performanc... for the same data for Intel i9-11980HK as a ballpark comparison point: 101 ns. Apple's results are basically inline with everything else in the industry, and are in fact slightly worse on latency. For GPU side, see https://chipsandcheese.com/2023/10/31/a-brief-look-at-apples-m2-pro-igpu/ https://chipsandcheese.com/2023/10/31/a-brief-look-at-apples... : "DRAM access takes more than 342 ns at the 128 MB test size. Going further sends latency beyond 400 ns, perhaps due to TLB misses. M2 Pro thus has higher DRAM access latency than AMD’s Phoenix, which has similar memory access latency to recent discrete GPUs." You will see: DRAM latency from M2 Pro's iGPU is up to 475ns, while iGPU on AMD 7840HS peaks (on the bad side) at 270ns. M2 (non-Pro) does better than M2 Pro peaking at 330ns or so, still roughly 20% worse than AMD. I'm sorry, but the exact negation of everything you said is in fact true.