4 ms·

True, but then again you get 400GB/sec of bandwidth and low power use that allows for 10-20 hours of usage in a tiny thin laptop. Sadly 7950x doesn't come clos

by sliken 3y ago

True, but then again you get 400GB/sec of bandwidth and low power use that allows for 10-20 hours of usage in a tiny thin laptop. Sadly 7950x doesn't come close to fitting in laptops, runs hot (even in the non-x flavor) and has 1/4th the memory bandwidth. I'd happily trade dimm slots for a fraction of the power usage and 4x the memory bandwidth (or 8x if you get the studio).

jeswin 3y ago
> True, but then again you get 400GB/sec of bandwidth For what, other than toy AI inference? More HEDT users would go with more RAM at DDR5 speeds than 400GB/sec of bandwidth. As you can see from Geekbench, the additional bandwidth has no real world implications for non-GPU use cases. But again, just as with everything else it depends.
- threeseed 3y ago
  > For what, other than toy AI inference? Processing multiple video or audio streams. Key use case for Macs.
  - deleted 3y ago
    [deleted]
- sliken 3y ago
  Simulations (games or HPC), emulation, and editing multiple streams of 8k videos. Why do you say "toy AI inference"? Getting 800GB/sec to 128GB of memory (on a desktop) or 400GB/sec to 128GB/sec on a laptop is hard to beat. You have to spend crazy money to get a GPU with that much ram and normal desktops like the mentioned 7950x are going to be 4-8 x slower. I've seen people get over 5 tokens/sec inference with 180B models, not what I'd call "toy AI inference".
  - seec 3y ago
    Most intel processors with dual channel will easily get 90GB/sec of bandwidth just for the CPU; if you build a performance-oriented machine you are supposed to put a dedicated GPU with that. The Apple silicon bandwidth is nothing special considering they must share it with a GPU. Dedicated GPU have as much or more bandwidth than Apple Silicon; and better than that, at the low end they are highly likely to have more RAM available. When you compare prices, even on laptops, it is likely that the competing windows PC will have both more bandwidth and more RAM available. Because 8GB of RAM in a dedicated GPU is not very rare at the price MacBook Pros are sold at, and those systems have loads of CPU RAM on top of it. Even with the update, a base M3 Pro only has 18GB of ram to share between CPU and GPU. If you let the GPU use 8GB like it would be able to in the windows laptop, suddenly you only have 10GB of system RAM. Which is going to be very limiting for many things. In the end, every other OEM just gave up on bandwidth because it does not matter for most things. Even Apple somewhat acknowledged that by lowering the bandwidth available for most SKUs; so, there is that. And stop quoting the higher-end bandwidth that is only available in extremely expensive SKUs : - 800GB/s is only available in the Mac Studio Ultra version and the cost equivalent would be a PC with two Nvidia 4090 that would completely crush it both bandwidth and speed wise. - 400GB/s is only available in at least 2K Mac Studio desktop or minimum 3K laptop. And it is nothing special compared to the available bandwidth dedicated GPUs have at this price. The bandwidth on the lower SKUs is whatever because it is barely better than what Intel has always provided in their CPUs with integrated graphics. It is a bit better but nothing special and there exist competing products with just as much bandwidth, not that this fact is particularly relevant for the type of tasks this low/mid-range hardware is supposed to carry out. Apple fanboys are so delusional its bordering insanity.
Sparkyte 3y ago
That's where the 7945X3D comes in. :)
- sliken 3y ago
  Does cache help some things sure, but not a replacement for bandwidth. I have seen cases where the zen 3 X3D wins against the zen 4 without X3D, especially on simulations, emulators, flight sims, etc. Sadly there seems to be movements all over the industry to depend on caching over bandwidth and generally they win benchmarks, but often lose on real world use. Intel N100 and similar embedded type chips for appliances/routers went from 128 bit wide to 64. The M2 Pro has 256 bit wide memory and the M3 pro went to 192 bit wide. The Nvidia 3060 Ti 256 bit wide memory has 4060 Ti went down to 128 bit wide. Sure the average performance is often higher (when cache friendly), but the performance is much more variable as the ratio between in cache performance and out of cache performance gets larger. GPUs with more cache and less bandwidth tend to get pickier about which games they run well on and the 1% lows get slower, which makers stuttering worse. It's sad that from that for normal desktops from a few $100 to a few $1000 all have 128 bit wide memory interfaces, unless you buy a mac and you can get 128, 256, 512, or 1024 bit wide.
  - Sparkyte 3y ago
    This is why I'm excited for the 8K chips we will have some actually comparable chips with wider bandwidths to put side by side to M series processors.

by sliken 3y ago

jeswin 3y ago

> True, but then again you get 400GB/sec of bandwidth For what, other than toy AI inference? More HEDT users would go with more RAM at DDR5 speeds than 400GB/sec of bandwidth. As you can see from Geekbench, the additional bandwidth has no real world implications for non-GPU use cases. But again, just as with everything else it depends.

threeseed 3y ago
> For what, other than toy AI inference? Processing multiple video or audio streams. Key use case for Macs.
- deleted 3y ago
  [deleted]
sliken 3y ago
Simulations (games or HPC), emulation, and editing multiple streams of 8k videos. Why do you say "toy AI inference"? Getting 800GB/sec to 128GB of memory (on a desktop) or 400GB/sec to 128GB/sec on a laptop is hard to beat. You have to spend crazy money to get a GPU with that much ram and normal desktops like the mentioned 7950x are going to be 4-8 x slower. I've seen people get over 5 tokens/sec inference with 180B models, not what I'd call "toy AI inference".
- seec 3y ago
  Most intel processors with dual channel will easily get 90GB/sec of bandwidth just for the CPU; if you build a performance-oriented machine you are supposed to put a dedicated GPU with that. The Apple silicon bandwidth is nothing special considering they must share it with a GPU. Dedicated GPU have as much or more bandwidth than Apple Silicon; and better than that, at the low end they are highly likely to have more RAM available. When you compare prices, even on laptops, it is likely that the competing windows PC will have both more bandwidth and more RAM available. Because 8GB of RAM in a dedicated GPU is not very rare at the price MacBook Pros are sold at, and those systems have loads of CPU RAM on top of it. Even with the update, a base M3 Pro only has 18GB of ram to share between CPU and GPU. If you let the GPU use 8GB like it would be able to in the windows laptop, suddenly you only have 10GB of system RAM. Which is going to be very limiting for many things. In the end, every other OEM just gave up on bandwidth because it does not matter for most things. Even Apple somewhat acknowledged that by lowering the bandwidth available for most SKUs; so, there is that. And stop quoting the higher-end bandwidth that is only available in extremely expensive SKUs : - 800GB/s is only available in the Mac Studio Ultra version and the cost equivalent would be a PC with two Nvidia 4090 that would completely crush it both bandwidth and speed wise. - 400GB/s is only available in at least 2K Mac Studio desktop or minimum 3K laptop. And it is nothing special compared to the available bandwidth dedicated GPUs have at this price. The bandwidth on the lower SKUs is whatever because it is barely better than what Intel has always provided in their CPUs with integrated graphics. It is a bit better but nothing special and there exist competing products with just as much bandwidth, not that this fact is particularly relevant for the type of tasks this low/mid-range hardware is supposed to carry out. Apple fanboys are so delusional its bordering insanity.

Sparkyte 3y ago

That's where the 7945X3D comes in. :)

sliken 3y ago
Does cache help some things sure, but not a replacement for bandwidth. I have seen cases where the zen 3 X3D wins against the zen 4 without X3D, especially on simulations, emulators, flight sims, etc. Sadly there seems to be movements all over the industry to depend on caching over bandwidth and generally they win benchmarks, but often lose on real world use. Intel N100 and similar embedded type chips for appliances/routers went from 128 bit wide to 64. The M2 Pro has 256 bit wide memory and the M3 pro went to 192 bit wide. The Nvidia 3060 Ti 256 bit wide memory has 4060 Ti went down to 128 bit wide. Sure the average performance is often higher (when cache friendly), but the performance is much more variable as the ratio between in cache performance and out of cache performance gets larger. GPUs with more cache and less bandwidth tend to get pickier about which games they run well on and the 1% lows get slower, which makers stuttering worse. It's sad that from that for normal desktops from a few $100 to a few $1000 all have 128 bit wide memory interfaces, unless you buy a mac and you can get 128, 256, 512, or 1024 bit wide.
- Sparkyte 3y ago
  This is why I'm excited for the 8K chips we will have some actually comparable chips with wider bandwidths to put side by side to M series processors.