4 ms·

> True, but then again you get 400GB/sec of bandwidth For what, other than toy AI inference? More HEDT users would go with more RAM at DDR5 speeds than 400GB/

by jeswin 3y ago

> True, but then again you get 400GB/sec of bandwidth For what, other than toy AI inference? More HEDT users would go with more RAM at DDR5 speeds than 400GB/sec of bandwidth. As you can see from Geekbench, the additional bandwidth has no real world implications for non-GPU use cases. But again, just as with everything else it depends.

threeseed 3y ago
> For what, other than toy AI inference? Processing multiple video or audio streams. Key use case for Macs.
- deleted 3y ago
  [deleted]
sliken 3y ago
Simulations (games or HPC), emulation, and editing multiple streams of 8k videos. Why do you say "toy AI inference"? Getting 800GB/sec to 128GB of memory (on a desktop) or 400GB/sec to 128GB/sec on a laptop is hard to beat. You have to spend crazy money to get a GPU with that much ram and normal desktops like the mentioned 7950x are going to be 4-8 x slower. I've seen people get over 5 tokens/sec inference with 180B models, not what I'd call "toy AI inference".
- seec 3y ago
  Most intel processors with dual channel will easily get 90GB/sec of bandwidth just for the CPU; if you build a performance-oriented machine you are supposed to put a dedicated GPU with that. The Apple silicon bandwidth is nothing special considering they must share it with a GPU. Dedicated GPU have as much or more bandwidth than Apple Silicon; and better than that, at the low end they are highly likely to have more RAM available. When you compare prices, even on laptops, it is likely that the competing windows PC will have both more bandwidth and more RAM available. Because 8GB of RAM in a dedicated GPU is not very rare at the price MacBook Pros are sold at, and those systems have loads of CPU RAM on top of it. Even with the update, a base M3 Pro only has 18GB of ram to share between CPU and GPU. If you let the GPU use 8GB like it would be able to in the windows laptop, suddenly you only have 10GB of system RAM. Which is going to be very limiting for many things. In the end, every other OEM just gave up on bandwidth because it does not matter for most things. Even Apple somewhat acknowledged that by lowering the bandwidth available for most SKUs; so, there is that. And stop quoting the higher-end bandwidth that is only available in extremely expensive SKUs : - 800GB/s is only available in the Mac Studio Ultra version and the cost equivalent would be a PC with two Nvidia 4090 that would completely crush it both bandwidth and speed wise. - 400GB/s is only available in at least 2K Mac Studio desktop or minimum 3K laptop. And it is nothing special compared to the available bandwidth dedicated GPUs have at this price. The bandwidth on the lower SKUs is whatever because it is barely better than what Intel has always provided in their CPUs with integrated graphics. It is a bit better but nothing special and there exist competing products with just as much bandwidth, not that this fact is particularly relevant for the type of tasks this low/mid-range hardware is supposed to carry out. Apple fanboys are so delusional its bordering insanity.

by jeswin 3y ago

threeseed 3y ago

> For what, other than toy AI inference? Processing multiple video or audio streams. Key use case for Macs.

deleted 3y ago
[deleted]

sliken 3y ago

Simulations (games or HPC), emulation, and editing multiple streams of 8k videos. Why do you say "toy AI inference"? Getting 800GB/sec to 128GB of memory (on a desktop) or 400GB/sec to 128GB/sec on a laptop is hard to beat. You have to spend crazy money to get a GPU with that much ram and normal desktops like the mentioned 7950x are going to be 4-8 x slower. I've seen people get over 5 tokens/sec inference with 180B models, not what I'd call "toy AI inference".

seec 3y ago
Most intel processors with dual channel will easily get 90GB/sec of bandwidth just for the CPU; if you build a performance-oriented machine you are supposed to put a dedicated GPU with that. The Apple silicon bandwidth is nothing special considering they must share it with a GPU. Dedicated GPU have as much or more bandwidth than Apple Silicon; and better than that, at the low end they are highly likely to have more RAM available. When you compare prices, even on laptops, it is likely that the competing windows PC will have both more bandwidth and more RAM available. Because 8GB of RAM in a dedicated GPU is not very rare at the price MacBook Pros are sold at, and those systems have loads of CPU RAM on top of it. Even with the update, a base M3 Pro only has 18GB of ram to share between CPU and GPU. If you let the GPU use 8GB like it would be able to in the windows laptop, suddenly you only have 10GB of system RAM. Which is going to be very limiting for many things. In the end, every other OEM just gave up on bandwidth because it does not matter for most things. Even Apple somewhat acknowledged that by lowering the bandwidth available for most SKUs; so, there is that. And stop quoting the higher-end bandwidth that is only available in extremely expensive SKUs : - 800GB/s is only available in the Mac Studio Ultra version and the cost equivalent would be a PC with two Nvidia 4090 that would completely crush it both bandwidth and speed wise. - 400GB/s is only available in at least 2K Mac Studio desktop or minimum 3K laptop. And it is nothing special compared to the available bandwidth dedicated GPUs have at this price. The bandwidth on the lower SKUs is whatever because it is barely better than what Intel has always provided in their CPUs with integrated graphics. It is a bit better but nothing special and there exist competing products with just as much bandwidth, not that this fact is particularly relevant for the type of tasks this low/mid-range hardware is supposed to carry out. Apple fanboys are so delusional its bordering insanity.