5 ms·

Huge, GPU heavy SoCs are not economical without the huge volume, software ecosystem and margins of Apple. Smaller, higher clocked SoCs make sense for Android.

Huge, GPU heavy SoCs are not economical without the huge volume, software ecosystem and margins of Apple. Smaller, higher clocked SoCs make sense for Android. And seperate CPUs+dGPUs are what users want on PCs. Intel and AMD tried to sell GPU heavy designs (eDRAM Broadwell, Vega M, Van Gogh (the Steam Deck SoC)) and PC OEMs unequivocally rejected them. And they are trying to make better cores, but again they have to balance die area and target servers and cheap consumer PCs with the same cores.

huijzer 3y ago
I think the main benefit of joining two dies together via some advanced interconnect is the latency. Memory lookups are much quicker on Apple Silicon which, in turn, avoids many wasted clock ticks waiting for data.
- brucethemoose2 3y ago
  Are you talking about the M1 Extreme? This is extremely expensive, and I suspect the dual die GPU only works because of Metal. TBH I prefer AMD's 7900 series approach of pushing the memory controller + cache out onto little memory controller dies. This allows for a monolithic GPU with a really wide bus, without all that wasted space on the controller/cache/pins. And those dies don't care about being split up since the memory access is interleaved anyway.
  - 58028641 3y ago
    Isn’t it Pro, Max, Ultra?
    - brucethemoose2 3y ago
      thonk I can't remember, you are probably right. There is the little, medium, and big die, and the 2x big die.
      - sliken 3y ago
        The biggest single die is the m2 max. Dual die chip is the ultra.
  - olliej 3y ago
    "M1 Extreme? This is extremely expensive" hence the name :D That said I think it's pro and ultra, and I can't recall which (if not both) use the magic interconnect
flangola7 3y ago
>And seperate CPUs+dGPUs are what users want on PCs In the age of unified memory and large AI models, that may not be the case for very long.
- smoldesu 3y ago
  What are you talking about? How does a unified memory architecture change the bottlenecks of AI inferencing? The fastest AI servers in existence today only exist in heterogeneous form.
  - wmf 3y ago
    If you want, say, 70 GB of VRAM your options are either unified memory or pay $35K.
    - smoldesu 3y ago
      Or you can swap to DRAM and stream over PCI. I do that all the time and it works fine, especially since you rarely need to load all 70gb into memory at once. If you're a serious (read: enterprise) customer, you can buy InfiniBand-enabled cards and get duplex bandwidth faster than the M2 Max's entire memory bus. 'Unified memory' isn't even a bullet point on their spec sheet, it means nothing to their customers when they have CUDA primatives that do the same thing faster at a larger scale.
      - justinclift 3y ago
        > and get duplex bandwidth faster than the M2 Max's entire memory bus. Doesn't seem like it. Looking at the wikipedia article for the M2 Max, that says: ... with up to 400 GB/s memory bandwidth. https://en.wikipedia.org/wiki/Apple_silicon#Apple_M2_Max https://en.wikipedia.org/wiki/Apple_silicon#Apple_M2_Max Looking at the current "Nvidia networking" product page, it lists ConnectX-7 adapters with 400Gb/s total bandwidth: https://www.nvidia.com/en-us/networking/infiniband-adapters/ https://www.nvidia.com/en-us/networking/infiniband-adapters/ So, accounting for the bit -> byte unit difference in those figures it seems like having eight ConnectX-7 adapters would roughly match up. However, the data sheet for those adapters seems to say those 400Gb/s adapters are for PCIe Gen5 x32 slots (not the standard x16 ones). Host interface: PCIe Gen5, up to x32 lanes Eight slots of x32 lanes doesn't quite seem possible, even with very latest generation AMD EPYC (9004 series) processors: https://www.amd.com/system/files/documents/epyc-9004-series-processors-data-sheet.pdf https://www.amd.com/system/files/documents/epyc-9004-series-... Those have 128 PCIe Gen5 lanes per cpu. Dual socket systems seem to expand that out a bit, allowing up to 160x usable PCIe lanes in a server. https://www.servethehome.com/pcie-lanes-and-bandwidth-increase-over-a-decade-for-intel-xeon-and-amd-epyc/ https://www.servethehome.com/pcie-lanes-and-bandwidth-increa... So, at least on paper it doesn't seem possible for a single server with a bunch of PCIe Infiniband links to actually match the bandwidth of the M2 Max memory bus. Maybe 3/4 of it though, which isn't terrible. :)
Dalewyn 3y ago
>And seperate CPUs+dGPUs are what users want on PCs. Intel and AMD tried to sell GPU heavy designs (eDRAM Broadwell, Vega M, Van Gogh (the Steam Deck SoC)) and PC OEMs unequivocally rejected them. That's because the perceived or real performance of integrated GPUs is crap compared to discrete GPUs, especially when the latter was much cheaper several years ago. These days though, discrete GPUs have gotten very expensive off the heels of cryptomining and now "AI", and there are signs integrated GPUs might see significant improvements in performance. If integrated GPUs can provide sufficient performance for the prices demanded, PC users will switch. They couldn't care less if their GPU is integrated or discrete, they care whether they can run Fortnite at 500 frames per second.
- Panzer04 3y ago
  I think it will always be hard to justify. The amount of different configurations proliferate very quickly, so you end up spending a lot of die area etc on parts that much of the market doesn't need. Intel and AMD seem to have settled on sticking a relatively small (but still significant die-wise) GPU, sufficient for 95% of consumers. They can't justify spending the die area needed for a large, powerful GPU that a relatively small segment of the market wants (and even then there's 2-3 cores from low end to high end, just for the GPU) To achieve the same market for today you're looking at 2x CPU dies and 3x GPU dies, or at least 6 different combinations of CPU and GPU, and you still have loads of people who don't want the GPU at all. It just doesn't make sense to fully integrate them unless die area is so cheap that power consumption is the main limiting factor (the big justification for lots of fixed function hardware - but that's all way smaller than a GPU
  - Dalewyn 3y ago
    >Intel and AMD seem to have settled on sticking a relatively small (but still significant die-wise) GPU Intel yes, but AMD only finally began integrating a GPU into most of their CPU line up with the latest Zen 4 generation. >a large, powerful [integrated] GPU that a relatively small segment of the market wants I think you're underselling the demand there, particularly if the M* line of CPUs with iGPUs from Apple are anything to go by. Like I said before, people want powerful graphics processing at an affordable (FSVO affordable with regards to Apple) price. >and you still have loads of people who don't want the GPU at all. Intel's been doing iGPUs for well over a decade now because "loads of people" do want iGPUs. Enthusiasts and professionals alike want them because they are a Lowest Common Denominator fallback for when shit hits the fan. People on a budget (whether monetary or electric) want them regardless of their poor performance because they need a display output, especially in recent years. And even if you have a discrete GPU anyway, an integrated GPU is still useful to have for offloading certain tasks. >It just doesn't make sense to fully integrate them unless die area is so cheap that power consumption is the main limiting factor Five years ago I would have completely agreed with you, but with how bloody expensive discrete GPUs are now I feel there's a vacated market segment where integrated GPUs with sufficient performance and reasonable pricing can come in to conquer and dominate; not unlike how discrete sound cards became extinct when integrated sound cards became Good Enough(tm).

huijzer 3y ago

I think the main benefit of joining two dies together via some advanced interconnect is the latency. Memory lookups are much quicker on Apple Silicon which, in turn, avoids many wasted clock ticks waiting for data.

brucethemoose2 3y ago
Are you talking about the M1 Extreme? This is extremely expensive, and I suspect the dual die GPU only works because of Metal. TBH I prefer AMD's 7900 series approach of pushing the memory controller + cache out onto little memory controller dies. This allows for a monolithic GPU with a really wide bus, without all that wasted space on the controller/cache/pins. And those dies don't care about being split up since the memory access is interleaved anyway.
- 58028641 3y ago
  Isn’t it Pro, Max, Ultra?
  - brucethemoose2 3y ago
    thonk I can't remember, you are probably right. There is the little, medium, and big die, and the 2x big die.
    - sliken 3y ago
      The biggest single die is the m2 max. Dual die chip is the ultra.
- olliej 3y ago
  "M1 Extreme? This is extremely expensive" hence the name :D That said I think it's pro and ultra, and I can't recall which (if not both) use the magic interconnect

flangola7 3y ago

>And seperate CPUs+dGPUs are what users want on PCs In the age of unified memory and large AI models, that may not be the case for very long.

smoldesu 3y ago
What are you talking about? How does a unified memory architecture change the bottlenecks of AI inferencing? The fastest AI servers in existence today only exist in heterogeneous form.
- wmf 3y ago
  If you want, say, 70 GB of VRAM your options are either unified memory or pay $35K.
  - smoldesu 3y ago
    Or you can swap to DRAM and stream over PCI. I do that all the time and it works fine, especially since you rarely need to load all 70gb into memory at once. If you're a serious (read: enterprise) customer, you can buy InfiniBand-enabled cards and get duplex bandwidth faster than the M2 Max's entire memory bus. 'Unified memory' isn't even a bullet point on their spec sheet, it means nothing to their customers when they have CUDA primatives that do the same thing faster at a larger scale.
    - justinclift 3y ago
      > and get duplex bandwidth faster than the M2 Max's entire memory bus. Doesn't seem like it. Looking at the wikipedia article for the M2 Max, that says: ... with up to 400 GB/s memory bandwidth. https://en.wikipedia.org/wiki/Apple_silicon#Apple_M2_Max https://en.wikipedia.org/wiki/Apple_silicon#Apple_M2_Max Looking at the current "Nvidia networking" product page, it lists ConnectX-7 adapters with 400Gb/s total bandwidth: https://www.nvidia.com/en-us/networking/infiniband-adapters/ https://www.nvidia.com/en-us/networking/infiniband-adapters/ So, accounting for the bit -> byte unit difference in those figures it seems like having eight ConnectX-7 adapters would roughly match up. However, the data sheet for those adapters seems to say those 400Gb/s adapters are for PCIe Gen5 x32 slots (not the standard x16 ones). Host interface: PCIe Gen5, up to x32 lanes Eight slots of x32 lanes doesn't quite seem possible, even with very latest generation AMD EPYC (9004 series) processors: https://www.amd.com/system/files/documents/epyc-9004-series-processors-data-sheet.pdf https://www.amd.com/system/files/documents/epyc-9004-series-... Those have 128 PCIe Gen5 lanes per cpu. Dual socket systems seem to expand that out a bit, allowing up to 160x usable PCIe lanes in a server. https://www.servethehome.com/pcie-lanes-and-bandwidth-increase-over-a-decade-for-intel-xeon-and-amd-epyc/ https://www.servethehome.com/pcie-lanes-and-bandwidth-increa... So, at least on paper it doesn't seem possible for a single server with a bunch of PCIe Infiniband links to actually match the bandwidth of the M2 Max memory bus. Maybe 3/4 of it though, which isn't terrible. :)

Dalewyn 3y ago

>And seperate CPUs+dGPUs are what users want on PCs. Intel and AMD tried to sell GPU heavy designs (eDRAM Broadwell, Vega M, Van Gogh (the Steam Deck SoC)) and PC OEMs unequivocally rejected them. That's because the perceived or real performance of integrated GPUs is crap compared to discrete GPUs, especially when the latter was much cheaper several years ago. These days though, discrete GPUs have gotten very expensive off the heels of cryptomining and now "AI", and there are signs integrated GPUs might see significant improvements in performance. If integrated GPUs can provide sufficient performance for the prices demanded, PC users will switch. They couldn't care less if their GPU is integrated or discrete, they care whether they can run Fortnite at 500 frames per second.

Panzer04 3y ago
I think it will always be hard to justify. The amount of different configurations proliferate very quickly, so you end up spending a lot of die area etc on parts that much of the market doesn't need. Intel and AMD seem to have settled on sticking a relatively small (but still significant die-wise) GPU, sufficient for 95% of consumers. They can't justify spending the die area needed for a large, powerful GPU that a relatively small segment of the market wants (and even then there's 2-3 cores from low end to high end, just for the GPU) To achieve the same market for today you're looking at 2x CPU dies and 3x GPU dies, or at least 6 different combinations of CPU and GPU, and you still have loads of people who don't want the GPU at all. It just doesn't make sense to fully integrate them unless die area is so cheap that power consumption is the main limiting factor (the big justification for lots of fixed function hardware - but that's all way smaller than a GPU
- Dalewyn 3y ago
  >Intel and AMD seem to have settled on sticking a relatively small (but still significant die-wise) GPU Intel yes, but AMD only finally began integrating a GPU into most of their CPU line up with the latest Zen 4 generation. >a large, powerful [integrated] GPU that a relatively small segment of the market wants I think you're underselling the demand there, particularly if the M* line of CPUs with iGPUs from Apple are anything to go by. Like I said before, people want powerful graphics processing at an affordable (FSVO affordable with regards to Apple) price. >and you still have loads of people who don't want the GPU at all. Intel's been doing iGPUs for well over a decade now because "loads of people" do want iGPUs. Enthusiasts and professionals alike want them because they are a Lowest Common Denominator fallback for when shit hits the fan. People on a budget (whether monetary or electric) want them regardless of their poor performance because they need a display output, especially in recent years. And even if you have a discrete GPU anyway, an integrated GPU is still useful to have for offloading certain tasks. >It just doesn't make sense to fully integrate them unless die area is so cheap that power consumption is the main limiting factor Five years ago I would have completely agreed with you, but with how bloody expensive discrete GPUs are now I feel there's a vacated market segment where integrated GPUs with sufficient performance and reasonable pricing can come in to conquer and dominate; not unlike how discrete sound cards became extinct when integrated sound cards became Good Enough(tm).