7 ms·

The GPU is monstrously good. Depending on the workload, the M1 series GPU using 120W could beat an RTX 3090 using 420W. Same with the CPU. Linux compiled faste

by jorvi 6mo ago

The GPU is monstrously good. Depending on the workload, the M1 series GPU using 120W could beat an RTX 3090 using 420W. Same with the CPU. Linux compiled faster on an M1 than on the fastest Intel i9 at the time, again using only 25% of the power budget. And the M-series has only gotten better. It is kind of sad Apple neglects helping developers optimize games for the M-series because iDevices and MacBooks could be the mobile gaming devices.

ethbr1 6mo ago
Apples and limes. The context of this thread isn't consumer chips, but Apple's analog to an H/B200.
- deleted 6mo ago
  [deleted]
- jimbokun 6mo ago
  Well Apple is in the consumer computing business.
  - bigyabai 6mo ago
    TFA is literally about a B2B deal, not consumer compute.
  - ethbr1 6mo ago
    * Powered by in-house models they've tried to train and in-house M-series inference servers
bigyabai 6mo ago
The GPUs are bottom-barrel for compute-focused industries. It is mobile-grade hardware that arguably can't even scale to prior Mac Pro workloads. > The GPU is monstrously good. Depending on the workload, the M1 series GPU using 120W could beat an RTX 3090 using 420W. You're just listing the TDP max of both chips. If you limit a 3090 to 120W then it would still run laps around an M1 Max in several workloads despite being an 8nm GPU versus a 5nm one. > It is kind of sad Apple neglects helping developers optimize games for the M-series Apple directly advocated for ports like Death Stranding, Cyberpunk 2077 and Resident Evil internally. Advocacy and optimization are not the issue, Apple's obsession over reinventing the wheel with Metal is what puts the Steam Deck ahead. Edit (response to matthewmacleod): > Bold of them to reinvent something that hadn't been invented yet. Vulkan was not the first open graphics API, as most Mac developers will happily inform you.
- matthewmacleod 6mo ago
  Apple's obsession over reinventing the wheel with Metal Bold of them to reinvent something that hadn't been invented yet.
  - deleted 6mo ago
    [deleted]
- brcmthrowaway 6mo ago
  > The GPUs are bottom-barrel for compute-focused industries. It is mobile-grade hardware that arguably can't even scale to prior Mac Pro workloads. Surprised Apple didn't create a TPU-like architecture. Another misstep from John Gianneadrea.
  - bigyabai 6mo ago
    I'm confused how anyone ever thought the NPU would be a good idea. The GPU is almost always underutilized on Mac and could do the brunt of the work for inference if it embraced GPGPU principles from the start. Creating a dedicated hardware block to alleviate a theoretical congestion issue is... bewildering. That goes for most NPUs I've seen. Apple had the technology to scale down a GPGPU-focused architecture just like Nvidia did. They had the money to take that risk, and had the chip design chops to take a serious stab at it. On paper, they could have even extended it to iPhone-level edge silicon similar to what Nvidia did with the Jetson and Tegra SOCs.
    - zozbot234 6mo ago
      I'm not sure even Apple thought that, given that they don't officially provide access to ANE internals under macOS (barring unsupported hacks). But if that was fixed, it could then be useful for improving the power efficiency of prefill, where the CPU/GPU hardware is quite weak (especially prior to the M5 Neural Accelerators).
    - easton 6mo ago
      I think they built the NPU with whatever models they needed to run on the iPhone in mind vs trying to build a general purpose chip, and then got lucky it was also useful for LLMs. (Like “I want to do object detection for cutting people into stickers on device without blowing a hole in the battery, make me a chip for that”.)
AuthAuth 6mo ago
>the M1 series GPU using 120W could beat an RTX 3090 using 420W You're cooked if you actually believe this
- wolvoleo 6mo ago
  Somehow Apple has always been able to sell their stuff as somehow Magic. Remember the megahertz myth? Apple hertzes and apple bytes are much better than PC hertzes and bytes because they are made by virgin elves during a full moon.
  - mschuster91 6mo ago
    > Apple hertzes and apple bytes are much better than PC hertzes and bytes because they are made by virgin elves during a full moon. The thing that Apple has always been excellent at is efficiency - even during the Intel era, MacBooks outclassed their Windows peers. Same CPU, same RAM, same disks, so it definitely wasn't the hardware, it was the software, that allowed Apple to pull much more real-world performance out of the same clock cycles and power usage. Windows itself, but especially third party drivers, are disastrous when it comes to code quality, and they are much much more generic (and thus inefficient) compared to Apple with its very small amount of different SKUs. Apple insisted on writing all drivers and IIRC even most of the firmware for embedded modules themselves to achieve that tight control... which was (in addition to the 2010-ish lead-free Soldergate) why they fired NVIDIA from making GPUs for Apple - NV didn't want to give Apple the specs any more to write drivers.
    - bigyabai 6mo ago
      > NV didn't want to give Apple the specs any more to write drivers. I think that's a valid demand, considering Nvidia's budding commitment to CUDA and other GPGPU paradigms. Apple, backing OpenCL, would have every reason to break Nvidia's code and ship half-baked drivers. They did it with AMD's GPUs later down the line, pretending like Vulkan couldn't be implemented so they could promote Metal. Apple wouldn't have made GeForce more efficient with their own firmware, they would have installed a Sword of Damocles over Nvidia's head.

by jorvi 6mo ago

ethbr1 6mo ago

Apples and limes. The context of this thread isn't consumer chips, but Apple's analog to an H/B200.

deleted 6mo ago
[deleted]
jimbokun 6mo ago
Well Apple is in the consumer computing business.
- bigyabai 6mo ago
  TFA is literally about a B2B deal, not consumer compute.
- ethbr1 6mo ago
  * Powered by in-house models they've tried to train and in-house M-series inference servers

bigyabai 6mo ago

The GPUs are bottom-barrel for compute-focused industries. It is mobile-grade hardware that arguably can't even scale to prior Mac Pro workloads. > The GPU is monstrously good. Depending on the workload, the M1 series GPU using 120W could beat an RTX 3090 using 420W. You're just listing the TDP max of both chips. If you limit a 3090 to 120W then it would still run laps around an M1 Max in several workloads despite being an 8nm GPU versus a 5nm one. > It is kind of sad Apple neglects helping developers optimize games for the M-series Apple directly advocated for ports like Death Stranding, Cyberpunk 2077 and Resident Evil internally. Advocacy and optimization are not the issue, Apple's obsession over reinventing the wheel with Metal is what puts the Steam Deck ahead. Edit (response to matthewmacleod): > Bold of them to reinvent something that hadn't been invented yet. Vulkan was not the first open graphics API, as most Mac developers will happily inform you.

matthewmacleod 6mo ago
Apple's obsession over reinventing the wheel with Metal Bold of them to reinvent something that hadn't been invented yet.
- deleted 6mo ago
  [deleted]
brcmthrowaway 6mo ago
> The GPUs are bottom-barrel for compute-focused industries. It is mobile-grade hardware that arguably can't even scale to prior Mac Pro workloads. Surprised Apple didn't create a TPU-like architecture. Another misstep from John Gianneadrea.
- bigyabai 6mo ago
  I'm confused how anyone ever thought the NPU would be a good idea. The GPU is almost always underutilized on Mac and could do the brunt of the work for inference if it embraced GPGPU principles from the start. Creating a dedicated hardware block to alleviate a theoretical congestion issue is... bewildering. That goes for most NPUs I've seen. Apple had the technology to scale down a GPGPU-focused architecture just like Nvidia did. They had the money to take that risk, and had the chip design chops to take a serious stab at it. On paper, they could have even extended it to iPhone-level edge silicon similar to what Nvidia did with the Jetson and Tegra SOCs.
  - zozbot234 6mo ago
    I'm not sure even Apple thought that, given that they don't officially provide access to ANE internals under macOS (barring unsupported hacks). But if that was fixed, it could then be useful for improving the power efficiency of prefill, where the CPU/GPU hardware is quite weak (especially prior to the M5 Neural Accelerators).
  - easton 6mo ago
    I think they built the NPU with whatever models they needed to run on the iPhone in mind vs trying to build a general purpose chip, and then got lucky it was also useful for LLMs. (Like “I want to do object detection for cutting people into stickers on device without blowing a hole in the battery, make me a chip for that”.)

AuthAuth 6mo ago

>the M1 series GPU using 120W could beat an RTX 3090 using 420W You're cooked if you actually believe this

wolvoleo 6mo ago
Somehow Apple has always been able to sell their stuff as somehow Magic. Remember the megahertz myth? Apple hertzes and apple bytes are much better than PC hertzes and bytes because they are made by virgin elves during a full moon.
- mschuster91 6mo ago
  > Apple hertzes and apple bytes are much better than PC hertzes and bytes because they are made by virgin elves during a full moon. The thing that Apple has always been excellent at is efficiency - even during the Intel era, MacBooks outclassed their Windows peers. Same CPU, same RAM, same disks, so it definitely wasn't the hardware, it was the software, that allowed Apple to pull much more real-world performance out of the same clock cycles and power usage. Windows itself, but especially third party drivers, are disastrous when it comes to code quality, and they are much much more generic (and thus inefficient) compared to Apple with its very small amount of different SKUs. Apple insisted on writing all drivers and IIRC even most of the firmware for embedded modules themselves to achieve that tight control... which was (in addition to the 2010-ish lead-free Soldergate) why they fired NVIDIA from making GPUs for Apple - NV didn't want to give Apple the specs any more to write drivers.
  - bigyabai 6mo ago
    > NV didn't want to give Apple the specs any more to write drivers. I think that's a valid demand, considering Nvidia's budding commitment to CUDA and other GPGPU paradigms. Apple, backing OpenCL, would have every reason to break Nvidia's code and ship half-baked drivers. They did it with AMD's GPUs later down the line, pretending like Vulkan couldn't be implemented so they could promote Metal. Apple wouldn't have made GeForce more efficient with their own firmware, they would have installed a Sword of Damocles over Nvidia's head.