2 ms·

Apple will gladly sell you a GPU with 192GB of memory, but your wallet won't like it.

kbenson 2y ago
Won't Nvidia, and Intel, and Qualcomm, and Falanx (who make the ARM Mali GPUs from what I can see), and Imagination Technologies (PowerVR) do the same? They each make a GPU, and if you pay them enough money I have a hard time beleiving they won't figure out how to slap enough RAM on a board for one of their existing products and making whatever changes are required.
nextaccountic 2y ago
The US government is looking into heavily limit availability of high end GPUs from now on. And the biggest and most effective bottleneck for AI right now is VRAM So maybe Apple is happy to sell huge GPUs like that but the government will probably put it under export controls like A100 and H100 already is
- rbanffy 2y ago
  Cue the PowerMac G4 TV ad. https://youtu.be/lb7EhYy-2RE https://youtu.be/lb7EhYy-2RE
rbanffy 2y ago
OTOH, it comes free with one of the finest Unix workstations ever made.
- Rinzler89 2y ago
  Which Unix workstation?
  - coolspot 2y ago
    They are referring to MacOS being included with expensive Mac hardware.
    - rbanffy 2y ago
      How many desktop systems can have 192GB visible to the GPU? How many cost less than a Mac?
      - Rinzler89 2y ago
        Just because it has a lot of GPU RAM doesn't mean it's actually useful for people doing ML work. How many companies use Macs for ML work instead of Nvidia and Cuda?
        rbanffy 2y ago
        It won’t be as fast as a high-end GPU like the MI300 series, but it’s enough to check whether the code works before running it on a high-end GPU-heavy machine and the large GPU-accessible RAM simplifies the code enormously, as you don’t have to partition and shuffle data between CPU and GPU.
        Rinzler89 2y ago
        Ok that's the theory but how many companies actually do that for their workflow. All the ML companies I saw use directly Cuda for prototyping to production and don't bother with Apple ML unless their target happens to be exclusively iPhones.
        rbanffy 2y ago
        Anyone doing heavy lifting and low-level tooling will be better to optimise for specialised training and inference engines. Usage will depend on where the abstraction layer is - if you want to see CUDA, then you'll need Nvidia. If all you care for is the size of the model and you know it's very large, then the Apple hardware becomes competitive. Besides, you'd be well served with a Mac as a development desktop anyway.
        Rinzler89 2y ago
- bitsandboots 2y ago
  It's easy to be best when you have no competition. Linux exists for the rest of us.
  - rbanffy 2y ago
    It’s good even if compared to Linux. Not perfect, but certainly not bad.

Won't Nvidia, and Intel, and Qualcomm, and Falanx (who make the ARM Mali GPUs from what I can see), and Imagination Technologies (PowerVR) do the same? They each make a GPU, and if you pay them enough money I have a hard time beleiving they won't figure out how to slap enough RAM on a board for one of their existing products and making whatever changes are required.

nextaccountic 2y ago

The US government is looking into heavily limit availability of high end GPUs from now on. And the biggest and most effective bottleneck for AI right now is VRAM So maybe Apple is happy to sell huge GPUs like that but the government will probably put it under export controls like A100 and H100 already is

rbanffy 2y ago
Cue the PowerMac G4 TV ad. https://youtu.be/lb7EhYy-2RE https://youtu.be/lb7EhYy-2RE

rbanffy 2y ago

OTOH, it comes free with one of the finest Unix workstations ever made.

Rinzler89 2y ago
Which Unix workstation?
- coolspot 2y ago
  They are referring to MacOS being included with expensive Mac hardware.
  - rbanffy 2y ago
    How many desktop systems can have 192GB visible to the GPU? How many cost less than a Mac?
    - Rinzler89 2y ago
      Just because it has a lot of GPU RAM doesn't mean it's actually useful for people doing ML work. How many companies use Macs for ML work instead of Nvidia and Cuda?
      - rbanffy 2y ago
        It won’t be as fast as a high-end GPU like the MI300 series, but it’s enough to check whether the code works before running it on a high-end GPU-heavy machine and the large GPU-accessible RAM simplifies the code enormously, as you don’t have to partition and shuffle data between CPU and GPU.
        Rinzler89 2y ago
        Ok that's the theory but how many companies actually do that for their workflow. All the ML companies I saw use directly Cuda for prototyping to production and don't bother with Apple ML unless their target happens to be exclusively iPhones.
        rbanffy 2y ago
        Anyone doing heavy lifting and low-level tooling will be better to optimise for specialised training and inference engines. Usage will depend on where the abstraction layer is - if you want to see CUDA, then you'll need Nvidia. If all you care for is the size of the model and you know it's very large, then the Apple hardware becomes competitive. Besides, you'd be well served with a Mac as a development desktop anyway.
        Rinzler89 2y ago
bitsandboots 2y ago
It's easy to be best when you have no competition. Linux exists for the rest of us.
- rbanffy 2y ago
  It’s good even if compared to Linux. Not perfect, but certainly not bad.