4 ms·

I hope this kickstarts some kind of M1 migration. There are so many ML projects I'd like to try, but they all depend on CUDA.

joshstrange 4y ago
Yep, I was just thinking the same thing. M1/M2 appears to be a huge untapped resource for ML stuff as this proves. I maxed out my MBP Max and this is probably the first time I'm actually fully using the GPU cores and it's pretty freaking cool. Creating landscapes or fictional characters (think D&D) is already super fun, I look forward to playing with img2img some more as well.
- zone411 4y ago
  The performance gap to the top-end Nvidia cards will get much larger as they release new cards later this year, though.
  - jdminhbg 4y ago
    Maybe, but I can buy a Mac, you just order one from Apple.
    - zone411 4y ago
      It's not clear if the shortages will happen with this new release as they did last time. Ethereum mining is going away and not as many people are stuck at home because of Covid. On the other hand, the performance increase looks to be substantial, increasing the demand.
    - wincy 4y ago
      An RTX 3090ti with 24GB of VRAM is widely available now that the crypto markets have crashed for $1150 or so. They were $2500 a year ago if you could find them.
      - sethhochberg 4y ago
        A twist on the above comment: I _already own_ an M2 Mac, but I'm never gonna buy a high-end GPU to play around with this sort of tech. If the things people (who aren't gamers, crypto miners, or ML researchers) already own can be useful for some hobby-level work in the space, we'll see a lot more work and experimentation in the space. Its super exciting stuff.
        fomine3 4y ago
        To be fair now many people have gaming PC. Perhaps more than who have their own M1 Pro/Max.
        ciex 4y ago
        It’s different kinds of people. I know more people with an apple mx computer than with a gaming computer (people have a console to game or nothing at all).
        onli 4y ago
        Perhaps, you think? ^^ The gaming PC market is huge. Have a look at https://www.businesswire.com/news/home/20210329005150/en/Global-Gaming-PC-and-Monitor-Market-Hit-New-Record-High-in-2020-According-to-IDC https://www.businesswire.com/news/home/20210329005150/en/Glo... to get some numbers. There is a list of shipments in a year. Apple sells a lot of units, but not nearly enough to match the accumulated household supplies of gaming PCs - in how much, 2 years, while gaming PCs and laptops are still being sold? Don't take that comment personally please, but this "perhaps" is a perfect example of being in a complete Apple bubble. It's so far from reality it is frankly unfathomable.
      - quitit 4y ago
        Arguably the cost effective solution is to use cloud services, since we're talking just a few seconds difference (or you might be lucky like one HN reader who got allocated an A100 today.) But to play devil's advocate there are clear strengths available to the different platforms. PCs can readily upgrade into high end GPUs, but the compromise is that this becomes a requirement as basic GPUs don't feature enough VRAM and CPU-only mode is woeful. On the mac side of things, the GPU is not going to be the latest and greatest, but the M-series features unified memory, so a relatively normal M-series mac is going to have the necessary hardware to load the models. Not the fastest (but still fast), and ready to go. (Also as it stands the M-series can offer additional pathways to optimisation.)
- icedchai 4y ago
  Same here. My M1 Max's GPUs were basically idling until this came along!
bee_rider 4y ago
Do they depend on CUDA, or are they just much better tuned for NVIDIA cards? I thought the whole ML ecosystem was based on training models and then running them on frameworks, where model was sorta like data and the framework handles the hardware? (albeit with models that can be tweaked to run more efficiently on different hardware) (I don't really know the ecosystem so it is definitely possible that they are more closely tied together than I thought).
- upbeat_general 4y ago
  From my experience the bigger frameworks may have support for non-CUDA devices (that is not just the CPU fallback) but many smaller libraries and models will not, and will only have a CUDA kernel for some specialized operation. I encounter this all the time in computer vision models.
- joshvm 4y ago
  The latter. The major frameworks, at least, can be run in CPU-only mode, with a hardware abstraction layer for other devices (like CUDA-capable cards, TPUs etc). So practically it means you need an Nvidia GPU to get anywhere in a reasonable amount of time, but if you're not super dependent on latency (for inference) then CPU is an option. In principle, CPUs can run much bigger model inputs (at the expense of even more latency) because RAM is an order of magnitude more available typically.
  - bee_rider 4y ago
    I was thinking (as someone who knows nothing about this really) that the Apple chips might be interesting because, while they obviously don't have the GPGPU grunt to compete with NVIDIA, they might have a more practical memory:compute ratio... depending on the application of course.
  - fomine3 4y ago
    Is there any blocker to have VRAM swap (on RAM or SSD)? It would make processing much slower, but it should be better than nothing (cause OOM) or alternatively run on CPU (more slower).
fezfight 4y ago
It's not really better to move from one closed eco system to another. We should collectively agree to strengthen more open platforms, shouldn't we?
inamberclad 4y ago
I'd rather see something more platform agnostic. I'm sad OpenCL isn't a bigger success.
- pixelpoet 4y ago
  OpenCL still works amazingly well on all platforms (e.g. two commercial programs of mine), it's just that everyone keeps saying it's dead and refusing to use it :(
danaris 4y ago
Gods, yes. I've been trying to get various kinds of deepfake-related projects to work on M1 (or even just on Mac—the Intel Macs haven't come with Nvidia cards for years now) for some time now, as we're trying to generate video stimuli that present different people doing the same things, or the same person doing several different things, for psych research. It's been an exercise in frustration.

joshstrange 4y ago

Yep, I was just thinking the same thing. M1/M2 appears to be a huge untapped resource for ML stuff as this proves. I maxed out my MBP Max and this is probably the first time I'm actually fully using the GPU cores and it's pretty freaking cool. Creating landscapes or fictional characters (think D&D) is already super fun, I look forward to playing with img2img some more as well.

zone411 4y ago
The performance gap to the top-end Nvidia cards will get much larger as they release new cards later this year, though.
- jdminhbg 4y ago
  Maybe, but I can buy a Mac, you just order one from Apple.
  - zone411 4y ago
    It's not clear if the shortages will happen with this new release as they did last time. Ethereum mining is going away and not as many people are stuck at home because of Covid. On the other hand, the performance increase looks to be substantial, increasing the demand.
  - wincy 4y ago
    An RTX 3090ti with 24GB of VRAM is widely available now that the crypto markets have crashed for $1150 or so. They were $2500 a year ago if you could find them.
    - sethhochberg 4y ago
      A twist on the above comment: I _already own_ an M2 Mac, but I'm never gonna buy a high-end GPU to play around with this sort of tech. If the things people (who aren't gamers, crypto miners, or ML researchers) already own can be useful for some hobby-level work in the space, we'll see a lot more work and experimentation in the space. Its super exciting stuff.
      - fomine3 4y ago
        To be fair now many people have gaming PC. Perhaps more than who have their own M1 Pro/Max.
        ciex 4y ago
        It’s different kinds of people. I know more people with an apple mx computer than with a gaming computer (people have a console to game or nothing at all).
        onli 4y ago
        Perhaps, you think? ^^ The gaming PC market is huge. Have a look at https://www.businesswire.com/news/home/20210329005150/en/Global-Gaming-PC-and-Monitor-Market-Hit-New-Record-High-in-2020-According-to-IDC https://www.businesswire.com/news/home/20210329005150/en/Glo... to get some numbers. There is a list of shipments in a year. Apple sells a lot of units, but not nearly enough to match the accumulated household supplies of gaming PCs - in how much, 2 years, while gaming PCs and laptops are still being sold? Don't take that comment personally please, but this "perhaps" is a perfect example of being in a complete Apple bubble. It's so far from reality it is frankly unfathomable.
    - quitit 4y ago
      Arguably the cost effective solution is to use cloud services, since we're talking just a few seconds difference (or you might be lucky like one HN reader who got allocated an A100 today.) But to play devil's advocate there are clear strengths available to the different platforms. PCs can readily upgrade into high end GPUs, but the compromise is that this becomes a requirement as basic GPUs don't feature enough VRAM and CPU-only mode is woeful. On the mac side of things, the GPU is not going to be the latest and greatest, but the M-series features unified memory, so a relatively normal M-series mac is going to have the necessary hardware to load the models. Not the fastest (but still fast), and ready to go. (Also as it stands the M-series can offer additional pathways to optimisation.)
icedchai 4y ago
Same here. My M1 Max's GPUs were basically idling until this came along!

bee_rider 4y ago

Do they depend on CUDA, or are they just much better tuned for NVIDIA cards? I thought the whole ML ecosystem was based on training models and then running them on frameworks, where model was sorta like data and the framework handles the hardware? (albeit with models that can be tweaked to run more efficiently on different hardware) (I don't really know the ecosystem so it is definitely possible that they are more closely tied together than I thought).

upbeat_general 4y ago
From my experience the bigger frameworks may have support for non-CUDA devices (that is not just the CPU fallback) but many smaller libraries and models will not, and will only have a CUDA kernel for some specialized operation. I encounter this all the time in computer vision models.
joshvm 4y ago
The latter. The major frameworks, at least, can be run in CPU-only mode, with a hardware abstraction layer for other devices (like CUDA-capable cards, TPUs etc). So practically it means you need an Nvidia GPU to get anywhere in a reasonable amount of time, but if you're not super dependent on latency (for inference) then CPU is an option. In principle, CPUs can run much bigger model inputs (at the expense of even more latency) because RAM is an order of magnitude more available typically.
- bee_rider 4y ago
  I was thinking (as someone who knows nothing about this really) that the Apple chips might be interesting because, while they obviously don't have the GPGPU grunt to compete with NVIDIA, they might have a more practical memory:compute ratio... depending on the application of course.
- fomine3 4y ago
  Is there any blocker to have VRAM swap (on RAM or SSD)? It would make processing much slower, but it should be better than nothing (cause OOM) or alternatively run on CPU (more slower).

fezfight 4y ago

It's not really better to move from one closed eco system to another. We should collectively agree to strengthen more open platforms, shouldn't we?

inamberclad 4y ago

I'd rather see something more platform agnostic. I'm sad OpenCL isn't a bigger success.

pixelpoet 4y ago
OpenCL still works amazingly well on all platforms (e.g. two commercial programs of mine), it's just that everyone keeps saying it's dead and refusing to use it :(

danaris 4y ago

Gods, yes. I've been trying to get various kinds of deepfake-related projects to work on M1 (or even just on Mac—the Intel Macs haven't come with Nvidia cards for years now) for some time now, as we're trying to generate video stimuli that present different people doing the same things, or the same person doing several different things, for psych research. It's been an exercise in frustration.