3 ms·

is anyone able to comment on the experience of PyTorch's M1 support?

by mi_lk 4y ago

DIVx0 4y ago
I've used it for Stable Diffusion on my M1 mbp, works with no problem. I am not a AI/ML person so my experience with M1 PyTorch is limited to just using SD.
jstx1 4y ago
If you use miniforge (https://github.com/conda-forge/miniforge https://github.com/conda-forge/miniforge) you can just 'pip install torch' and it works.
- gcarvalho 4y ago
  What's different about miniforge in this regard? Does miniforge come with additional package indexes pre-configured?
  - simongr3dal 4y ago
    Pretty much. Conda installed with miniforge uses a different channel, with a focus on having packages available for more architectures.
- tomtom1337 4y ago
  I think you mean “conda install torch”, no? Miniforge would only impact the conda installer?
  - jstx1 4y ago
    Surprisingly, no. I just tried it: - 'pip install torch' doesn't work inside of a plain Python env (not miniforge) - 'conda install torch' doesn't work inside miniforge - 'pip install torch' works inside miniforge
    - tomtom1337 4y ago
      Huh! That’s curious! I wonder what causes the different behaviour!
glial 4y ago
https://pytorch.org/docs/stable/notes/mps.html https://pytorch.org/docs/stable/notes/mps.html In my very limited experience it's not a ton faster than using a CPU backend though.
jackosdev 4y ago
Yeah works great, Tensorflow works great as well, but if using pip need to install tensorflow-metal and tensorflow-macos. Plus I always need to downgrade protobuf to 3.20. And couldn't get the C bindings for Tensorflow to work through metal, only through CPU. Pretty good machine learning experience on M1.
ryemigie 4y ago
It doesn’t support a few things, and has bugs in a few of the functions but besides that it works OK. Need to make sure your batch size is high enough to fully utilise the GPU cores. I think it was maybe 2-4 times faster than the CPU in my case.
Birch-san 4y ago
Most things require workarounds, some things aren't possible (or we haven't found workaround yet) and it's not as fast as CUDA. But stable-diffusion inference works, and so does textual inversion training. I was also able to run training of a T5 model with just a couple of tweaks. I'd stick with PyTorch 1.12.1 for now. 1.13 has problems with backpropagation (I get NaN gradients now when I attempt CLIP-guided diffusion -- I think this applies to training too), and some einsum formulations are 50% slower (there is a patch to fix this; I expect it'll be merged soon), making big self-attention matmuls slow and consequently making stable-diffusion inference ~6% slower.

is anyone able to comment on the experience of PyTorch's M1 support?

by mi_lk 4y ago

is anyone able to comment on the experience of PyTorch's M1 support?

I've used it for Stable Diffusion on my M1 mbp, works with no problem. I am not a AI/ML person so my experience with M1 PyTorch is limited to just using SD.

jstx1 4y ago

If you use miniforge (https://github.com/conda-forge/miniforge https://github.com/conda-forge/miniforge) you can just 'pip install torch' and it works.

gcarvalho 4y ago
What's different about miniforge in this regard? Does miniforge come with additional package indexes pre-configured?
- simongr3dal 4y ago
  Pretty much. Conda installed with miniforge uses a different channel, with a focus on having packages available for more architectures.
tomtom1337 4y ago
I think you mean “conda install torch”, no? Miniforge would only impact the conda installer?
- jstx1 4y ago
  Surprisingly, no. I just tried it: - 'pip install torch' doesn't work inside of a plain Python env (not miniforge) - 'conda install torch' doesn't work inside miniforge - 'pip install torch' works inside miniforge
  - tomtom1337 4y ago
    Huh! That’s curious! I wonder what causes the different behaviour!

glial 4y ago

https://pytorch.org/docs/stable/notes/mps.html https://pytorch.org/docs/stable/notes/mps.html In my very limited experience it's not a ton faster than using a CPU backend though.

jackosdev 4y ago

Yeah works great, Tensorflow works great as well, but if using pip need to install tensorflow-metal and tensorflow-macos. Plus I always need to downgrade protobuf to 3.20. And couldn't get the C bindings for Tensorflow to work through metal, only through CPU. Pretty good machine learning experience on M1.

ryemigie 4y ago

It doesn’t support a few things, and has bugs in a few of the functions but besides that it works OK. Need to make sure your batch size is high enough to fully utilise the GPU cores. I think it was maybe 2-4 times faster than the CPU in my case.

Birch-san 4y ago

Most things require workarounds, some things aren't possible (or we haven't found workaround yet) and it's not as fast as CUDA. But stable-diffusion inference works, and so does textual inversion training. I was also able to run training of a T5 model with just a couple of tweaks. I'd stick with PyTorch 1.12.1 for now. 1.13 has problems with backpropagation (I get NaN gradients now when I attempt CLIP-guided diffusion -- I think this applies to training too), and some einsum formulations are 50% slower (there is a patch to fix this; I expect it'll be merged soon), making big self-attention matmuls slow and consequently making stable-diffusion inference ~6% slower.