5 ms·
PyTorch 1.13
- jstx1 4y ago> Inspired by Google® JAX, functorch is a library that offers composable vmap (vectorization) and autodiff transforms. It enables advanced autodiff use cases that would otherwise be tricky to express in PyTorch It's very nice that they're open about borrowing ideas that they like from other projects.
- mi_lk 4y agois anyone able to comment on the experience of PyTorch's M1 support?
- DIVx0 4y agoI've used it for Stable Diffusion on my M1 mbp, works with no problem. I am not a AI/ML person so my experience with M1 PyTorch is limited to just using SD.
- jstx1 4y agoIf you use miniforge (https://github.com/conda-forge/miniforge https://github.com/conda-forge/miniforge) you can just 'pip install torch' and it works.
- gcarvalho 4y agoWhat's different about miniforge in this regard? Does miniforge come with additional package indexes pre-configured?
- simongr3dal 4y agoPretty much. Conda installed with miniforge uses a different channel, with a focus on having packages available for more architectures.
- tomtom1337 4y agoI think you mean “conda install torch”, no? Miniforge would only impact the conda installer?
- jstx1 4y agoSurprisingly, no. I just tried it: - 'pip install torch' doesn't work inside of a plain Python env (not miniforge) - 'conda install torch' doesn't work inside miniforge - 'pip install torch' works inside miniforge
- tomtom1337 4y agoHuh! That’s curious! I wonder what causes the different behaviour!
- glial 4y agohttps://pytorch.org/docs/stable/notes/mps.html https://pytorch.org/docs/stable/notes/mps.html In my very limited experience it's not a ton faster than using a CPU backend though.
- jackosdev 4y agoYeah works great, Tensorflow works great as well, but if using pip need to install tensorflow-metal and tensorflow-macos. Plus I always need to downgrade protobuf to 3.20. And couldn't get the C bindings for Tensorflow to work through metal, only through CPU. Pretty good machine learning experience on M1.
- ryemigie 4y agoIt doesn’t support a few things, and has bugs in a few of the functions but besides that it works OK. Need to make sure your batch size is high enough to fully utilise the GPU cores. I think it was maybe 2-4 times faster than the CPU in my case.
- Birch-san 4y agoMost things require workarounds, some things aren't possible (or we haven't found workaround yet) and it's not as fast as CUDA. But stable-diffusion inference works, and so does textual inversion training. I was also able to run training of a T5 model with just a couple of tweaks. I'd stick with PyTorch 1.12.1 for now. 1.13 has problems with backpropagation (I get NaN gradients now when I attempt CLIP-guided diffusion -- I think this applies to training too), and some einsum formulations are 50% slower (there is a patch to fix this; I expect it'll be merged soon), making big self-attention matmuls slow and consequently making stable-diffusion inference ~6% slower.
- TheChaplain 4y agoI'm curious about PyTorch but never really found a problem that it would be useful for.. Any ideas?
- jstx1 4y agoIt's used for deep learning.
- pjmlp 4y agoBest part of PyTorch seems to be the availability of Java and C++ bindings as well, so we can skip the Python part.