5 ms·
Requirements: > A decent computer with either a CUDA supported graphics card or M1 processor. Why so? How does an M1 processor replace CUDA in a way a x86_64
by gbighin 4y ago
Requirements:
> A decent computer with either a CUDA supported graphics card or M1 processor.
Why so? How does an M1 processor replace CUDA in a way a x86_64 processor can't? Do they use ARM assembly?
- pavlov 4y agoIt’s not the ARM core but the integrated GPU in the M1. It has access to the entire main memory unlike a traditional GPU with its own local VRAM.
- gbighin 4y agoOh, interesting! But does it support CUDA? How is the integrated GPU used for ML tasks?
- Filligree 4y agoIt does not support CUDA; SD does not require CUDA.
- pavlov 4y agoI believe there’s a Tensorflow acceleration adapter for Apple’s ML API which uses Metal behind the scenes.
- hnarayanan 4y agoBoth PyTorch and TensorFlow offer backends for Metal that works pretty well on Apple Silicon.
- dagmx 4y agoTo add to what people said, most of these ML models target an ML library like TensorFlow or PyTorch. Those in turn have hardware accelerated backends. Traditionally they’ve only had CUDA backends but Apple ported large chunks of both to Metal as well. So none of these libraries really target CUDA. In fact they’d run fine without a supported GPU but much slower.
- malshe 4y agopytorch can use the GPUs on M1 macs. Sebastian Raschka's post explains it nicely and shows some benchmarks too. https://sebastianraschka.com/blog/2022/pytorch-m1-gpu.html https://sebastianraschka.com/blog/2022/pytorch-m1-gpu.html From his post: if you want to run PyTorch code on the GPU, use torch.device("mps") analogous to torch.device("cuda") on an Nvidia GPU.
- crucialfelix 4y agoIn some cases there are operations not supported on mps. For those set: os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1" and it will run on cpu if some operation isn't supported
- malshe 4y agoExcellent! Thanks