3 ms·
Got a M5 air recently - my first dive into MacOS land so trying to figure this out too. Seems essentially impossible to get: * pytorch * GPU acceleration *
by Havoc 6mo ago
Got a M5 air recently - my first dive into MacOS land so trying to figure this out too.
Seems essentially impossible to get:
* pytorch
* GPU acceleration
* VM/container like isolation
The virtio-gpu layer gets closest but seems to only pass through graphics GPU not compute GPU so no pytorch
- plufz 6mo agoI need this too, and looked quite a lot on it a year ago. I haven’t had time to check out the recent developments with Docker Model Runner (vllm-metal) or podman libkrun. Did neither of those work for you?
- Havoc 6mo agovllm-metal isn't GPU access but rather a openai compatible end point which I can already do via lm studio endpoint over network >podman libkrun Haven't tried it but research suggests its really shaky still. podman libkrun exposes vulkan while torch expects mps on macs. Sounds like one can force vulkan but that's apparently slow and beta-ish?
- emmelaich 6mo agoI got torch to run in a Cirruslabs Tart instance.
- Havoc 6mo agoBy "Instance" do you mean their cloud platform?
- emmelaich 6mo agoNah, just locally on my macair. TBF, I only got to the point that using device=mps_device didn't fail. I used Sonoma at the time and the image for the vm was ghcr.io/cirruslabs/macos-sequoia-xcode:16.2-beta-3. Python 3.12, as well, because torch didn't work with later versions. import torch mps_device = torch.device("mps") print('device is', mps_device) x = torch.ones(1, device=mps_device) print(x)
- adastra22 6mo agobrew install tart
- binsquare 6mo ago[dead]