6 ms·
Amazing project. My only blocker for installing Asahi on my Mac Studio (M1 ultra) is that llama.cpp performance is very bad when compared to using the Metal ba
by tarruda 28d ago
Amazing project.
My only blocker for installing Asahi on my Mac Studio (M1 ultra) is that llama.cpp performance is very bad when compared to using the Metal backend on the same hardware.
- zer0zzz 28d agoI have similar issues with my ultra. Have put it on a laptop, but my desk Setup with my multiple desktop computers is usb-c video based ( kind of a poor man’s kvm) , and asahi doesn’t do usb-c video yet
- apatheticonion 28d agoRight!? Linux can be configured to idle at less than 500mb of ram - by contrast, with some aggressive shenanigans, MacOS can idle at 2-3gb. When running inference, I would love to use all of my hardware please Apple.
- aurareturn 27d agoIt idles at 2-3GB but when your apps need the RAM, does it free up system memory so that it's closer to 500MB?
- tarruda 27d agoI use my Mac as a headless server to run LLMs, and never even start a desktop session so it always stays at pre-login state without any applications running. Even so, I never saw it using less than 2G in htop when logged via SSH.