11 ms·
It should work fine if you're on Apple silicon, everything else is unfortunately second-class citizen, because I don't have any other devices and it's currently
by cztomsik 3y ago
It should work fine if you're on Apple silicon, everything else is unfortunately second-class citizen, because I don't have any other devices and it's currently just me working on it. But if you'd like to maintain the windows, linux or intel-based mac releases, ping me on discord and I'd be happy to discuss and to accept PRs.
- deleted 3y ago[deleted]
- rgbrgb 3y agoedit: [deleted]
- cztomsik 3y agoIntel works in Ava, it's just slow, for some reason.
- rgbrgb 3y agoah, got it. yeah the llama.cpp speeds I've seen are like 5-10 tokens/second on intel but I've been impressed with the relatively ancient hardware llama.cpp works on (2017 iMac with 8GB RAM at 5.5 t/s with a 7B model!).
- cztomsik 3y agoit's ok, you can put the link back I don't mind the competition :) yes, I know it definitely works, so it has to be some stupid mistake but it's impossible to reproduce it without actual intel mac :-/ I had enough trouble with supporting monterey, which also does not work 100% AFAIK