4 ms·
Does this require M2 AI capabilities or can it also run on other platforms?
by shim__ 3y ago
Does this require M2 AI capabilities or can it also run on other platforms?
- guywhocodes 3y agoSeems to be running on llama.cpp, so it's going to be a question of performance. I don't have any M-cpu but on my 13th gen i5 I can run mistral at about 6.5 tokens per second. Which seems comparable to what this is.
- SushiHippie 3y agoIt just uses llama.cpp as a "backend", so anywhere where llama.cpp works this should work too if I see this correctly.