2 ms·
Curious if you tested llama.cpp and still found oMLX faster? I haven't tried the latter myself, might give it a go.
by regexorcist 5mo ago
Curious if you tested llama.cpp and still found oMLX faster? I haven't tried the latter myself, might give it a go.
- egorfine 5mo agoOh yeah I did test various solutions and different settings and quants Llama is about 1/3 slower on Apple Silicon.