3 ms·
Somewhat faster on small models. Requires new format. Not sure what the goal is for this project? Not seeing how this presents adequate benefits to get adopted
by dcreater 1y ago
Somewhat faster on small models. Requires new format.
Not sure what the goal is for this project? Not seeing how this presents adequate benefits to get adopted by the community
- koakuma-chan 1y agoWritten in Rust is a big one for me.
- worldsavior 1y agoIt's utilizing Apple ANE and probably other optimization tools provided by Apple's framework. Not sure if llama.cpp uses them, but if they're not then the benchmark on GitHub says it all.