4 ms·
I was looking for this and to even build something like this in order to have a pure Rust stack for my backend, as well as embed local inference on edge like on
by satvikpendem 2mo ago
I was looking for this and to even build something like this in order to have a pure Rust stack for my backend, as well as embed local inference on edge like on mobile, but llama.cpp is just too far advanced to catch up, it seems. There are some but I'm not sure how good they are.
- aspbee555 2mo agoI even found a rust rewrite of llama.cpp, the cpu performance was horrible and vulkan caused my video to stutter it was way better/easier to use rust bindings to llama.cpp