I was asking the same question, turns out mistral.rs [0] has pretty good abstractions in order to not depend and package llama.cpp for every platform. [0] http

I was asking the same question, turns out mistral.rs [0] has pretty good abstractions in order to not depend and package llama.cpp for every platform. [0] https://github.com/EricLBuehler/mistral.rs https://github.com/EricLBuehler/mistral.rs