I run it with Llama.cpp on my RTX 3090. Also using the same Unsloth model. My config is similar to: https://github.com/noonghunna/club-3090/blob/master/docs/en

I run it with Llama.cpp on my RTX 3090. Also using the same Unsloth model. My config is similar to: https://github.com/noonghunna/club-3090/blob/master/docs/engines/LLAMA_CPP.md https://github.com/noonghunna/club-3090/blob/master/docs/eng... I need to try out some of the other set ups mentioned in this repo for increased TPS.