I'd love to use local models, but seems like most of the easy to use software out there (LM Studio, Backyard AI, koboldcpp) doesn't really play all that nicely

I'd love to use local models, but seems like most of the easy to use software out there (LM Studio, Backyard AI, koboldcpp) doesn't really play all that nicely with my Intel Arc GPU and it's painfully slow on my Ryzen 5 4500. Even my M1 MacBook isn't that fast at generating text with even 7B models. I wonder if llama.cpp with SYCL could help, will have to try it out: https://github.com/ggerganov/llama.cpp/blob/master/README-sycl.md https://github.com/ggerganov/llama.cpp/blob/master/README-sy... But even if that worked, I'd still have the problem that IDEs and whatever else I have open already eats most of the 32 GB of RAM my desktop PC has. Whereas if I ran a small code model on the MacBook and connected to it through my PC, it'd still probably be too slow for autocomplete, when compared to GitHub Copilot and less accurate than ChatGPT or Phind for most stuff.