4 ms·

Thanks! Did you try it with lmstudio? I actually never tried it with that. Only published ollama, llamfile, llama.cpp native/prompt - and unofficially tested vL

Thanks! Did you try it with lmstudio? I actually never tried it with that. Only published ollama, llamfile, llama.cpp native/prompt - and unofficially tested vLLM, but never lmstudio.

tmjdev 5mo ago
I couldn't get it working with lmstudio. I have bonsai-8B running with llama-cpp and am attempting to build a harness for it. Looking good so far, I just got it started but Forge made tool calling work pretty quickly!
- zambelli 5mo ago
  Very cool! I'll try to get an issue open on lmstudio support and add it to the backlog.
DeathArrow 5mo ago
Maybe you can update the guide and tell us how to use vLLM with Forge when you find some time?
- zambelli 5mo ago
  [dead]
monster_truck 5mo ago
Yessir! Been a longtime fan of it, I've spent too many fuckin years wrangling python, especially pytorch, especially on AMD, dep issues for fun and profit... they don't get enough flowers. It's oai compat, no thorns.