4 ms·
> For my local setup, I’m currently [..] and LM Studio as the inference server, although it would likely be faster if I just used llama.cpp directly Is there a
by andix 4mo ago
> For my local setup, I’m currently [..] and LM Studio as the inference server, although it would likely be faster if I just used llama.cpp directly
Is there any truth to this claim? LM Studio uses llama.cpp to run the models. I guess the overhead of LM Studio should be minimal.
After all LM Studio is a really easy way to host models, are there really major drawbacks?
- cuvinny 4mo agoLM Studio has a lot less tuning options when you launch it. Also it is precompiled so you don't have the latest (and sometimes buggy) releases so you may have to wait a few days to try out a MTP or a new model. LM Studio is easier though.