3 ms·
Sure, but shouldn’t the programs to run the LLMs go “the user has this much vram and the model is this size, so I’ll start with sensible defaults based on that”
by Auracle 3mo ago
Sure, but shouldn’t the programs to run the LLMs go “the user has this much vram and the model is this size, so I’ll start with sensible defaults based on that”?
You could override, obviously.
- zargon 3mo agoYes, llama.cpp does that.