34 ms·

Well there's a lot of knobs to turn if you want to improve performance. You can always point an LLM at the model card, give it your info, and have it write up t

by xienze 2mo ago

Well there's a lot of knobs to turn if you want to improve performance. You can always point an LLM at the model card, give it your info, and have it write up the command.

Auracle 2mo ago
Sure, but shouldn’t the programs to run the LLMs go “the user has this much vram and the model is this size, so I’ll start with sensible defaults based on that”? You could override, obviously.
- zargon 2mo ago
  Yes, llama.cpp does that.