3 ms·

If you want factual answers from a local model it might help to turn the temperature down.

• deleted 3y ago
  [deleted]
• jrflowers 3y ago
  > If you want factual answers from a local model it might help to turn the temperature down. This makes sense. If you interact with a language model and it says something wrong it is your fault
  • astrange 3y ago
    You're not "interacting with a language model", you're running a program (llama.cpp) with a sampling algorithm which is not set to maximum factualness by default. It's like how you have to set x264 to the anime tuning or the film tuning depending on what you run it on.
• fragmede 3y ago
  It would also help if I had more VRAM and wasn't running a 7B parameter 4-bit quantized model.