4 ms·
That runs into the issue that nobody runs LLMs with a temperature of zero.
by josefx 1y ago
That runs into the issue that nobody runs LLMs with a temperature of zero.
- bird0861 1y agoNot true. Perhaps very few do, but some do in fact run them at 0. I've done it myself. There are many small models that will gladly perform well in QA with temp 0. Of course there are few situations where this is the recommended setup -- we all know RAG takes less than a billion parameters now to do effectively. But nevertheless there are people who do this, and there are plausibly some use cases for it.