5 ms·
The part about injecting randomness is the most intersting bit of the article. So if you want your LLM responses to be more distributed (beyond what setting th
by paxys 8mo ago
The part about injecting randomness is the most intersting bit of the article.
So if you want your LLM responses to be more distributed (beyond what setting the temperature will allow), add some random english words to the start of the prompt.
- egeozcan 8mo agoIn a way that sounds like setting the seed.
- paxys 8mo agoKinda, but the same seed will not guarantee the same result the next time around.
- FrancoisBosun 8mo agoMeh, I tell it "use uuidgen and get your randomness from that". Of course, that won't work on ChatGPT web, but works well enough on the command line.
- BoingBoomTschak 8mo agoSounds like dithering to me.
- FloorEgg 8mo agoFwiw: I didn't read the post carefully, this is just a passing by comment. For my own use case I was trying to test consistency or an evaluation process and found that injecting a UUID into the system prompt (busting cache) made a material difference. Without it, resubmitting the same inputs in close time intervals (e.g. 1, 5, or 30 min) would produce very consistent evaluations. Adding the UUID would decrease consistency (showing true evaluation consistency not artificially improved by catching) and highlight ambiguous evaluation criteria that was causing problems. So I wonder how much prompt caching is a factor here. I think these LLM providers (all of them) are caching several layers beyond just tokenization.