4 ms·
Both answers were run through the LLM. Answer variability is caused by different web links being returned and the way we sample answers from the LLM. We're work
by rushingcreek 4y ago
Both answers were run through the LLM. Answer variability is caused by different web links being returned and the way we sample answers from the LLM. We're working on making it more consistent.
- dhc02 4y agoConsistency has to be a tough problem to solve for a service like this, since randomness in the choice of each token is part of the magic sauce that makes LLMs work.
- rushingcreek 4y agoIt is definitely a hard problem. There are ways to ensure consistency, such as using beam search decoding, which is deterministic. But that comes with other tradeoffs regarding answer quality.
- zamalek 4y agoI prefer less consistency, it allows me to keep probing until I get the style of answer I'm after.