3 ms·
So what if the output is stochastic? LLM's have self consistency, so you can repeat the inference several times and pick the most frequent output.
by computerex 2y ago
So what if the output is stochastic? LLM's have self consistency, so you can repeat the inference several times and pick the most frequent output.
- saulpw 2y agoAnd you can pay in time and/or $$ for the privilege of having to do do this extra unnecessary work.
- dazilcher 2y agoMost frequent output does not imply correctness, LLMs often are confidently wrong. They can't even perform basic arithmetic (which is not surprising since they operate at the syntactic level, oblivious to any semantic rules), yet people seem to think offloading more complex tasks with strict correctness requirements is a good idea. Boggles the mind tbh.