3 ms·
How long does it take the operator to sift through those 10,000 attempts to find the successful one, when it's not a contrived benchmark where the desired answe
by jsheard 2y ago
How long does it take the operator to sift through those 10,000 attempts to find the successful one, when it's not a contrived benchmark where the desired answer is already known ahead of time? LLMs generally don't know when they've failed, they just barrel forwards and leave the user to filter out the junk responses.
- awesomeMilou 2y agoI have an idea! We should train an LLM with reasoning capabilities to sift through all the attempts! /s
- johnny22 2y agowhy /s ? Isn't that an approach some people are actually trying to take?