4 ms·
> running the task through each model and then picking the cheapest correct option What system knows what the correct option is and how does it know it?
by andsoitis 3mo ago
> running the task through each model and then picking the cheapest correct option
What system knows what the correct option is and how does it know it?
- crazylogger 3mo agoCertainly sounds like a "P vs NP" style conjecture that shouldn't be possible in practice, save for certain generalizations, such as "this is a cybersecurity task, we know Fable will refuse (and score zero), so we just route to K3".
- beng-nl 3mo agoI agree: it feels to me that judging this implies knowing whether there is a solution at all (or a solution available per model, example: whether either model will answer it given known guardrails), which is as powerful as answering the question in the first place (the router can answer the decision problem, which can polynomially be transformed into getting a specific solution). It reminds me of someone I met at a poster session who had an incredible project: his ai could include a confidence score with its answer, and he had evidence that x% confident answers were in fact correct x% of the time. That also gave me the gut feeling of that being impossible (in the general case) as it implies more powerful capability than the ai answering the question, in an oracle like way.
- nickysielicki 3mo agoP vs NP?! What?
- asimpletune 3mo ago[dead]