4 ms·
There’s something odd about their Track 1 results: Flan-base gets an alright score, but Flan-large gets 0.0. That seems very fishy as it sounds this is pretty
by jerpint 3y ago
There’s something odd about their Track 1 results:
Flan-base gets an alright score, but Flan-large gets 0.0. That seems very fishy as it sounds this is pretty much a multiple choice kind of task. I would not expect a score less than 1/N choices for models with no physical reasoning
- dragonwriter 3y agoJust because the question is multiple choice doesn't mean an LLM without some imposed constraint (grammars, etc.) will respond in a way which respects the multiple choice nature of the question. Going of on a tangent in response would seem to be non-agreement.