5 ms·
This is obvious... but some human knowledge outside the question is needed. (Alice is female, sister is female, brother is male, relation of sisters and brothe
by ape4 2y ago
This is obvious... but some human knowledge outside the question is needed. (Alice is female, sister is female, brother is male, relation of sisters and brothers). In a school math "word problem" you don't need this kind of info.
- theptip 2y agoLLMs perform well on “human knowledge” like you are describing. They don’t lack knowledge of simple relational facts like these. Nonetheless it would be interesting to see the problem reformulated in purely mathematical terms, I suspect models would perform better. That is the point though - models are showing an inability to generalize their capabilities from one domain (maths / abstract logic) into other domains (conversational reasoning).