5 ms·
This is cool but would be infinitely more valuable if it could explain _why_ GPT4 is good or bad at each task, to help build intuition.
by Ozzie_osman 3y ago
This is cool but would be infinitely more valuable if it could explain _why_ GPT4 is good or bad at each task, to help build intuition.
- mgl 3y agoAsking „why” may be a little bit too much with regards to a non-deterministic token prediction machine trained on an unknown set of strings. Just saying.
- sebzim4500 3y agoWould be nice, but no one can really do that at the moment. There are specific tasks (especially character level ones) which are hard due to the tokenizer, but even that isn't all that convincing since there are plenty of character level tasks which GPT-4 can do pretty well. If you use it a lot you build an intuition for what kinds of tasks it will do well on, but it's not exactly rigorous.
- capableweb 3y ago> Would be nice, but no one can really do that at the moment. Why not? Someone could definitely build up database on why GPT is bad at some things and good at others. There is already good explanations for why it's terrible at math, why it doesn't handle single characters/numbers well and so on.