3 ms·
This + some benchmarks are shitty thus rational model should be allowed to not answer them but ask claryfying questions.
by machiaweliczny 3y ago
This + some benchmarks are shitty thus rational model should be allowed to not answer them but ask claryfying questions.
- belval 3y agoYes, a lot of those have pretty egregious annotation mistakes. Once you get in high percentage it's often worth going through your dataset with your model prediction and compare. Obviously you can't do that on academic benchmarks (though some papers still do).