3 ms·
Is this benchmark a good one? Any benchmark which reflects a task that humans do is a good one, unless it has specific weaknesses that a computer exploits. I'
by nl 7y ago
Is this benchmark a good one?
Any benchmark which reflects a task that humans do is a good one, unless it has specific weaknesses that a computer exploits.
I'd use models that are written for this is my work, so I find it useful.
I feel like this is true of any new benchmark.... give some smart folks a few months and they can now beat the task.
In NLP work this use not to be the case. 5 years ago we were stuck at a local maximum.
And this undervalues this task - this bridges the gap between unstructured and structured data. In many ways it is the holy grail for many tasks.