4 ms·
>The problem is not even posed as "well, this data is overfitted in papers or we are solving narrower problems". I mean it does get into that: "They additiona
by Levitz 5y ago
>The problem is not even posed as "well, this data is overfitted in papers or we are solving narrower problems".
I mean it does get into that:
"They additionally note that blind adherence to this small number of ‘gold’ datasets encourages researchers to achieve results that are overfitted (i.e. that are dataset-specific and not likely to perform anywhere near as well on real-world data, on new academic or original datasets, or even necessarily on different datasets in the ‘gold standard’)."
- omarhaneef 5y agoTrue, I guess my complaint is that they don't get into the mechanisms. The sibling comment on benchmark lottery paper lays this out. But I should modify.
- deleted 5y ago[deleted]