4 ms·
> Most of the time it seems trivial to beat the old state of the art, just use BERT or XLnet on a task where nobody applied it before If it was a high-return f
by inimino 7y ago
> Most of the time it seems trivial to beat the old state of the art, just use BERT or XLnet on a task where nobody applied it before
If it was a high-return fruit somebody would be doing it. Not necessarily publishing papers about it or trying to beat useless artificial benchmarks on it.
- The_rationalist 7y agoIf it was a high-return fruit somebody would be doing it." Not necessarily. Not necessarily publishing papers about it Yes. trying to beat useless artificial benchmarks on it. Wtf is this bullshit? AI benchmarcks are what direct progress in AI and allow to quantify it. And they are less and less artificial and more and more real world: E.g quora, reddit, Wikipedia and Facebook datasets.
- inimino 7y ago>"If it was a high-return fruit somebody would be doing it." Not necessarily. With high likelihood given current funding of ML with $$$ applications but yes, not necessarily. > AI benchmarcks are what direct progress in AI Sadly this is largely true. The AI benchmarks + culture around it are the bullshit. What actually moves forward the field of AI is: - accessible - reproducible - comprehensible results done with some thought and reasoning which is explained well, published well, and justified by more than some #$!& "our F1 score went up by 2 therefore our approach makes sense" bullshit. AI benchmarks have done as much to retard progress in AI as they have to promote it. Current AI benchmark top scores are gamification for big companies to waste even more resources running algorithms they can't explain. They are not machine learning, they are machine pissing contests.