4 ms·
> _"they obviously are not smart enough to import tensorflow as tf"_ Yeah. This is part of why DistilBERT (and the fact that you can do pretty well without BER
by s_Hogg 7y ago
> _"they obviously are not smart enough to import tensorflow as tf"_
Yeah. This is part of why DistilBERT (and the fact that you can do pretty well without BERT) is interesting, to me. It seems like for a very long time there have been people complaining about certain individuals and orgs throwing cash/compute at problems to look good rather than solve anything. The difference is nowadays it's starting to be less of a fringe view, thank heavens.
The most interesting thing about NLP (as someone who works in it), is precisely that it is very, very hard to get anywhere. And that in turn is why the field keeps turning up so many new NN designs: the flipside the author rightly points out is that this has to happen for data as well, if we aren't to fool ourselves about our progress.
Great read.
- inimino 7y agoAgreed that understanding lags behind when money and cheaper computation (god I hate that term "compute" used as a verb, learn to English my dudes) drives the SOTA rather than actual understanding (on both researchers' and computers' part). There's a reason we haven't seen a hard takeoff with BERT writing BERT+1 in even better Python and it probably isn't because nobody is trying, it's because BERT is too stupid to write a computer program.
- dlkf 7y ago> The difference is nowadays it's starting to be less of a fringe view, thank heavens. Well put. The timing is good too, because NLU-is-just-around-the-corner hype is starting to have some really negative social consequences. Yesterday a disturbing article about automated test scoring in the states was trending on HN.
- lonelappde 7y agoMass production essay grading was a disaster before AI showed up. AI is just automating the mistakes of the past.
- grandmczeb 7y agoAutomatic essay grading has been around since the early 2000s. It has literally nothing to do with the current NLP hype.