4 ms·

An armchair effort to redefine the goalposts and judge NLP, but proof is in the pudding. For now, large language models are the best flavor. NLP models are alre

An armchair effort to redefine the goalposts and judge NLP, but proof is in the pudding. For now, large language models are the best flavor. NLP models are already useful even in this early stage.

ampdepolymerase 5y ago
I look forward to when something like Siri or Google Assistant is hooked up to GPT-3. The current voice assistant ML systems are useless for anything but the most basic of tasks.
emodendroket 5y ago
They are certainly useful, but it seems quite plausible that they could continue being useful without ever "solving" the problem, in the general sense.
criticaltinker 5y ago
> An armchair effort Given the authors credentials and publication history [1] it's a bit disingenuous to call this an 'armchair effort'. [1] https://scholar.google.com/citations?user=i5sEc1YAAAAJ&hl=en&oi=ao https://scholar.google.com/citations?user=i5sEc1YAAAAJ&hl=en...
- ppod 5y ago
  That's not a particularly impressive publication record.
- mnky9800n 5y ago
  My h index is twice as high and that's a terrible h index still lol
  - axiom92 5y ago
    This person's h-index is 10x yours: https://scholar.google.com/citations?user=OHbcFkcAAAAJ&hl=en&oi=ao https://scholar.google.com/citations?user=OHbcFkcAAAAJ&hl=en...! Source: https://www.nature.com/articles/d41586-019-02479-7 https://www.nature.com/articles/d41586-019-02479-7
    - mnky9800n 5y ago
      Ahaha. I should try harder.
- axiom92 5y ago
  In case you were highlighting the person's h-index or citation count, there are a few shortcomings that these metrics suffer from [1], making it very hard to judge scientific impact. EDIT: Specifically, self-citation is the biggest issue, though often there are very good reasons to do it. For example, if you are working on a new area there might not be a lot of work other than yours. However, mindless self-citation grows the number of citation at O(N), as your n^th paper cites the (n - 1) papers before it. The total citation count grows by O(N^2)). [1] https://www.nature.com/articles/d41586-019-02479-7 https://www.nature.com/articles/d41586-019-02479-7
- visarga 5y ago
  OK, I took a look, no evidence of neural nets, just semantic stuff which is previous gen AI and the citation count is not impressive.
mcswell 5y ago
I don't think "Is NLP useful?" is the question they're trying to address. Hammers are useful, even in a world where there are lots of screws. NLP is useful, even in a world where there is lots of language (in English, even) that current NLP systems don't do well on.

An armchair effort to redefine the goalposts and judge NLP, but proof is in the pudding. For now, large language models are the best flavor. NLP models are already useful even in this early stage.

ampdepolymerase 5y ago

I look forward to when something like Siri or Google Assistant is hooked up to GPT-3. The current voice assistant ML systems are useless for anything but the most basic of tasks.

emodendroket 5y ago

They are certainly useful, but it seems quite plausible that they could continue being useful without ever "solving" the problem, in the general sense.

criticaltinker 5y ago

> An armchair effort Given the authors credentials and publication history [1] it's a bit disingenuous to call this an 'armchair effort'. [1] https://scholar.google.com/citations?user=i5sEc1YAAAAJ&hl=en&oi=ao https://scholar.google.com/citations?user=i5sEc1YAAAAJ&hl=en...

ppod 5y ago
That's not a particularly impressive publication record.
mnky9800n 5y ago
My h index is twice as high and that's a terrible h index still lol
- axiom92 5y ago
  This person's h-index is 10x yours: https://scholar.google.com/citations?user=OHbcFkcAAAAJ&hl=en&oi=ao https://scholar.google.com/citations?user=OHbcFkcAAAAJ&hl=en...! Source: https://www.nature.com/articles/d41586-019-02479-7 https://www.nature.com/articles/d41586-019-02479-7
  - mnky9800n 5y ago
    Ahaha. I should try harder.
axiom92 5y ago
In case you were highlighting the person's h-index or citation count, there are a few shortcomings that these metrics suffer from [1], making it very hard to judge scientific impact. EDIT: Specifically, self-citation is the biggest issue, though often there are very good reasons to do it. For example, if you are working on a new area there might not be a lot of work other than yours. However, mindless self-citation grows the number of citation at O(N), as your n^th paper cites the (n - 1) papers before it. The total citation count grows by O(N^2)). [1] https://www.nature.com/articles/d41586-019-02479-7 https://www.nature.com/articles/d41586-019-02479-7
visarga 5y ago
OK, I took a look, no evidence of neural nets, just semantic stuff which is previous gen AI and the citation count is not impressive.

mcswell 5y ago

I don't think "Is NLP useful?" is the question they're trying to address. Hammers are useful, even in a world where there are lots of screws. NLP is useful, even in a world where there is lots of language (in English, even) that current NLP systems don't do well on.