4 ms·

NLP is nowhere near being solved.

• mach1ne 4y ago
  Depending on definition, it is solved.
  • deleted 4y ago
    [deleted]
  • chartpath 4y ago
    Even if that were true, LLMs don't give any kind of "handles" on the semantics. You just get what you get and have to hope it is tuned for your domain. This is 100% fine for generic consumer-facing services where the training data is representative, but for specialized and jargon-filled domains where there has to be a very opinionated interpretation of words, classical NLU is really the only ethical choice IMHO.
  • gattilorenz 4y ago
    You're using the wrong definition, then. /s Where is some evidence that NLP is 'solved'? What does it even mean? OpenAI itself acknowledges the fundamental limitations of ChatGPT and the method of training it, but apparently everybody is happily sweeping them under the rug: "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows." (from https://openai.com/blog/chatgpt https://openai.com/blog/chatgpt ) Certainly ChatGPT/GPT-4 are impressive accomplishments, and it doesn't mean they won't be useful, but we were pretty sure in the past that we had "solved" AI or that we were just about to crack it, just give it a few years... except there's always a new rabbit hole to fall into waiting for you.
    • gonzo41 4y ago
      It'd be great if GPT could provide it's sources for the text it generated. I've been asking it about lyrics from songs that I know of, but where I can't find the original artist listed. I was hoping chat gpt had consumed a stack of lyrics and I could just ask it, "What song has this chorus or one similar to X..." It didn't work. Instead it firmly stated the wrong answer. And when I gave it time ranges it just noped out of there. I think If I could ask it a question and it could go, I've used these 20-100 sources directly to synthesize this information, it'd be very helpful.
      • IanCal 4y ago
        Have you tried bing chat? That search & sourcing is exactly what it does.
        • gonzo41 4y ago
          Not yet, I'll try at work on my windows box. Thanks.
        • snowwrestler 4y ago
          Sure, but the sources list is generated by the same system that generated the text, so it’s equally subject to hallucinations. Some examples in here: https://dkb.blog/p/bing-ai-cant-be-trusted https://dkb.blog/p/bing-ai-cant-be-trusted To answer the question above, these systems cannot provide sources because they don’t work that way. Their source for everything is, basically, everything. They are trained on a huge corpus of text data and every output depends on that entire training. They have no way to distinguish or differentiate which piece of the training data was the “actual” or “true” source of what they generated. It’s like the old questions “which drop caused the flood” or “which pebble caused the landslide”.
          • IanCal 4y ago
            The issues are good to know about but > Their source for everything is, basically, everything. They are trained on a huge corpus of text data and every output depends on that entire training. Bing chat is explicitly taking in extra data. It's a distinctly different setup from chatgpt.
• nothrowaways 4y ago
  Only If you want to keep doing it the old lematization way.