3 ms·

> If you ask an LLM about something that isn't very written about, it'll make up nonsense, substituting things that are nearby (in some sense). So what a human

> If you ask an LLM about something that isn't very written about, it'll make up nonsense, substituting things that are nearby (in some sense). So what a human might do if they wanted to sound smart but had no idea about the something.

taneq 3y ago
This is the most disturbing thing about LLMs, imo. Not that "LLMs aren't intelligent, they just say something plausible and if they don't know what they're talking about they'll fluently bs about it" but that "when forced to respond to a question out of their depth humans sound exactly like LLMs... oh no humans aren't intelligent" and in fact we really don't have a solid definition of intelligence beyond "yeah but did it work though?" The only way to distinguish genius and madness, the old saying goes, is in the results.
ben_w 3y ago
When ChatGPT was new, some people did describe it as "Mansplaining as a service". much as I like AI, I can see why.
- seanmcdirmid 3y ago
  It’s great if you understand what it is doing, it is really just search 2.0, but it’s a huge leap from search 1.0. It can’t do original research. But most humans aren’t doing original research either.
  - ben_w 3y ago
    I'm old enough to remember actual (web) search 1.0, e.g. AltaVista. LLMs today are a bit like Wikipedia circa 2004: simultaneously fantastic and yet also flawed and used for disinformation campaigns… and surprisingly bad at advanced mathematics.
    - seanmcdirmid 3y ago
      I remember visiting DEC/Compaq WRL (which originated Alta vista) as a new grad student, and seeing my internship mentor at DEC SRC quit after my internship and go to a new search engine startup in Palo Alto that wasn’t Alta visa (it worked out really well for him). But ya, wiki was magic when it came out, and it got better over time, but still not perfect. This is similar, it is going to have an impact even if it has lots of flaws.
dkjaudyeqooe 3y ago
That's a human being actually quite intelligent. The LLM on the other hand has no higher order intelligence and the ability to create nonsense with intent, knowing it's nonsense, it merely does what it's programmed to do, ie produce nonsense sometimes. But more generally, listing any human behavior and its similarity doesn't prove LLM intelligence.
- seanmcdirmid 3y ago
  It doesn’t. But it might be impossible for us to tell in general that a human wrote something rather than an LLM. The Turing test is toast.

taneq 3y ago

This is the most disturbing thing about LLMs, imo. Not that "LLMs aren't intelligent, they just say something plausible and if they don't know what they're talking about they'll fluently bs about it" but that "when forced to respond to a question out of their depth humans sound exactly like LLMs... oh no humans aren't intelligent" and in fact we really don't have a solid definition of intelligence beyond "yeah but did it work though?" The only way to distinguish genius and madness, the old saying goes, is in the results.

ben_w 3y ago

When ChatGPT was new, some people did describe it as "Mansplaining as a service". much as I like AI, I can see why.

seanmcdirmid 3y ago
It’s great if you understand what it is doing, it is really just search 2.0, but it’s a huge leap from search 1.0. It can’t do original research. But most humans aren’t doing original research either.
- ben_w 3y ago
  I'm old enough to remember actual (web) search 1.0, e.g. AltaVista. LLMs today are a bit like Wikipedia circa 2004: simultaneously fantastic and yet also flawed and used for disinformation campaigns… and surprisingly bad at advanced mathematics.
  - seanmcdirmid 3y ago
    I remember visiting DEC/Compaq WRL (which originated Alta vista) as a new grad student, and seeing my internship mentor at DEC SRC quit after my internship and go to a new search engine startup in Palo Alto that wasn’t Alta visa (it worked out really well for him). But ya, wiki was magic when it came out, and it got better over time, but still not perfect. This is similar, it is going to have an impact even if it has lots of flaws.

dkjaudyeqooe 3y ago

That's a human being actually quite intelligent. The LLM on the other hand has no higher order intelligence and the ability to create nonsense with intent, knowing it's nonsense, it merely does what it's programmed to do, ie produce nonsense sometimes. But more generally, listing any human behavior and its similarity doesn't prove LLM intelligence.

seanmcdirmid 3y ago
It doesn’t. But it might be impossible for us to tell in general that a human wrote something rather than an LLM. The Turing test is toast.