4 ms·

I am getting tired of takes like this. LLMs are parrots, but they parrot human language which seems to encode our reasoning abilities into its structure. So if

I am getting tired of takes like this. LLMs are parrots, but they parrot human language which seems to encode our reasoning abilities into its structure. So if you can mimic language well enough, you start to mimic reasoning abilities. I feel like we should have all moved on from the fact that LLMs don't have a very precise memory? This is (not entirely, but mostly) fixed by including the information you want to query in the prompt itself, like Bing does. ChatGPT is much better at summarization than precise fact recall. And most of the time, it doesn't matter. When I'm using ChatGPT to learn or troubleshoot something, it's a jumping off point, not the end of the story. As long as everyone understands that, there's no problem. I have yet to see anyone who treats its output as 100% fact, and no AI company claims it either.

latexr 3y ago
> As long as everyone understands that, there's no problem. But everyone doesn’t understand that, so there is a problem. > I have yet to see anyone who treats its output as 100% fact Yet they exist. https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-s...
- garbanz0 3y ago
  Everyone as in every human being on earth, or everyone as in most reasonable people? I'm not convinced at all. ChatGPT tells you it can lie before you can even ask it anything.
  - latexr 3y ago
    > as in every human being on earth Not every human being on Earth knows what ChatGPT is, so that’s a non-starter. > or everyone as in most reasonable people That qualifier is doing some heavy lifting. It’s so vague it can be used to excuse anything¹. Not only reasonable people (however we’re going to define that) have access to the tool. > ChatGPT tells you it can lie before you can even ask it anything. And what you learn fast by doing user support of any tech tool is that most users don’t read instructions even if it’s shoved in their faces. I personally know people that think ChatGPT is the source of truth, despite that disclaimer. ¹ https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman
fudged71 3y ago
Right. People keep treating ChatGPT like Google Search, expecting to OUTPUT facts. In reality it’s actually better as a Calculator For Words. In the same way that with a calculator, if you put bullshit in you will get bullshit out. In many cases you need to INPUT facts. This is automated with things like RAG. I think this is why ChatGPT is extending deeply into RAG, because it matches more closely how users are expecting the tool to work.
- garbanz0 3y ago
  I think some people approach this looking for ways the reality of ChatGPT is inconsistent with the hype. But if you just look at it plainly as-is and compare it to where we were 5 years ago, it's pretty amazing, lies and hallucinations and all.
macleginn 3y ago
> So if you can mimic language well enough, you start to mimic reasoning abilities. Mimicking reasoning abilities is not the same thing as actually having/developing them.
- garbanz0 3y ago
  It's a whole lot better than nothing.
- flyval 3y ago
  Huh? Can you explain the difference? That’s like saying “Oh, you didn’t get the right answer to that math question, you just mimicked giving me the right answer”. Yeah, but if I gave you the right answer, then I gave you the right answer, and I must’ve gotten it somehow.
  - macleginn 3y ago
    One common mimicking approach in this situation is to remember a (very) large number of math questions, such that the vast majority of new questions will be from this set of very similar to its elements. However, given a slightly non-trivial math question, albeit an easy one, a mimicking system will fail. The system will still be (kinda) practically useful but not trustworthy.

latexr 3y ago

> As long as everyone understands that, there's no problem. But everyone doesn’t understand that, so there is a problem. > I have yet to see anyone who treats its output as 100% fact Yet they exist. https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-sanctions.html https://www.nytimes.com/2023/06/08/nyregion/lawyer-chatgpt-s...

garbanz0 3y ago
Everyone as in every human being on earth, or everyone as in most reasonable people? I'm not convinced at all. ChatGPT tells you it can lie before you can even ask it anything.
- latexr 3y ago
  > as in every human being on earth Not every human being on Earth knows what ChatGPT is, so that’s a non-starter. > or everyone as in most reasonable people That qualifier is doing some heavy lifting. It’s so vague it can be used to excuse anything¹. Not only reasonable people (however we’re going to define that) have access to the tool. > ChatGPT tells you it can lie before you can even ask it anything. And what you learn fast by doing user support of any tech tool is that most users don’t read instructions even if it’s shoved in their faces. I personally know people that think ChatGPT is the source of truth, despite that disclaimer. ¹ https://en.wikipedia.org/wiki/No_true_Scotsman https://en.wikipedia.org/wiki/No_true_Scotsman

fudged71 3y ago

Right. People keep treating ChatGPT like Google Search, expecting to OUTPUT facts. In reality it’s actually better as a Calculator For Words. In the same way that with a calculator, if you put bullshit in you will get bullshit out. In many cases you need to INPUT facts. This is automated with things like RAG. I think this is why ChatGPT is extending deeply into RAG, because it matches more closely how users are expecting the tool to work.

garbanz0 3y ago
I think some people approach this looking for ways the reality of ChatGPT is inconsistent with the hype. But if you just look at it plainly as-is and compare it to where we were 5 years ago, it's pretty amazing, lies and hallucinations and all.

macleginn 3y ago

> So if you can mimic language well enough, you start to mimic reasoning abilities. Mimicking reasoning abilities is not the same thing as actually having/developing them.

garbanz0 3y ago
It's a whole lot better than nothing.
flyval 3y ago
Huh? Can you explain the difference? That’s like saying “Oh, you didn’t get the right answer to that math question, you just mimicked giving me the right answer”. Yeah, but if I gave you the right answer, then I gave you the right answer, and I must’ve gotten it somehow.
- macleginn 3y ago
  One common mimicking approach in this situation is to remember a (very) large number of math questions, such that the vast majority of new questions will be from this set of very similar to its elements. However, given a slightly non-trivial math question, albeit an easy one, a mimicking system will fail. The system will still be (kinda) practically useful but not trustworthy.