2 ms·

Just go and ask ChatGPT or Claude something that can't possibly be in its training set. Make something up. If it is only memorising answers then it will be impo

Just go and ask ChatGPT or Claude something that can't possibly be in its training set. Make something up. If it is only memorising answers then it will be impossible for it to get the correct result. A simple nonsense programming task would suffice. For example "write a Python function to erase every character from a string unless either of its adjacent characters are also adjacent to it in the alphabet. The string only contains lowercase a-z" That task isn't anywhere in its training set so they can't memorise the answer. But I bet ChatGPT and Claude can still do it. Honestly this is sooooo obvious to anyone that has used these tools, it's really insane that people are still parroting (heh) the "it just memorises" line.

troupo 1y ago
People who say that LLMs memorize stuff are just as clueless who assume that there's any reasoning happening. They generate statistically plausible answers (to simplify the answer) based on the training set and weights they have.
- Tijdreiziger 1y ago
  What if that’s all we’re doing, though?
  - troupo 1y ago
    Most of us definitely do :) Or we do it most of the time :)
imiric 1y ago
LLMs don't "memorize" concepts like humans do. They generate output based on token patterns in their training data. So instead of having to be trained on every possible problem, they can still generate output that solves it by referencing the most probable combination of tokens for the specified input tokens. To humans this seems like they're truly solving novel problems, but it's merely a trick of statistics. These tools can reference and generate patterns that no human ever could. This is what makes them useful and powerful, but I would argue not intelligent.
- IshKebab 1y ago
  > To humans this seems like they're truly solving novel problems Because they are. This is some crazy semantic denial. I should stop engaging with this nonsense. We have AI that is kind of close to passing the Turing test and people still say it's not intelligent...
  - alternatex 1y ago
    Depending on the interviewer, you could make a non-AI program pass the Turing test. It's quite a meaningless exercise.
    - IshKebab 1y ago
      Obviously I mean for a sophisticated interviewer. Not nonsense like the Loebner prize.
      - tough 1y ago
        The Turing test is contrived to chatting via textual interface. These machines are only able to output text. It seems hard to think they could reasonably think any -normal- person. Tech only feels like magic if you don't know how it works
  - imiric 1y ago
    > Because they _are_. Not really. Most of those seemingly novel problems are permutations of existing ones, like the one you mentioned. A solution is simply a specific permutation of tokens in the training data which humans are not able to see. This doesn't mean that the permutation is something that previously didn't exist, let alone that it's something that is actually correct, but those scenarios are much rarer. None of this is to say that these tools can't be useful, but thinking that this is intelligence is delusional. > We have AI that is kind of close to passing the Turing test and people still say it's not intelligent... The Turing test was passed arguably decades ago. It's not a test of intelligence. It's an _imitation game_ where the only goal is to fool humans into thinking they're having a text conversation with another human. LLMs can do this very well.
deleted 1y ago
[deleted]

troupo 1y ago

People who say that LLMs memorize stuff are just as clueless who assume that there's any reasoning happening. They generate statistically plausible answers (to simplify the answer) based on the training set and weights they have.

Tijdreiziger 1y ago
What if that’s all we’re doing, though?
- troupo 1y ago
  Most of us definitely do :) Or we do it most of the time :)

imiric 1y ago

LLMs don't "memorize" concepts like humans do. They generate output based on token patterns in their training data. So instead of having to be trained on every possible problem, they can still generate output that solves it by referencing the most probable combination of tokens for the specified input tokens. To humans this seems like they're truly solving novel problems, but it's merely a trick of statistics. These tools can reference and generate patterns that no human ever could. This is what makes them useful and powerful, but I would argue not intelligent.

IshKebab 1y ago
> To humans this seems like they're truly solving novel problems Because they are. This is some crazy semantic denial. I should stop engaging with this nonsense. We have AI that is kind of close to passing the Turing test and people still say it's not intelligent...
- alternatex 1y ago
  Depending on the interviewer, you could make a non-AI program pass the Turing test. It's quite a meaningless exercise.
  - IshKebab 1y ago
    Obviously I mean for a sophisticated interviewer. Not nonsense like the Loebner prize.
    - tough 1y ago
      The Turing test is contrived to chatting via textual interface. These machines are only able to output text. It seems hard to think they could reasonably think any -normal- person. Tech only feels like magic if you don't know how it works
- imiric 1y ago
  > Because they _are_. Not really. Most of those seemingly novel problems are permutations of existing ones, like the one you mentioned. A solution is simply a specific permutation of tokens in the training data which humans are not able to see. This doesn't mean that the permutation is something that previously didn't exist, let alone that it's something that is actually correct, but those scenarios are much rarer. None of this is to say that these tools can't be useful, but thinking that this is intelligence is delusional. > We have AI that is kind of close to passing the Turing test and people still say it's not intelligent... The Turing test was passed arguably decades ago. It's not a test of intelligence. It's an _imitation game_ where the only goal is to fool humans into thinking they're having a text conversation with another human. LLMs can do this very well.

deleted 1y ago

[deleted]