4 ms·

Here was my test at ChatGPT 3.5.[0] I made up a novel game, and it figured it out. The test is simple, but it made me doubt absolute arguments that LLMs are not

Here was my test at ChatGPT 3.5.[0] I made up a novel game, and it figured it out. The test is simple, but it made me doubt absolute arguments that LLMs are not able to reason, in some way. There is a question at the end of that comment, would love to hear other options. [0] https://news.ycombinator.com/item?id=35442147 https://news.ycombinator.com/item?id=35442147

ninetyninenine 2y ago
My thread has been voted down and it’s getting stale. The few remaining people are biased towards there point of view and are unlikely to entertain anything that will trigger a change in their established world view. Most people will use this excuse to avoid responding to or even looking at your link here. It is compelling evidence.
- cristiancavalli 2y ago
  I’d settle for these things being able to do value comparison consistently well, play a game of tic tac toe more than once correctly or use a UI after an update and not fail horrendously to move the needle a little bit for me. People claiming these things selectively reason while also not being able to explain why seems a lot like magical thinking to me rather than entertaining the possibility you might be projecting onto something that is really damn-well engineered to make your anthropomorphize it.
cristiancavalli 2y ago
How does this prove reasoning? The thread you point to has several question in it that remain unanswered that ask the same question? How is this not entirely derivative too — there’s a huge number of these kind of 3-box “games” (although I don’t see this as a game really) so something very similar to this is probably in the training data a lot. Writing code to factor a number is definitely very common. Variation of this are also very common interview questions for interns (at least when I was interviewing)
- consumer451 2y ago
  I was unable to find my exact "game" in google's index. Therefore, how does my example not qualify as this, at least: > Analogical reasoning involves the comparison of two systems in relation to their similarity. It starts from information about one system and infers information about another system based on the resemblance between the two systems. https://en.wikipedia.org/wiki/Logical_reasoning#Analogical https://en.wikipedia.org/wiki/Logical_reasoning#Analogical
  - cristiancavalli 2y ago
    Is it actually reasoning though or just pattern matching? Seems like to compare one should also “know” which your above response indicates they do not. I guess the real question is “does moving down a stochastic gradient of probabilities suffice as reasoning to you” and my awnser is no because you don’t need reason to find the nearest neighbor in this architecture. In this case the model is not actively comparing and inferring its simply associating without “knowing”
    - consumer451 2y ago
      There are many types of reasoning, and LLMs appear to do some of them.
      - cristiancavalli 2y ago
        Repeating a point without proffering evidence only makes it seems as if you don’t have anything to argue of substance.
        consumer451 2y ago
        [flagged]
        cristiancavalli 2y ago

ninetyninenine 2y ago

My thread has been voted down and it’s getting stale. The few remaining people are biased towards there point of view and are unlikely to entertain anything that will trigger a change in their established world view. Most people will use this excuse to avoid responding to or even looking at your link here. It is compelling evidence.

cristiancavalli 2y ago
I’d settle for these things being able to do value comparison consistently well, play a game of tic tac toe more than once correctly or use a UI after an update and not fail horrendously to move the needle a little bit for me. People claiming these things selectively reason while also not being able to explain why seems a lot like magical thinking to me rather than entertaining the possibility you might be projecting onto something that is really damn-well engineered to make your anthropomorphize it.

cristiancavalli 2y ago

How does this prove reasoning? The thread you point to has several question in it that remain unanswered that ask the same question? How is this not entirely derivative too — there’s a huge number of these kind of 3-box “games” (although I don’t see this as a game really) so something very similar to this is probably in the training data a lot. Writing code to factor a number is definitely very common. Variation of this are also very common interview questions for interns (at least when I was interviewing)

consumer451 2y ago
I was unable to find my exact "game" in google's index. Therefore, how does my example not qualify as this, at least: > Analogical reasoning involves the comparison of two systems in relation to their similarity. It starts from information about one system and infers information about another system based on the resemblance between the two systems. https://en.wikipedia.org/wiki/Logical_reasoning#Analogical https://en.wikipedia.org/wiki/Logical_reasoning#Analogical
- cristiancavalli 2y ago
  Is it actually reasoning though or just pattern matching? Seems like to compare one should also “know” which your above response indicates they do not. I guess the real question is “does moving down a stochastic gradient of probabilities suffice as reasoning to you” and my awnser is no because you don’t need reason to find the nearest neighbor in this architecture. In this case the model is not actively comparing and inferring its simply associating without “knowing”
  - consumer451 2y ago
    There are many types of reasoning, and LLMs appear to do some of them.
    - cristiancavalli 2y ago
      Repeating a point without proffering evidence only makes it seems as if you don’t have anything to argue of substance.
      - consumer451 2y ago
        [flagged]
        cristiancavalli 2y ago