3 ms·

We're kind of splitting straws here. Whether GPT understands some of the rules of chess or whether it's completely relying on modulation between states, or whet

We're kind of splitting straws here. Whether GPT understands some of the rules of chess or whether it's completely relying on modulation between states, or whether it's using some other strategy -- it seems to be pretty obviously approaching chess using a different strategy than a human being would. And I think that's the main point that janalsncm was getting at -- GPT thinks about the world and thinks about predictions and thinks about subjects like chess differently than a human would. It does not appear to "understand" chess in the same way that a human does when approaching the game. Of course, you can have different definitions of understanding, GPT might be looking at chess more as a series of patterns or thinking about the shape of a game in a way that a human might not. But the value in understanding the distinction is that GPT is going to be good at some things that a human is bad at, and bad at some things that a human is good at, and understanding that it approaches tasks differently than humans is key to understanding why GPT struggles or succeeds at certain tasks. I'm not going to run around defending other people's comments but for whatever it is worth, I don't think janalsncm's comment was unreasonable or lacked nuance. It didn't go out of its way to clarify that GPT isn't useless, it didn't go out of its way to say that it's still impressive that a language model can get the performance that it does get, but the basic idea of "hey, this is acting different than a human and maybe it shouldn't be directly compared to a human, and the deviations in its behavior should tell us something about the model" is sound. It's not just sound, it's also useful advice for people to internalize. On a purely practical level, you will get much better results out of GPT if you modify your prompts and approaches to cater to its strengths rather than its weaknesses. To do that, you have to understand what the weaknesses and strengths are and you have to understand that imitation of human behavior does not necessarily imply that GPT is approaching or understanding those tasks in remotely the same way that a human being would. It is useful information for prompting GPT and you will get better results out of GPT if you understand that it's very often not approaching tasks from first principles. And chess is frankly a really good example of that, it's a great example of a task where the end result initially suggests that humans and LLMs are approaching a task identically, and then upon further examination reveals that in all likelyhood, the model does not have the same first-principles or understanding of the rulesets that a human player does. It's striking because for a human to get to the same level of play they would need to have a fundamental understanding of the rulesets, and GPT is able to get to that level of play seemingly without building that level of understanding. That is an interesting thing to learn about GPT.

famouswaffles 3y ago
I don't know how relevant this argument even is anymore considering I was wrong about the move rate. Is 1 in ~1650 really underperforming a 1800 Elo Human ? How much so ? https://github.com/adamkarvonen/chess_gpt_eval https://github.com/adamkarvonen/chess_gpt_eval That said, >if you understand that it's very often not approaching tasks from first principles You don't how humans play chess ie what the brain computes. You don't know what "first principles" actually are for intelligence or reasoning or whatever. If "first principles" were what we thought they were, we wouldn't be using GPT or artificial neural networks for nearly all computer intelligence tasks. We would be using symbolic systems. Unfortunately the "first principles" systems we labored for decades to build mostly failed and we abandoned them once NNs had the compute to work. Either we have no clue what these "first principles" are or that intelligence cannot generally be delineated as a system of "first principles"(ie intelligence is not actually logically tractable). My bet is both. Intelligence is not logically tractable and because we have no idea how intelligence works, we foolishly assumed so Either way, Humans or any other animals we've observed are certainly not first principling their way out of chess or really anything.
- danShumway 3y ago
  > I don't know how relevant this argument even is anymore considering I was wrong about the move rate. Is 1 in ~1650 really underperforming a 1800 Elo Human ? How much so ? Yes, I would still be surprised if an 1800 elo human makes 1/1650 illegal moves during formal matches. > If "first principles" were what we thought they were, we wouldn't be using GPT or artificial neural networks for nearly all computer intelligence tasks. We would be using symbolic systems. This would only be true if you believe in a singular definition of intelligence where you believe there is only one right way to get an answer. There are lots of reasons we don't train AI from first principles, one of the biggest being that neural networks are just flat-out easier to train. Training from first principles is annoying and time consuming and requires either an understanding of first principles or some simulation or methodology to consistently demonstrate those principles to the AI. That is hard to do. But first principles and predicate logic haven't failed as a way to accomplishing tasks or as a way to reason about problems. First principles are ill-suited for an extremely specific field. And now people are acting like however AIs work that has to necessarily match how humans learn? That's not the case, what I am getting at when I talk about diversity of intelligence is that you don't need to map to human reasoning in order to build tools that are useful and that generally solve certain categories of tasks, sometimes even better than humans do. It's incredibly limiting to define what kinds of intelligence do and don't exist in the world based purely on what is useful for training an AI. ---- Humans observably do not learn the same way as GPT does, we do not start out by learning language and then logic, we do the opposite. Any study of any infant will show you that humans develop logical capabilities before we develop language skills. However that process is happening, it factually, observably happens in the opposite order than what we see occurring in LLMs. We do not need to know exactly how that process works to know that it is different from how GPT learns. And we certainly don't need to look at a poorly understood process and assume that it necessarily must match whatever the current state-of-the-art methodology we have to simulate it. As you accurately say, we do not know how to define intelligence broadly -- so it's wildly reductive to say "but we do know how to build neural networks, and when we tried to replicate first principles or composite logic that didn't work well, so probably that doesn't exist" -- I just don't view that as a serious take. I still feel like you're falling into the trap of getting really invested into showing that GPT is meeting some human standard of reasoning, and I don't understand why it matters whether or not GPT is meeting that standard. Why is it a problem to say, "we don't understand how humans work, but we're seeing surprising differences between GPT's capabilities and the capabilities of comparable humans, and that suggests there is something very different going on under the hood."? Does that actually make GPT any less interesting or useful? I don't think so.

famouswaffles 3y ago

I don't know how relevant this argument even is anymore considering I was wrong about the move rate. Is 1 in ~1650 really underperforming a 1800 Elo Human ? How much so ? https://github.com/adamkarvonen/chess_gpt_eval https://github.com/adamkarvonen/chess_gpt_eval That said, >if you understand that it's very often not approaching tasks from first principles You don't how humans play chess ie what the brain computes. You don't know what "first principles" actually are for intelligence or reasoning or whatever. If "first principles" were what we thought they were, we wouldn't be using GPT or artificial neural networks for nearly all computer intelligence tasks. We would be using symbolic systems. Unfortunately the "first principles" systems we labored for decades to build mostly failed and we abandoned them once NNs had the compute to work. Either we have no clue what these "first principles" are or that intelligence cannot generally be delineated as a system of "first principles"(ie intelligence is not actually logically tractable). My bet is both. Intelligence is not logically tractable and because we have no idea how intelligence works, we foolishly assumed so Either way, Humans or any other animals we've observed are certainly not first principling their way out of chess or really anything.

danShumway 3y ago
> I don't know how relevant this argument even is anymore considering I was wrong about the move rate. Is 1 in ~1650 really underperforming a 1800 Elo Human ? How much so ? Yes, I would still be surprised if an 1800 elo human makes 1/1650 illegal moves during formal matches. > If "first principles" were what we thought they were, we wouldn't be using GPT or artificial neural networks for nearly all computer intelligence tasks. We would be using symbolic systems. This would only be true if you believe in a singular definition of intelligence where you believe there is only one right way to get an answer. There are lots of reasons we don't train AI from first principles, one of the biggest being that neural networks are just flat-out easier to train. Training from first principles is annoying and time consuming and requires either an understanding of first principles or some simulation or methodology to consistently demonstrate those principles to the AI. That is hard to do. But first principles and predicate logic haven't failed as a way to accomplishing tasks or as a way to reason about problems. First principles are ill-suited for an extremely specific field. And now people are acting like however AIs work that has to necessarily match how humans learn? That's not the case, what I am getting at when I talk about diversity of intelligence is that you don't need to map to human reasoning in order to build tools that are useful and that generally solve certain categories of tasks, sometimes even better than humans do. It's incredibly limiting to define what kinds of intelligence do and don't exist in the world based purely on what is useful for training an AI. ---- Humans observably do not learn the same way as GPT does, we do not start out by learning language and then logic, we do the opposite. Any study of any infant will show you that humans develop logical capabilities before we develop language skills. However that process is happening, it factually, observably happens in the opposite order than what we see occurring in LLMs. We do not need to know exactly how that process works to know that it is different from how GPT learns. And we certainly don't need to look at a poorly understood process and assume that it necessarily must match whatever the current state-of-the-art methodology we have to simulate it. As you accurately say, we do not know how to define intelligence broadly -- so it's wildly reductive to say "but we do know how to build neural networks, and when we tried to replicate first principles or composite logic that didn't work well, so probably that doesn't exist" -- I just don't view that as a serious take. I still feel like you're falling into the trap of getting really invested into showing that GPT is meeting some human standard of reasoning, and I don't understand why it matters whether or not GPT is meeting that standard. Why is it a problem to say, "we don't understand how humans work, but we're seeing surprising differences between GPT's capabilities and the capabilities of comparable humans, and that suggests there is something very different going on under the hood."? Does that actually make GPT any less interesting or useful? I don't think so.