7 ms·

> Sure. Point is that even the best players will make an illegal move (even if rarely) so a computer that makes an illegal move is not grounds for "does not und

> Sure. Point is that even the best players will make an illegal move (even if rarely) so a computer that makes an illegal move is not grounds for "does not understand chess" just because it doesn't always make a legal move. You're just making the exact same mistake a second time in a row. It is evidence; maybe not full on undeniable 100% proof, but it should shift your priors about how you think about AI, and if it doesn't I would say you're ignoring obvious statistical evidence. An analogy: sometimes cell phone batteries explode. If 1/300 cellphone batteries for a specific brand start exploding, you would not say, "that's not proof of anything, all cell phone batteries have a risk of exploding." You would say, "huh, something is probably going very wrong at the factory." Observing an abnormally high rate of something that should be much more uncommon is actually evidence that something could be going wrong with the underlying process. Observing that a computer has an abnormally high rates of illegal moves is evidence that it struggles to actually understand the rules of chess and that it might be imitating play instead of understanding it. > Yes because humans usually learn the rules first then play. That feels like you're agreeing with the original poster? The computer is imitating play that it's seen before, it's not starting from a baseline understanding of how chess works or why those moves are being made. I would say a human grandmaster who regularly makes illegal moves at a rate of 1/300 does not have as deep or complete of an understanding of the game as other grandmasters who do not make those errors at nearly the same level or frequency. I think that's what most people would say about that person.

famouswaffles 3y ago
>You're just making the exact same mistake a second time in a row. It is evidence; maybe not full on undeniable 100% proof, but it should shift your priors about how you think about AI, and if it doesn't I would say you're ignoring obvious statistical evidence. Anything can be called a mistake if you ignore crucial context. I replied to someone else addressing very specific points he/she brought up. >would say a human grandmaster who regularly makes illegal moves at a rate of 1/300 does not have as deep or complete of an understanding of the game as other grandmasters who do not make those errors at nearly the same level or frequency. I think that's what most people would say about that person. A GPT grandmaster would make even less mistakes. That aside because I understand it's not your main point, again this is more nuance than who I was replying to. You have said "does not have as complete an understanding" where the person I replied to said "does not understand the game at all". I feel like you want to argue against a point that was not present in the original discussion I was having. >That feels like you're agreeing with the original poster? The original poster is lacking the nuance to have me agree with him.
- danShumway 3y ago
  We're kind of splitting straws here. Whether GPT understands some of the rules of chess or whether it's completely relying on modulation between states, or whether it's using some other strategy -- it seems to be pretty obviously approaching chess using a different strategy than a human being would. And I think that's the main point that janalsncm was getting at -- GPT thinks about the world and thinks about predictions and thinks about subjects like chess differently than a human would. It does not appear to "understand" chess in the same way that a human does when approaching the game. Of course, you can have different definitions of understanding, GPT might be looking at chess more as a series of patterns or thinking about the shape of a game in a way that a human might not. But the value in understanding the distinction is that GPT is going to be good at some things that a human is bad at, and bad at some things that a human is good at, and understanding that it approaches tasks differently than humans is key to understanding why GPT struggles or succeeds at certain tasks. I'm not going to run around defending other people's comments but for whatever it is worth, I don't think janalsncm's comment was unreasonable or lacked nuance. It didn't go out of its way to clarify that GPT isn't useless, it didn't go out of its way to say that it's still impressive that a language model can get the performance that it does get, but the basic idea of "hey, this is acting different than a human and maybe it shouldn't be directly compared to a human, and the deviations in its behavior should tell us something about the model" is sound. It's not just sound, it's also useful advice for people to internalize. On a purely practical level, you will get much better results out of GPT if you modify your prompts and approaches to cater to its strengths rather than its weaknesses. To do that, you have to understand what the weaknesses and strengths are and you have to understand that imitation of human behavior does not necessarily imply that GPT is approaching or understanding those tasks in remotely the same way that a human being would. It is useful information for prompting GPT and you will get better results out of GPT if you understand that it's very often not approaching tasks from first principles. And chess is frankly a really good example of that, it's a great example of a task where the end result initially suggests that humans and LLMs are approaching a task identically, and then upon further examination reveals that in all likelyhood, the model does not have the same first-principles or understanding of the rulesets that a human player does. It's striking because for a human to get to the same level of play they would need to have a fundamental understanding of the rulesets, and GPT is able to get to that level of play seemingly without building that level of understanding. That is an interesting thing to learn about GPT.
famouswaffles 3y ago
I think there are also multiple dimensions of understanding. Right that it makes more illegal moves than a similarly competent human but if it can modulate its level of play better than any other human or computer (Stockfish's reduced Elo is just brain-dead moves or mistakes interspersed with brilliant ones at a certain rate, humans of course do it much better but still struggle with it overall) then I think that counts for something.
- danShumway 3y ago
  Of course it counts for something, but if I can vent another frustration, it is possible for GPT to be a useful tool for some tasks or for it to do impressive things or for it to exhibit really interesting behavior without it being comparable to a human being. I think the comparisons between GPT and humans and the common insistence I see on HN that GPT does understand the world in the same way that humans do is both reductive and unimaginative. It's reductive in the sense that it oversimplifies human behavior and reasoning, and its unimaginative in the sense that it seems to be stemming from this idea that GPT's behavior only "counts for something" if it can be compared to a human being. If GPT is playing chess at 1800 elo without understanding the rules, that is fundamentally interesting and it extremely "counts for something." It doesn't become less interesting if we say that GPT doesn't understand the rules of chess, if anything it becomes more interesting. It is extremely interesting that it appears at least at first glance like GPT might be able to get decent human-level performance at a game using a set of heuristics that may not be based on the rules of the game. That is very different from how humans approach tasks, and seeing an extremely different approach to reasoning that yields decent or at least decent-ish results is fascinating. Forcing it into a human framework and pushing the idea that GPT is understanding the game like a human -- it's honestly kind of limiting, it's closing off a lot of interesting discussion about non-human approaches to accomplishing goals. There is (in general, not saying from you specifically) an attitude on HN that pointing out that LLMs are not like humans and that they don't approach reasoning the same way and that they very often aren't reasoning about tasks in a way that maps to commonly used definitions of human reasoning -- that it means that LLMs are useless or they don't count for anything. But that's just not the case. Clearly the LLM is doing something interesting, is it so important that we claim that it understands the task the same way that a human does? Is it really a problem if we acknowledge that there is a lot of evidence that LLMs are accomplishing tasks using orthogonal approaches that don't map to human understanding? That doesn't make them any less interesting.

famouswaffles 3y ago

>You're just making the exact same mistake a second time in a row. It is evidence; maybe not full on undeniable 100% proof, but it should shift your priors about how you think about AI, and if it doesn't I would say you're ignoring obvious statistical evidence. Anything can be called a mistake if you ignore crucial context. I replied to someone else addressing very specific points he/she brought up. >would say a human grandmaster who regularly makes illegal moves at a rate of 1/300 does not have as deep or complete of an understanding of the game as other grandmasters who do not make those errors at nearly the same level or frequency. I think that's what most people would say about that person. A GPT grandmaster would make even less mistakes. That aside because I understand it's not your main point, again this is more nuance than who I was replying to. You have said "does not have as complete an understanding" where the person I replied to said "does not understand the game at all". I feel like you want to argue against a point that was not present in the original discussion I was having. >That feels like you're agreeing with the original poster? The original poster is lacking the nuance to have me agree with him.

danShumway 3y ago
We're kind of splitting straws here. Whether GPT understands some of the rules of chess or whether it's completely relying on modulation between states, or whether it's using some other strategy -- it seems to be pretty obviously approaching chess using a different strategy than a human being would. And I think that's the main point that janalsncm was getting at -- GPT thinks about the world and thinks about predictions and thinks about subjects like chess differently than a human would. It does not appear to "understand" chess in the same way that a human does when approaching the game. Of course, you can have different definitions of understanding, GPT might be looking at chess more as a series of patterns or thinking about the shape of a game in a way that a human might not. But the value in understanding the distinction is that GPT is going to be good at some things that a human is bad at, and bad at some things that a human is good at, and understanding that it approaches tasks differently than humans is key to understanding why GPT struggles or succeeds at certain tasks. I'm not going to run around defending other people's comments but for whatever it is worth, I don't think janalsncm's comment was unreasonable or lacked nuance. It didn't go out of its way to clarify that GPT isn't useless, it didn't go out of its way to say that it's still impressive that a language model can get the performance that it does get, but the basic idea of "hey, this is acting different than a human and maybe it shouldn't be directly compared to a human, and the deviations in its behavior should tell us something about the model" is sound. It's not just sound, it's also useful advice for people to internalize. On a purely practical level, you will get much better results out of GPT if you modify your prompts and approaches to cater to its strengths rather than its weaknesses. To do that, you have to understand what the weaknesses and strengths are and you have to understand that imitation of human behavior does not necessarily imply that GPT is approaching or understanding those tasks in remotely the same way that a human being would. It is useful information for prompting GPT and you will get better results out of GPT if you understand that it's very often not approaching tasks from first principles. And chess is frankly a really good example of that, it's a great example of a task where the end result initially suggests that humans and LLMs are approaching a task identically, and then upon further examination reveals that in all likelyhood, the model does not have the same first-principles or understanding of the rulesets that a human player does. It's striking because for a human to get to the same level of play they would need to have a fundamental understanding of the rulesets, and GPT is able to get to that level of play seemingly without building that level of understanding. That is an interesting thing to learn about GPT.

famouswaffles 3y ago

I think there are also multiple dimensions of understanding. Right that it makes more illegal moves than a similarly competent human but if it can modulate its level of play better than any other human or computer (Stockfish's reduced Elo is just brain-dead moves or mistakes interspersed with brilliant ones at a certain rate, humans of course do it much better but still struggle with it overall) then I think that counts for something.

danShumway 3y ago
Of course it counts for something, but if I can vent another frustration, it is possible for GPT to be a useful tool for some tasks or for it to do impressive things or for it to exhibit really interesting behavior without it being comparable to a human being. I think the comparisons between GPT and humans and the common insistence I see on HN that GPT does understand the world in the same way that humans do is both reductive and unimaginative. It's reductive in the sense that it oversimplifies human behavior and reasoning, and its unimaginative in the sense that it seems to be stemming from this idea that GPT's behavior only "counts for something" if it can be compared to a human being. If GPT is playing chess at 1800 elo without understanding the rules, that is fundamentally interesting and it extremely "counts for something." It doesn't become less interesting if we say that GPT doesn't understand the rules of chess, if anything it becomes more interesting. It is extremely interesting that it appears at least at first glance like GPT might be able to get decent human-level performance at a game using a set of heuristics that may not be based on the rules of the game. That is very different from how humans approach tasks, and seeing an extremely different approach to reasoning that yields decent or at least decent-ish results is fascinating. Forcing it into a human framework and pushing the idea that GPT is understanding the game like a human -- it's honestly kind of limiting, it's closing off a lot of interesting discussion about non-human approaches to accomplishing goals. There is (in general, not saying from you specifically) an attitude on HN that pointing out that LLMs are not like humans and that they don't approach reasoning the same way and that they very often aren't reasoning about tasks in a way that maps to commonly used definitions of human reasoning -- that it means that LLMs are useless or they don't count for anything. But that's just not the case. Clearly the LLM is doing something interesting, is it so important that we claim that it understands the task the same way that a human does? Is it really a problem if we acknowledge that there is a lot of evidence that LLMs are accomplishing tasks using orthogonal approaches that don't map to human understanding? That doesn't make them any less interesting.