4 ms·

Just because a move has never been played, doesn't make it illegal, only untried. Only when the ruleset itself is laid out as overarching the data (as in AlphaG

Just because a move has never been played, doesn't make it illegal, only untried. Only when the ruleset itself is laid out as overarching the data (as in AlphaGo/Zero) should they always be respected. Things that are observed to be true are called (in math/science) "laws" aka rules of thumb. Actual rules have other names. Most people have a world-view/mental-model of how money works. Few actually know the rules and what are possible things under the extreme conditions. Edit: in the case of AlphaGo/Zero the interesting thing wasn't the rules. They were hard-coded. The interesting thing is that it likely did form mental/world-models in higher layers above the rules in the form of tactics and strategies, these are the valuable parts.

nickelpro 4y ago
If the only understanding of the rules is what has been tried/untried, that is definitionally a statistical model. A valid rules model does not require to see all possible rules be played to understand the legality of a move. So the LLM under study here could be at most, an invalid rules model. But an invalid rules model is indistinguishable from a statistical model, and the authors give no evidence for why we should believe it to be the former instead of the latter.
- pyinstallwoes 4y ago
  So, quantum mechanics?
- karmakaze 4y ago
  An interesting comparison is between AlphaZero and MuZero[0], the latter having not been given the rules and has improved upon AlphaZero in Go. > "MuZero really is discovering for itself how to build a model and understand it just from first principles." — David Silver, DeepMind So yes to both: statistical and a world model. [0] https://en.wikipedia.org/wiki/MuZero https://en.wikipedia.org/wiki/MuZero
crosen99 4y ago
> Just because a move has never been played, doesn't make it illegal, only untried. Well, I think this is nickelpro's point. Namely that the model "does not have a systemic understanding of the rules, only a statistical inference of them." However, I would still counter nickelpro's point with the idea that what we call systemic understanding is fallible - and most likely statistical in nature. Another response below gives the example of a human understanding the rules of chess but still making incorrect moves. I might even say that any system truly capable of understanding is inherently fallible. A digital calculator or conventional database is impressive in its accuracy and consistency precisely because the rigor within its programming operates beyond the realm of understanding.
- nickelpro 4y ago
  I think the argument that human knowledge is itself statistical is far more interesting than the claim that neural nets have accurate models of the systems that feed their training data; but neither claim has much to do with the other.
  - crosen99 4y ago
    I'm not sure why you say these claims are unrelated. The article claims to have found the existence of concepts embedded in the trained parameters of a model. (You called this having a "systemic understanding of the rules.") You are saying that for this to be so, the model would have to perform with 100% accuracy. The general form of this argument is that for a system of intelligence to be endowed with a conceptual understanding of a ruleset, it would have to implement that ruleset with 100% accuracy, right? Well, given that human's don't implement rulesets with 100% accuracy, we'd then need to concede that either 100% consistency is not a requirement for understanding or else human's do not operate with understanding.
    - nickelpro 4y ago
      Exactly, the claim that humans do not operate with perfect understanding is far more interesting (and IMHO, likely) than the idea that LLMs are spontaneously producing accurate models of the systems used to generate their data. And again, the claims are unrelated. There's no reason to believe that humans and LLMs operate under similar principles, and a ton of reasons to believe that they absolutely do not.
deleted 4y ago
[deleted]

nickelpro 4y ago

If the only understanding of the rules is what has been tried/untried, that is definitionally a statistical model. A valid rules model does not require to see all possible rules be played to understand the legality of a move. So the LLM under study here could be at most, an invalid rules model. But an invalid rules model is indistinguishable from a statistical model, and the authors give no evidence for why we should believe it to be the former instead of the latter.

pyinstallwoes 4y ago
So, quantum mechanics?
karmakaze 4y ago
An interesting comparison is between AlphaZero and MuZero[0], the latter having not been given the rules and has improved upon AlphaZero in Go. > "MuZero really is discovering for itself how to build a model and understand it just from first principles." — David Silver, DeepMind So yes to both: statistical and a world model. [0] https://en.wikipedia.org/wiki/MuZero https://en.wikipedia.org/wiki/MuZero

crosen99 4y ago

> Just because a move has never been played, doesn't make it illegal, only untried. Well, I think this is nickelpro's point. Namely that the model "does not have a systemic understanding of the rules, only a statistical inference of them." However, I would still counter nickelpro's point with the idea that what we call systemic understanding is fallible - and most likely statistical in nature. Another response below gives the example of a human understanding the rules of chess but still making incorrect moves. I might even say that any system truly capable of understanding is inherently fallible. A digital calculator or conventional database is impressive in its accuracy and consistency precisely because the rigor within its programming operates beyond the realm of understanding.

nickelpro 4y ago
I think the argument that human knowledge is itself statistical is far more interesting than the claim that neural nets have accurate models of the systems that feed their training data; but neither claim has much to do with the other.
- crosen99 4y ago
  I'm not sure why you say these claims are unrelated. The article claims to have found the existence of concepts embedded in the trained parameters of a model. (You called this having a "systemic understanding of the rules.") You are saying that for this to be so, the model would have to perform with 100% accuracy. The general form of this argument is that for a system of intelligence to be endowed with a conceptual understanding of a ruleset, it would have to implement that ruleset with 100% accuracy, right? Well, given that human's don't implement rulesets with 100% accuracy, we'd then need to concede that either 100% consistency is not a requirement for understanding or else human's do not operate with understanding.
  - nickelpro 4y ago
    Exactly, the claim that humans do not operate with perfect understanding is far more interesting (and IMHO, likely) than the idea that LLMs are spontaneously producing accurate models of the systems used to generate their data. And again, the claims are unrelated. There's no reason to believe that humans and LLMs operate under similar principles, and a ton of reasons to believe that they absolutely do not.

deleted 4y ago

[deleted]