3 ms·

Any NN "trained on" data sampled from an abstract complete outcome space (eg., a game with formal rules; mathematical sequences, etc) can often represent that s

Any NN "trained on" data sampled from an abstract complete outcome space (eg., a game with formal rules; mathematical sequences, etc) can often represent that space completely. It comes down to whether you can form conditional probability models of the rules, and that's usually possible because that's what abstract rules are. > I have one kjsdhlisrnj and I add another kjsdhlisrnj, tell me how many kjsdhlisrnj I now have. 1. P(number-word|tell me how many...) > P(other-kinds-of-words|tell me how many...) 2. P(two|I have one ... I add another ...) > P(one|...) > P(three|...) > others This is trivial.

IanCal 3y ago
Right, learning more abstract rules about how things work is the goal and where the value comes in. Not all algorithms are able to do this, even if they can do what you describe in your first comment. That's why they're interesting, othellogpt is interesting because it builds a world model.
- mjburgess 3y ago
  It builds a model of a "world" whose structure is conditional probabilities, this is circular. It's like saying you can use a lego model to build a model of another lego model. All the papers which "show" NNs building "world" models arent using any world. It's lego modelling lego. The lack of a world model only matters when the data NNs are trained on aren't valid measures of the world that data is taken to model. All the moves of a chess game are a complete model of chess. All the books ever written aren't a model of, well, anything -- the structure of the universe isnt the structure of text tokens. The only reason all statistical algorithms, including NNs, appear to model the actual world is because patterns in data give this appearance: P(The Sun is Hot) > P(The Sun is Cold) -- there is no model of the sun here. The reason P("The Sun is Hot") seems to model the sun, is because we can read the english words "sun" and "hot" -- it is we who think the machine which generates this text does so semantically.. but the people who wrote that phrase in the dataset did so; the machine is just generating "hot" because of that dataset.
  - IanCal 3y ago
    Othellogpt is fed only moves and builds a model of the current board state in its activations. It never sees a board. > It's like saying you can use a lego model to build a model of another lego model. No it's like using a description of piece placements and having a picture in mind about what the current model looks like.
    - mjburgess 3y ago
      The "board" is abstract. Any game of this sort is defined by a series of conditional probabilities: {P(Pawn_on_sqare_blah|previous_moves) ... etc.} What all statistical learning algorithms model is sets of conditional probabilities. So any stat alg is a model of a set of these rules... that's the "clay" of these models. The problem is the physical world isn't anything like this. The reason I say, "I liked that TV show" is because I had a series of mental states caused by the TV show over time (and so on). This isnt representable as a set of conditional probs in the same way. You could imagine, at the end of history, there being a total set of all possible conditional probabilities: P(I liked show|my_mental_states, time, person, location, etc.) -- this would be uncomputable, but it could be supposed. If you had that dataset then yes, NNs would learn the entire structure of the world, because that's the dataset. The problem is that the world cannot be represented in this fashion, not that NNs could model it if it could be. A decision tree could. P(I liked the TV show) doesnt follow from any dataset ever collected. It follows from my mental states. So no NN can ever model it. They can model frequency associations of these phrases in historical text documents: this isnt a model of hte world
      - IanCal 3y ago
        > Any game of this sort is defined by a series of conditional probabilities: {P(Pawn_on_sqare_blah|previous_moves) ... etc.} That would always be 1 or 0, but also that data is not fed into othellogpt. That is not the dataset. It is not fed in board states at all. It learns it, but it is not the dataset.
        mjburgess 3y ago

IanCal 3y ago

Right, learning more abstract rules about how things work is the goal and where the value comes in. Not all algorithms are able to do this, even if they can do what you describe in your first comment. That's why they're interesting, othellogpt is interesting because it builds a world model.

mjburgess 3y ago
It builds a model of a "world" whose structure is conditional probabilities, this is circular. It's like saying you can use a lego model to build a model of another lego model. All the papers which "show" NNs building "world" models arent using any world. It's lego modelling lego. The lack of a world model only matters when the data NNs are trained on aren't valid measures of the world that data is taken to model. All the moves of a chess game are a complete model of chess. All the books ever written aren't a model of, well, anything -- the structure of the universe isnt the structure of text tokens. The only reason all statistical algorithms, including NNs, appear to model the actual world is because patterns in data give this appearance: P(The Sun is Hot) > P(The Sun is Cold) -- there is no model of the sun here. The reason P("The Sun is Hot") seems to model the sun, is because we can read the english words "sun" and "hot" -- it is we who think the machine which generates this text does so semantically.. but the people who wrote that phrase in the dataset did so; the machine is just generating "hot" because of that dataset.
- IanCal 3y ago
  Othellogpt is fed only moves and builds a model of the current board state in its activations. It never sees a board. > It's like saying you can use a lego model to build a model of another lego model. No it's like using a description of piece placements and having a picture in mind about what the current model looks like.
  - mjburgess 3y ago
    The "board" is abstract. Any game of this sort is defined by a series of conditional probabilities: {P(Pawn_on_sqare_blah|previous_moves) ... etc.} What all statistical learning algorithms model is sets of conditional probabilities. So any stat alg is a model of a set of these rules... that's the "clay" of these models. The problem is the physical world isn't anything like this. The reason I say, "I liked that TV show" is because I had a series of mental states caused by the TV show over time (and so on). This isnt representable as a set of conditional probs in the same way. You could imagine, at the end of history, there being a total set of all possible conditional probabilities: P(I liked show|my_mental_states, time, person, location, etc.) -- this would be uncomputable, but it could be supposed. If you had that dataset then yes, NNs would learn the entire structure of the world, because that's the dataset. The problem is that the world cannot be represented in this fashion, not that NNs could model it if it could be. A decision tree could. P(I liked the TV show) doesnt follow from any dataset ever collected. It follows from my mental states. So no NN can ever model it. They can model frequency associations of these phrases in historical text documents: this isnt a model of hte world
    - IanCal 3y ago
      > Any game of this sort is defined by a series of conditional probabilities: {P(Pawn_on_sqare_blah|previous_moves) ... etc.} That would always be 1 or 0, but also that data is not fed into othellogpt. That is not the dataset. It is not fed in board states at all. It learns it, but it is not the dataset.
      - mjburgess 3y ago