3 ms·

Yes, I also quoted that part from the article. This does not address that the attention Matrix does not represent all learned parameters. Even supposing that th

Yes, I also quoted that part from the article. This does not address that the attention Matrix does not represent all learned parameters. Even supposing that the form of the attention matrix guarantees the correct functioning of the algorithm why was that not used as the metric to decide convergence? "We detected that the model had converged by looking for 1024 training batches with perfect predictions, and that it could perfectly run 100 Life games for 100 steps." This would be superfluous (and even a pretty bizarre methodology) if the shape of the attention matrix was proof that the Network performed the actual game of life algorithm. Just to be clear, I am not saying that the NN isn't converging to performing some computation that would also be seen in other algorithms. I am saying that the paper does not investigate whether the resulting NN actually performs the game of life algorithm. The convolution part is certainly evidence, but I think it would have been worthwhile to look at the actual resulting Net and figure out if the trained weights together actually formed an algorithm. This is also the only way to determine the truth of the initial claim, that this isn't just a statistical model, but rather an actual algorithm.

montebicyclelo 1y ago
> the paper does not investigate whether the resulting NN actually performs the game of life algorithm How could it not be computing the game of life algorithm? Given that it gets 100% accuracy over multiple steps on a bunch random game boards it's never seen before. And then based on the structure of the net, and by examining the attention layers and finding that it's doing 3 by 3 average pooling, we can see that the attention layer produces a set of tokens, where each token contains the information of the number of neighbours it had, and its previous state. This then goes through a classifier layer, which decides it's next state, given that information. Further evidence for that: it was possible to use linear probes to confirm that the tokens that had been through the attention layer contained the information about the number of neighbours and the previous state. From all of this, it's clear that the model is running the Game of Life properly.
- constantcrying 1y ago
  Do you not understand the difference between empirical evidence and mathematical proof? Surely every person talking about NN research should be aware of that distinction. > How could it not be computing the game of life algorithm? Given that it gets 100% accuracy over multiple steps on a bunch random game boards it's never seen before. This is such an insane statement.
  - montebicyclelo 1y ago
    > This is such an insane statement. In what way? Maybe you mean something different when you say computing the game of Life algorithm.

montebicyclelo 1y ago

> the paper does not investigate whether the resulting NN actually performs the game of life algorithm How could it not be computing the game of life algorithm? Given that it gets 100% accuracy over multiple steps on a bunch random game boards it's never seen before. And then based on the structure of the net, and by examining the attention layers and finding that it's doing 3 by 3 average pooling, we can see that the attention layer produces a set of tokens, where each token contains the information of the number of neighbours it had, and its previous state. This then goes through a classifier layer, which decides it's next state, given that information. Further evidence for that: it was possible to use linear probes to confirm that the tokens that had been through the attention layer contained the information about the number of neighbours and the previous state. From all of this, it's clear that the model is running the Game of Life properly.

constantcrying 1y ago
Do you not understand the difference between empirical evidence and mathematical proof? Surely every person talking about NN research should be aware of that distinction. > How could it not be computing the game of life algorithm? Given that it gets 100% accuracy over multiple steps on a bunch random game boards it's never seen before. This is such an insane statement.
- montebicyclelo 1y ago
  > This is such an insane statement. In what way? Maybe you mean something different when you say computing the game of Life algorithm.