6 ms·

Neural Chess

38 points by fagnerbrack 3y ago

Imnimo 3y ago
Am I reading right that the network here is an MLP with a single hidden layer of 50 neurons? It's a fun project, but I think the author would have benefitted from spending more time on finding a reasonable network architecture instead of jumping into the (admittedly alluring) quagmire of RL.
seunosewa 3y ago
Should he have introduced some randomization to prevent the repetition of moves?
- falcor84 3y ago
  I'll just put this here: ----------------------------- Sussman attains enlightenment ----------------------------- In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. “What are you doing?”, asked Minsky. “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied. “Why is the net wired randomly?”, asked Minsky. “I do not want it to have any preconceptions of how to play”, Sussman said. Minsky then shut his eyes. “Why do you close your eyes?”, Sussman asked his teacher. “So that the room will be empty.” At that moment, Sussman was enlightened.
  - ukuina 3y ago
    I feel dumb, but I do not understand this. Randomization of topology and weights is not a terrible starting point, is it?
    - CamperBob2 3y ago
      Some say that Minsky was responsible for about -50 years of progress in the AI field [1], so they'd say that no, there's not much enlightenment to be found in his work. 1: https://ai.stackexchange.com/questions/1288/did-minsky-and-papert-know-that-multi-layer-perceptrons-could-solve-xor https://ai.stackexchange.com/questions/1288/did-minsky-and-p...
    - Imnimo 3y ago
      I don't know if my reading is the intended one, but the way I've always interpreted this is not that random initialization is bad. Rather, the error Sussman makes is saying that a randomly initialized network has no preconceptions of how to play. It does - it's just that the preconceptions are randomized and so you don't know what they are. Similarly, closing your eyes does not make the room empty, it just means you don't know what's in it. That said, I don't think my interpretation really matches the intent of the poster above.
roenxi 3y ago
The article's claim that we can see the "video here" is false - the link is broken. The video is here: https://www.youtube.com/watch?v=DcYLT37ImBY https://www.youtube.com/watch?v=DcYLT37ImBY. It is a really interesting showcase and something is happening there that the writer at pvdz.ee didn't seem to find as interesting as I do. There is a concept that I bumped into of "Novelty Search" in the genetic algorithm space. Rather than trying to guess what behaviour is "good" and reward it, instead reward some novelty heuristic. Agents explore the space more thoroughly. The video is a really quite a good showcase of the technique - he only encodes hints on how to play after a pure novelty based approach is shown to fail. pvdz.ee seems to be trying to encode his game knowledge into the reward function early and seemed unsatisfied in the results based on his discussion. If I had a Nvidia graphics card instead of an AMD one, an experiment I'd have run a few years ago was a Go AI that tried to find a novel move heuristic to see if that sped up training. But that was a few years ago, I was defeated by the poor compute driver situation and haven't felt the urge since.
- deleted 3y ago
  [deleted]
gwern 3y ago
> Actually, my first thought was to just feed these pgn strings to a network and create an LLM-esque sort of network. But I wasn't convinced that I was going to get that anywhere within reasonable time. A number of people have done this in various forms (including, advertisement, Shawn Presser & myself way back in January 2020 with GPT-2: https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-game/ https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-... ); eg https://github.com/ricsonc/transformers-play-chess/blob/master/README.md https://github.com/ricsonc/transformers-play-chess/blob/mast... It's actually extremely easy, you could get started within hours at this point, with all the infrastructure for finetuning LLMs these days. Beyond the hobbyist/research level, people have noted that ChatGPT-3/4 are pretty good at chess, and the reason for that seems to be that they dumped an ELO-filtered PGN dataset into the training corpus (for unclear reasons, given how much data that would be but also how useless that would seem to be).
- matt_daemon 3y ago
  Also a nice article[0] featured on here[1] recently. [0] https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c... [1] https://news.ycombinator.com/item?id=38893456 https://news.ycombinator.com/item?id=38893456
xianshou 3y ago
Sadly, the main point this post reinforces is the emptiness of modern credentialing. A full degree in "cognitive AI" equipped the author to make exactly none of the correct choices in the network architecture and training. A valiant attempt, but five weeks with Pytorch and Github would have beaten the entire university education process :/
- janalsncm 3y ago
  I think that’s a little harsh. Chess programming and ML both require fairly sophisticated understanding. And a lot of approaches which seem reasonable beforehand just don’t work when you try them. It also seems like the author isn’t familiar with a lot of the work which has been done previously which probably would’ve given them a head start on this. Leela, Maia, and Alpha Zero would be good reference material.

38 points by fagnerbrack 3y ago

Am I reading right that the network here is an MLP with a single hidden layer of 50 neurons? It's a fun project, but I think the author would have benefitted from spending more time on finding a reasonable network architecture instead of jumping into the (admittedly alluring) quagmire of RL.

seunosewa 3y ago

Should he have introduced some randomization to prevent the repetition of moves?

falcor84 3y ago
I'll just put this here: ----------------------------- Sussman attains enlightenment ----------------------------- In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. “What are you doing?”, asked Minsky. “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied. “Why is the net wired randomly?”, asked Minsky. “I do not want it to have any preconceptions of how to play”, Sussman said. Minsky then shut his eyes. “Why do you close your eyes?”, Sussman asked his teacher. “So that the room will be empty.” At that moment, Sussman was enlightened.
- ukuina 3y ago
  I feel dumb, but I do not understand this. Randomization of topology and weights is not a terrible starting point, is it?
  - CamperBob2 3y ago
    Some say that Minsky was responsible for about -50 years of progress in the AI field [1], so they'd say that no, there's not much enlightenment to be found in his work. 1: https://ai.stackexchange.com/questions/1288/did-minsky-and-papert-know-that-multi-layer-perceptrons-could-solve-xor https://ai.stackexchange.com/questions/1288/did-minsky-and-p...
  - Imnimo 3y ago
    I don't know if my reading is the intended one, but the way I've always interpreted this is not that random initialization is bad. Rather, the error Sussman makes is saying that a randomly initialized network has no preconceptions of how to play. It does - it's just that the preconceptions are randomized and so you don't know what they are. Similarly, closing your eyes does not make the room empty, it just means you don't know what's in it. That said, I don't think my interpretation really matches the intent of the poster above.

roenxi 3y ago

The article's claim that we can see the "video here" is false - the link is broken. The video is here: https://www.youtube.com/watch?v=DcYLT37ImBY https://www.youtube.com/watch?v=DcYLT37ImBY. It is a really interesting showcase and something is happening there that the writer at pvdz.ee didn't seem to find as interesting as I do. There is a concept that I bumped into of "Novelty Search" in the genetic algorithm space. Rather than trying to guess what behaviour is "good" and reward it, instead reward some novelty heuristic. Agents explore the space more thoroughly. The video is a really quite a good showcase of the technique - he only encodes hints on how to play after a pure novelty based approach is shown to fail. pvdz.ee seems to be trying to encode his game knowledge into the reward function early and seemed unsatisfied in the results based on his discussion. If I had a Nvidia graphics card instead of an AMD one, an experiment I'd have run a few years ago was a Go AI that tried to find a novel move heuristic to see if that sped up training. But that was a few years ago, I was defeated by the poor compute driver situation and haven't felt the urge since.

deleted 3y ago
[deleted]

gwern 3y ago

> Actually, my first thought was to just feed these pgn strings to a network and create an LLM-esque sort of network. But I wasn't convinced that I was going to get that anywhere within reasonable time. A number of people have done this in various forms (including, advertisement, Shawn Presser & myself way back in January 2020 with GPT-2: https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-game/ https://slatestarcodex.com/2020/01/06/a-very-unlikely-chess-... ); eg https://github.com/ricsonc/transformers-play-chess/blob/master/README.md https://github.com/ricsonc/transformers-play-chess/blob/mast... It's actually extremely easy, you could get started within hours at this point, with all the infrastructure for finetuning LLMs these days. Beyond the hobbyist/research level, people have noted that ChatGPT-3/4 are pretty good at chess, and the reason for that seems to be that they dumped an ELO-filtered PGN dataset into the training corpus (for unclear reasons, given how much data that would be but also how useless that would seem to be).

matt_daemon 3y ago
Also a nice article[0] featured on here[1] recently. [0] https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c... [1] https://news.ycombinator.com/item?id=38893456 https://news.ycombinator.com/item?id=38893456

xianshou 3y ago

Sadly, the main point this post reinforces is the emptiness of modern credentialing. A full degree in "cognitive AI" equipped the author to make exactly none of the correct choices in the network architecture and training. A valiant attempt, but five weeks with Pytorch and Github would have beaten the entire university education process :/

janalsncm 3y ago
I think that’s a little harsh. Chess programming and ML both require fairly sophisticated understanding. And a lot of approaches which seem reasonable beforehand just don’t work when you try them. It also seems like the author isn’t familiar with a lot of the work which has been done previously which probably would’ve given them a head start on this. Leela, Maia, and Alpha Zero would be good reference material.