4 ms·

It's very unlikely. The moves it makes are not always good. Any public facing chess engine API or the API they chose would have been at least 'good'.

adamrezich 4y ago
if that's the case then I agree it is unlikely, but then I would posit it is highly unlikely that it flawlessly follows the rules of chess. why is this interesting then? of course an LLM can create the appearance of "knowing how to play chess" that passes basic scrutiny, given that chess notation is a thing and all, but that's totally separate from having "learned" an ironclad "understanding" of the rules of a relatively basic turn-based board game. if Microsoft wanted to tout that its "AI" could actually play chess, then the obvious thing to do would be to hook in a chess AI system as I suggested, as opposed to somehow engineering (or prompt "engineering") a better-able-to-follow-the-rules-of-chess language model. again, this is how nearly everything is going to be going forward in this space: creating a product that has the appearances of being able to do so many different things, but in reality is many different specific subsystems wrapped in a nice language model package. I'm getting really tired of seeing all these "look at what I made the 'AI' do!!" posts that really aren't interesting at all when you get past the headline.
- famouswaffles 4y ago
  It's interesting because it does demonstrate an understanding of chess, just like othello gpt demonstrated an understanding of othello. It follows the rules of chess and makes legal rules the vast majority of times. Making illegal moves occasionally doesn't mean you don't understand chess. Not if you learned chess in a self supervisory fashion. It's also a vast improvement over the chess performance of cGPT which makes legal moves on basically random chance and flies off the handle quickly. It's also just plain interesting that a language model can pick up how to play these games from training on text in the manner they do. If you don't find the idea that LLMs are universal computation engines interesting then i don't know what to tell you.

adamrezich 4y ago

if that's the case then I agree it is unlikely, but then I would posit it is highly unlikely that it flawlessly follows the rules of chess. why is this interesting then? of course an LLM can create the appearance of "knowing how to play chess" that passes basic scrutiny, given that chess notation is a thing and all, but that's totally separate from having "learned" an ironclad "understanding" of the rules of a relatively basic turn-based board game. if Microsoft wanted to tout that its "AI" could actually play chess, then the obvious thing to do would be to hook in a chess AI system as I suggested, as opposed to somehow engineering (or prompt "engineering") a better-able-to-follow-the-rules-of-chess language model. again, this is how nearly everything is going to be going forward in this space: creating a product that has the appearances of being able to do so many different things, but in reality is many different specific subsystems wrapped in a nice language model package. I'm getting really tired of seeing all these "look at what I made the 'AI' do!!" posts that really aren't interesting at all when you get past the headline.

famouswaffles 4y ago
It's interesting because it does demonstrate an understanding of chess, just like othello gpt demonstrated an understanding of othello. It follows the rules of chess and makes legal rules the vast majority of times. Making illegal moves occasionally doesn't mean you don't understand chess. Not if you learned chess in a self supervisory fashion. It's also a vast improvement over the chess performance of cGPT which makes legal moves on basically random chance and flies off the handle quickly. It's also just plain interesting that a language model can pick up how to play these games from training on text in the manner they do. If you don't find the idea that LLMs are universal computation engines interesting then i don't know what to tell you.