3 ms·
Why build a chess engine these days, just use an LLM?
by deadbabe 2y ago
Why build a chess engine these days, just use an LLM?
- ziddoap 2y agoFun, a learning experience, practicing a skill set, curiosity, etc. Take your pick.
- emptybits 2y agoWhy consume when you can create? Building is often more fun and almost always more educational than just using. Even in “failure”. :-)
- achierius 2y agoLLMs are quite bad at chess actually, even compared to human players -- and certainly compared to proper modern chess engines
- PaulHoule 2y agoThey usually struggle to always generate valid moves. And that's pivotal. If you have a program which always makes valid moves and gives up when it has lost you wrote a proper chess playing program. It may play badly, but it plays.
- ElFitz 2y agoThat could now be achieved by precomputing all valid moves and using outlines[0] or Structured Outputs[1] to constrain the output. [0]: https://github.com/dottxt-ai/outlines https://github.com/dottxt-ai/outlines [1]: https://openai.com/index/introducing-structured-outputs-in-the-api/ https://openai.com/index/introducing-structured-outputs-in-t...
- janalsncm 2y agoYour question sounds flippant but it’s actually quite deep. Why aren’t LLMs good at chess? The answer is likely that to be good at chess at some level requires search (and probably some ordering constraint on your evaluation function too that I’m not smart enough to figure out). LLMs aren’t searching. They are memorizing. This explains their extremely poor performance on out of domain positions, whereas Stockfish can easily understand them.
- deadbabe 2y agoI think chess is a great example to use to demonstrate that LLMs don’t actually know anything, and will be entirely unsuitable to replace humans on anything except thoroughly solved and documented problems.