4 ms·
"GPT-4 plays chess at a strong club level" according to @BorisMPower, whose X/Twitter profile says "Member of Technical Staff @OpenAI". And today @GrantSlatton
by bumbledraven 3y ago
"GPT-4 plays chess at a strong club level" according to @BorisMPower, whose X/Twitter profile says "Member of Technical Staff @OpenAI". And today @GrantSlatton claimed that gpt-3.5-turbo-instruct plays chess at 1800+ vs Stockfish when prompted using PGN.
On Aug 10, 2023, @BorisMPower wrote (https://x.com/BorisMPower/status/1689838493806333953 https://x.com/BorisMPower/status/1689838493806333953):
> [...] GPT-4 plays chess at a strong club level when properly prompted, which is impossible to achieve without having a good internal model of the game.
> Even at Go, the model does ~10x better than random, by essentially picking up on locality being a strong signal.
> I don’t think anything has been published unfortunately. ELO is around 1800
On Sep 18, 2023, @GrantSlatton wrote (https://x.com/GrantSlatton/status/1703913578036904431 https://x.com/GrantSlatton/status/1703913578036904431):
> The new GPT model, gpt-3.5-turbo-instruct, can play chess around 1800 Elo.
> I had previously reported that GPT cannot play chess, but it appears this was just the RLHF'd chat models. The pure completion model succeeds.
> The new model readily beats Stockfish Level 4 (1700) and still loses respectably to Level 5 (2000). Never attempted illegal moves. Used clever opening sacrifice, and incredibly cheeky pawn & king checkmate, allowing the opponent to uselessly promote.
> https://lichess.org/K6Q0Lqda https://lichess.org/K6Q0Lqda
> I used this PGN style prompt to mimic a grandmaster game.
> The highlighting is a bit wrong. GPT made all its own moves, I input Stockfish moves manually.
> h/t to @zswitten for this prompt style
> [OpenAI Playground screenshot showing PGN game in the prompt]
I was able to reproduce this just now: gpt-3.5-turbo-instruct, prompted with PGN, defeated Stockfish Level 4 (1700?) on LiChess (https://lichess.org/D39lnanQ https://lichess.org/D39lnanQ).
Here are the prompts/code I used: https://github.com/jordancurve/gpt-vs-stockfish/blob/main/game.go https://github.com/jordancurve/gpt-vs-stockfish/blob/main/ga...