4 ms·
PS: I ran and as suspected got-3.5-turbo-instruct does not beat stockfish, it is not even close "Final Results: gpt-3.5-turbo-instruct: Wins=0, Losses=6, Draws=
by fsndz 2y ago
PS: I ran and as suspected got-3.5-turbo-instruct does not beat stockfish, it is not even close
"Final Results: gpt-3.5-turbo-instruct: Wins=0, Losses=6, Draws=0, Rating=1500.00 stockfish: Wins=6, Losses=0, Draws=0, Rating=1500.00"
https://www.loom.com/share/870ea03197b3471eaf7e26e9b17e1754?sid=073f0b78-7be9-42af-95c2-240d3b49d1c8 https://www.loom.com/share/870ea03197b3471eaf7e26e9b17e1754?...
- computerex 2y agoMaybe there's some difference in the setup because the OP reports that the model beats stockfish (how they had it configured) every single game.
- Filligree 2y agoOP had stockfish at its weakest preset.
- fsndz 2y agoDid the same and gpt-3.5-turbo-instruct still lost all the games. maybe a diff in stockfish version ? I am using stockfish 16
- mannykannot 2y agoThat is a very pertinent question, especially if Stockfish has been used to generate training data.
- golol 2y agoYou have to get the model to think in PGN data. It's crucial to use the exact PGN format it sae in its training data and to give it few shot examples.