4 ms·
I would bet a lot of money that Astra can follow the rules of chess (perhaps if repeated within the context window). Also, this is a different argument than wha
by hackinthebochs 17d ago
I would bet a lot of money that Astra can follow the rules of chess (perhaps if repeated within the context window). Also, this is a different argument than what I responded to.
- minraws 17d agoI can write you a benchmark to prove it even with a heavy handed system prompt Astra will make an illegal move during the course of the games first few moves are generally ok since it's just throwing out learned moves.
- hackinthebochs 17d agoI'd genuinely like to see the results of that.
- janalsncm 17d agoI would definitely take you up on that.
- simianwords 17d agohttps://www.chessbench.org/ https://www.chessbench.org/ >GPT-6 Astra xHigh: 0.06% rejected moves
- frde_me 16d agoI wonder if I would do better as a human, maybe? Or would I happen to have one move in 1500+ that's not valid? I could see myself messing up something at some point if the board is complicated enough and trying an illegal move, perhaps if a piece somewhere would attack my king if I moved another piece. Even through I do know the rules of chess, and I have played a few games once every so often.