8 ms·
If something has general intelligence it should be able to read the rules of a game and follow them. Therefore an artificial general intelligence (AGI) should b
by janalsncm 2mo ago
If something has general intelligence it should be able to read the rules of a game and follow them. Therefore an artificial general intelligence (AGI) should be able to do this.
So we have a situation where very powerful and influential people are saying we will have AGI in 6 months (if we don’t already), yet the facts on the ground are so clearly pointing in the opposite direction.
- hackinthebochs 2mo agoI would bet a lot of money that Astra can follow the rules of chess (perhaps if repeated within the context window). Also, this is a different argument than what I responded to.
- minraws 2mo agoI can write you a benchmark to prove it even with a heavy handed system prompt Astra will make an illegal move during the course of the games first few moves are generally ok since it's just throwing out learned moves.
- hackinthebochs 2mo agoI'd genuinely like to see the results of that.
- janalsncm 2mo agoI would definitely take you up on that.
- simianwords 2mo agohttps://www.chessbench.org/ https://www.chessbench.org/ >GPT-6 Astra xHigh: 0.06% rejected moves
- frde_me 2mo agoI wonder if I would do better as a human, maybe? Or would I happen to have one move in 1500+ that's not valid? I could see myself messing up something at some point if the board is complicated enough and trying an illegal move, perhaps if a piece somewhere would attack my king if I moved another piece. Even through I do know the rules of chess, and I have played a few games once every so often.
- Gregkion 2mo agoSo we humans are not a general intelligence then? And the stuff i'm using LLMs daily is just fake? I see i see. I will see myself out of this weird discussion while I let an LLM continue doing a lot of interesting things.
- dosisking 2mo ago[flagged]
- zahlman 2mo ago> So we humans are not a general intelligence then? No, because we can, in fact, generally read the rules of a game and then follow them. It's actually a hobby for many of us. > And the stuff i'm using LLMs daily is just fake? This misses the point completely.
- hackinthebochs 2mo ago> generally read the rules of a game and then follow them How many times do you think chess.com prevents illegal moves from being executed? Even Super GM's fall for mate-in-1's occasionally, which is functionally equivalent to missing a pin or a check. This idea that LLMs failing to only ever make legal moves undermines their intelligence doesn't pass the smell test.
- diehunde 2mo agoDo you play chess ? Do you even know what an illegal move is ?
- hackinthebochs 2mo agoIf you have something to contribute to the discussion, just say it
- zahlman 2mo agoChess.com has to accommodate people who haven't learned the rules yet on the low end. On the high end, people are commonly playing fast enough that they're often outlining sequences of multiple "pre-moves" during the opponent's turn in order to avoid losing on time. And no, I would not agree with that functional equivalence.