5 ms·
GPTs can model simpler games like Othello https://thegradient.pub/othello/ https://thegradient.pub/othello/ https://arxiv.org/abs/2210.13382 https://arxiv.org
by codeulike 4y ago
GPTs can model simpler games like Othello
https://thegradient.pub/othello/ https://thegradient.pub/othello/
https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382
with enough training data it could probably model chess - not well enough to win but well enough to make legal moves
You think that I am misunderstanding whats going on and anthropomorphising ChatGPT. I know how ChatGPT works, my position is that we might be overestimating our selves, and underestimating the power of emergent phenomenon.
- kilgnad 4y agoExactly. The science is starting to realize the emergent effects of LLMs. These things can literally learn simply by "looking over your shoulder". We thought that you had to explicitly program hierarchical structures of causal reasoning into the network but science is showing that these structures are emergent. People won't believe you if you show actual evidence. You have to throw them a scientific paper written by an "expert" lol. And even then they will find a hard time changing their viewpoint. Its so strange why people are trying to downplay it all when even the science is showing they're wrong. They have to throw accusations around of anthropomorphisation. Seriously? It's very easy to identify the bias of anthropomorphisation. Anyone can easily tiptoe around that bias with a simple argument. Clearly what's going on with chatGPT is much more complex then that. I recommend people stop using that word in this topic. It's akin to accusing someone they have brain damage. Clearly they don't.
- deleted 4y ago[deleted]
- Retric 4y agoThe context is limited to the last 2048 tokens which is insufficient to model some valid chess games. Thus no amount of training would be enough without simplifying the problem.
- codeulike 4y agoIt could handle 300 moves or so with that many tokens, seems like plenty. I propose that if you gathered enough chess transcripts like this: e4 e5 Nc3 Nc6 f4 exf4 d4 d5 Bxf4 Bb4 exd5 Qxd5 Kf2 Qh4+ .... And fed them into a blank GPT then it would learn chess like a language and be able to make legal moves most of the time. It would do this by inferring and modelling the board and the rules. It wouldn't be a great player but it would be able to make moves. This is bascially what the Othello paper I linked above is all about. They used GPT-2 I think. Chess is harder but I reckon could be done with a bigger model and more training data.
- Retric 4y agoSure often playing valid moves is possible, but not only is it moving the goalposts but different approaches actually produce skilled players using neural networks. Anyway, my point was less about the game than how its failure show what’s going on better than it’s successes. The GPT approach is optimized for chat bots and its successes have more to do with exploiting how we approach communication than anything that can turn into AGI.
- codeulike 4y agoI'm not moving the goalposts. You said: "It’s not operating from some model of the game" I'm saying: "it would model the rules of chess, if you fed it enough game transcripts"
- Retric 4y agoIf it can’t strictly play valid movies then it isn’t modeling the game. Instead it’s modeling something else which is somewhat related to the game. Aka someone playing tick tack toe who moves on top of their opponent’s move isn’t playing tick tack toe.
- codeulike 4y agoIf a human had only ever seen Othello moves in notation form and never seen the board and had to infer the rules, they'd probably be about 99.99% accurate. They might make a mistake about 1 move in 1000 - fail to spot something, or encounter some edge case of rules they hadn't been able to infer. That's how accurate GPT-2 was.