4 ms·
You’re anthropomorphizing these chat bots. They may associate those words in specific contexts, but they don’t have that kind of abstract understanding. The be
by Retric 4y ago
You’re anthropomorphizing these chat bots. They may associate those words in specific contexts, but they don’t have that kind of abstract understanding.
The best way to understand them isn’t to look at what they do well but where they fail. A great example is how ChatGPT can initially make a few chess moves that seem reasonable, but very quickly it stops making valid moves. It’s not operating from some model of the game but rather imitating sequences of moves it’s seen before. The best analogy isn’t cognition but someone trying to make a better version of “lorem ipsum” for whatever promo you’re giving it.
- 2OEH8eoCRo0 4y ago> You’re anthropomorphizing these chat bots. Oh? don't anthropomorphize the thing we are supposed to "chat" with? It's basically what they're designed for except when they rudely tell you "I'm just an LLM!"
- codeulike 4y agoGPTs can model simpler games like Othello https://thegradient.pub/othello/ https://thegradient.pub/othello/ https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 with enough training data it could probably model chess - not well enough to win but well enough to make legal moves You think that I am misunderstanding whats going on and anthropomorphising ChatGPT. I know how ChatGPT works, my position is that we might be overestimating our selves, and underestimating the power of emergent phenomenon.
- kilgnad 4y agoExactly. The science is starting to realize the emergent effects of LLMs. These things can literally learn simply by "looking over your shoulder". We thought that you had to explicitly program hierarchical structures of causal reasoning into the network but science is showing that these structures are emergent. People won't believe you if you show actual evidence. You have to throw them a scientific paper written by an "expert" lol. And even then they will find a hard time changing their viewpoint. Its so strange why people are trying to downplay it all when even the science is showing they're wrong. They have to throw accusations around of anthropomorphisation. Seriously? It's very easy to identify the bias of anthropomorphisation. Anyone can easily tiptoe around that bias with a simple argument. Clearly what's going on with chatGPT is much more complex then that. I recommend people stop using that word in this topic. It's akin to accusing someone they have brain damage. Clearly they don't.
- deleted 4y ago[deleted]
- Retric 4y agoThe context is limited to the last 2048 tokens which is insufficient to model some valid chess games. Thus no amount of training would be enough without simplifying the problem.
- codeulike 4y agoIt could handle 300 moves or so with that many tokens, seems like plenty. I propose that if you gathered enough chess transcripts like this: e4 e5 Nc3 Nc6 f4 exf4 d4 d5 Bxf4 Bb4 exd5 Qxd5 Kf2 Qh4+ .... And fed them into a blank GPT then it would learn chess like a language and be able to make legal moves most of the time. It would do this by inferring and modelling the board and the rules. It wouldn't be a great player but it would be able to make moves. This is bascially what the Othello paper I linked above is all about. They used GPT-2 I think. Chess is harder but I reckon could be done with a bigger model and more training data.
- Retric 4y agoSure often playing valid moves is possible, but not only is it moving the goalposts but different approaches actually produce skilled players using neural networks. Anyway, my point was less about the game than how its failure show what’s going on better than it’s successes. The GPT approach is optimized for chat bots and its successes have more to do with exploiting how we approach communication than anything that can turn into AGI.
- codeulike 4y agoI'm not moving the goalposts. You said: "It’s not operating from some model of the game" I'm saying: "it would model the rules of chess, if you fed it enough game transcripts"
- Retric 4y agoIf it can’t strictly play valid movies then it isn’t modeling the game. Instead it’s modeling something else which is somewhat related to the game. Aka someone playing tick tack toe who moves on top of their opponent’s move isn’t playing tick tack toe.
- sebzim4500 4y agoIt took me ages to learn how to play chess without a board, I'm not surprised that chatgpt can't do it yet.