4 ms·
When you talk to children who haven't developed the skill of talking completely, you still get the sense that there's a lot going on inside that they're unable
by perryizgr8 5y ago
When you talk to children who haven't developed the skill of talking completely, you still get the sense that there's a lot going on inside that they're unable to express. Sometimes they will show it to you with their actions. I wonder if chat bots are also struggling with the language, but have an internal story that's playing out, desperate to be understood. But they can't, because the only interface they have is a stream of text.
- parkingrift 5y agoI can't speak to that because I honestly have no idea. ...but GPT3 doesn't seem to have some inner monologue. It is just flat unable to recall context or understand the environment. The only opportunity I had to chat with GPT3 I said hi, and then asked how old it was. GPT3 gave me some answer about how it was created in 2016 but it keeps evolving. I said wow that's pretty young. GPT3 says yes I am young, but learning fast. I then asked GPT3 at what point would it consider itself old. GPT3 says 10 years ago, maybe 20 or 30 years ago. I respond to GPT3 with confusion. GPT3 then starts talking about Bitcoin. Literally.
- immibis 5y agoI remember reading a paper showing that GPT-x bots are more likely to get simple tasks correct (e.g. multi-digit math) if you ask them about each step. This suggests the text stream really is all there is. Well, there is some internal state, but not a lot that isn't text. (For example, if you ask it to add 123 and 457, it'll probably get it wrong. But if you ask it what to do first, it'll say add 3 and 7. If you ask it what that is, it'll say 0 carry 1. And so on)