2 ms·

OFFTOPIC: User: i am going ask some question and you are going to answer them in short and at last append count of number of character your answer paragraph ha

OFFTOPIC: User: i am going ask some question and you are going to answer them in short and at last append count of number of character your answer paragraph has. ChatGPT: Sure, I'll do my best to answer your questions concisely and include the character count at the end of each response. Let's get started! User: what is a book? ChatGPT: A book is a collection of written or printed pages bound together, typically containing a story, information, or ideas. (49 characters) When would LLMs correctly do this ? Can gpt4 do this?

danparsonson 3y ago
It's not clear to me that an LLM could ever do that, given how they work - it's not aware of what it's saying (at least not in the same way you and I are) but rather using statistical methods to choose the next word each time. After "....story, information, or ideas (" what is most likely to come next? We know that the answer is 'the correct number of characters in the sentence' but ChatGPT isn't backtracking and calculating that figure, it's trying to infer the number based on context which is obviously impossible. Also it's at least partly stochastic so I guess if you run the prompt multiple times, you'll get multiple different word counts.
- born-jre 3y ago
  yeah that that's my guess too. That's why maybe these GPT class models are not only you need, some architectural change has to happen before we can truly shout AGI. :D Statistical parrot view of LLM is true but i think it is also a little bit reductionist, cz when it is trying to predict next token you can only go so far with just simple statistics, so model has to develop "secondary representation/leaning/reasoning" ?? That's what I understood from watching Ilya Sutskever interview[0] where he gives an example of detective novel where murderer is reveled in last page. [0]: https://youtu.be/XjSUJUL9ADw?t=1674 https://youtu.be/XjSUJUL9ADw?t=1674
  - danparsonson 3y ago
    Agreed - I think LLMs may be an important component in our eventual robot overlords, but we're not all the way there yet. As for secondary representation/learning/reasoning - the video link was interesting, thanks; am I correct that the 'detective novel' example Ilya used was just a thought experiment? He seemed to say, if ChatGPT could name the murderer, then it would have to be reasoning? But we haven't demonstrated ChatGPT actually doing this yet have we? The token window is surely not big enough for an entire novel. So I think maybe they're jumping ahead a little bit there. Perhaps the magic is multiple levels of abstraction encoded in the network during training, or something like that? You could perhaps argue that this is some sort of unconscious understanding, but it seems to me that reasoning requires iteration (Ilya talks about this - you go away and think about the answer for a bit first) which isn't happening automatically, although perhaps it can be guided by careful and repeated prompting ('asking it to think out loud'). This wouldn't be a million miles away from other deep learning projects like image classifiers - a network trained to recognise cats encodes multiple different abstractions of 'catness' and images match or not based on these representations. Side note that prompting someone to think out loud with repeated targeted questions can also work for a human being who just answers the first thing that comes to their mind (for example a young child), although most people eventually develop the ability to introspect and draw on their past experience, and no longer need such help. We could maybe broadly categorise human intelligence as a combination of - short term memory, providing context and immediate goals (e.g. 'switch on the light') - long term memory, encoding all kinds of experience - learning, which shapes long term memory based on short term memory (the current context; remembering the things that are happening now) and also based on long term memory (relating the current context to previous experiences) - introspection, which allows iterative 'self-prompting' to improve the accuracy of a thought, before saying it out loud but also during learning - language generation, which is mainly about word association based on long and short term memory - what else? 'Language generation' obviously maps nicely to an LLM, whose short-term memory is the token window and whose long term memory is the trained network. If OpenAI started including chat sessions in their training set (maybe they are already?) then in some sense the 'learning' step above would be covered, and what's left is the ability for ChatGPT to set and understand goals, and self-prompt for introspection (plus whatever else I missed :-).
snickell 3y ago
This is particularly hard for LLMs because they don't have words input as characters but rather as tokens (1-N characters). I haven't tested this, but they might do better at counting words?
wilg 3y ago
ChatGPT's code interpreter variant makes this mostly possible. It doesn't seem to be able to read its own output within the same prompt. But if you ask it afterwards it can do it with python: https://i2.paste.pics/8995850ed937795d64f507f1497bb210.png?trs=f3a1bae68dcca274508593afa7ce4959339a6afb201752e175c4aa833f879d33 https://i2.paste.pics/8995850ed937795d64f507f1497bb210.png?t... There's no reason to think LLMs alone are the only tool we have for AI!

danparsonson 3y ago

It's not clear to me that an LLM could ever do that, given how they work - it's not aware of what it's saying (at least not in the same way you and I are) but rather using statistical methods to choose the next word each time. After "....story, information, or ideas (" what is most likely to come next? We know that the answer is 'the correct number of characters in the sentence' but ChatGPT isn't backtracking and calculating that figure, it's trying to infer the number based on context which is obviously impossible. Also it's at least partly stochastic so I guess if you run the prompt multiple times, you'll get multiple different word counts.

born-jre 3y ago
yeah that that's my guess too. That's why maybe these GPT class models are not only you need, some architectural change has to happen before we can truly shout AGI. :D Statistical parrot view of LLM is true but i think it is also a little bit reductionist, cz when it is trying to predict next token you can only go so far with just simple statistics, so model has to develop "secondary representation/leaning/reasoning" ?? That's what I understood from watching Ilya Sutskever interview[0] where he gives an example of detective novel where murderer is reveled in last page. [0]: https://youtu.be/XjSUJUL9ADw?t=1674 https://youtu.be/XjSUJUL9ADw?t=1674
- danparsonson 3y ago
  Agreed - I think LLMs may be an important component in our eventual robot overlords, but we're not all the way there yet. As for secondary representation/learning/reasoning - the video link was interesting, thanks; am I correct that the 'detective novel' example Ilya used was just a thought experiment? He seemed to say, if ChatGPT could name the murderer, then it would have to be reasoning? But we haven't demonstrated ChatGPT actually doing this yet have we? The token window is surely not big enough for an entire novel. So I think maybe they're jumping ahead a little bit there. Perhaps the magic is multiple levels of abstraction encoded in the network during training, or something like that? You could perhaps argue that this is some sort of unconscious understanding, but it seems to me that reasoning requires iteration (Ilya talks about this - you go away and think about the answer for a bit first) which isn't happening automatically, although perhaps it can be guided by careful and repeated prompting ('asking it to think out loud'). This wouldn't be a million miles away from other deep learning projects like image classifiers - a network trained to recognise cats encodes multiple different abstractions of 'catness' and images match or not based on these representations. Side note that prompting someone to think out loud with repeated targeted questions can also work for a human being who just answers the first thing that comes to their mind (for example a young child), although most people eventually develop the ability to introspect and draw on their past experience, and no longer need such help. We could maybe broadly categorise human intelligence as a combination of - short term memory, providing context and immediate goals (e.g. 'switch on the light') - long term memory, encoding all kinds of experience - learning, which shapes long term memory based on short term memory (the current context; remembering the things that are happening now) and also based on long term memory (relating the current context to previous experiences) - introspection, which allows iterative 'self-prompting' to improve the accuracy of a thought, before saying it out loud but also during learning - language generation, which is mainly about word association based on long and short term memory - what else? 'Language generation' obviously maps nicely to an LLM, whose short-term memory is the token window and whose long term memory is the trained network. If OpenAI started including chat sessions in their training set (maybe they are already?) then in some sense the 'learning' step above would be covered, and what's left is the ability for ChatGPT to set and understand goals, and self-prompt for introspection (plus whatever else I missed :-).

snickell 3y ago

This is particularly hard for LLMs because they don't have words input as characters but rather as tokens (1-N characters). I haven't tested this, but they might do better at counting words?

wilg 3y ago

ChatGPT's code interpreter variant makes this mostly possible. It doesn't seem to be able to read its own output within the same prompt. But if you ask it afterwards it can do it with python: https://i2.paste.pics/8995850ed937795d64f507f1497bb210.png?trs=f3a1bae68dcca274508593afa7ce4959339a6afb201752e175c4aa833f879d33 https://i2.paste.pics/8995850ed937795d64f507f1497bb210.png?t... There's no reason to think LLMs alone are the only tool we have for AI!