5 ms·

This is the key thing with the limitations of LLMs at present, it can't "remember" what it has or hasn't done, by design.

damontal 3y ago
It’s not saying it can’t remember. It’s saying it doesn’t have the ability to do what it did.
- hammyhavoc 3y ago
  > I just had ChatGPT write me a poem, and then in a new session, pasted the poem back in and asked it if it wrote it. > "No, I did not write that poem." Literally what the comment is saying. It wrote something and can't remember that it wrote something. Remembering that it wrote something is fundamental to answering the question of if it wrote something. Otherwise, how would it ascertain the difference between someone whose style mimics that of ChatGPT output and actual ChatGPT output? No memory means no certainty. It's different from asking it if a comment was generated by an LLM, in which the response heavily leaned on the word "appears". However, LLMs have no understanding of what it's given, it's all just tokens and Bayesian inference.
emptysongglass 3y ago
It doesn't "know" either. We continue to anthromopomorphize these models, which put uncredited doomsayers like Eliezer Yudkowsky on a pedestal.
- circuit10 3y ago
  What’s wrong with the word “know”? And with Eliezer Yudkowsky?
  - hammyhavoc 3y ago
    Because it's a word prediction engine. It's just tokens and Bayesian inference. It has no understanding of what it's doing. If a math textbook has sentence x written in it, does the textbook know this information, or does it just have this information stored in it in a human-readable format? LLMs only present their stored data in a human-readable format on the output, the data set itself isn't readable.
    - hackinthebochs 3y ago
      It would be nice if the naysayers could offer an argument in favor of it being "just" bayesian inference, or even explain what it would mean for it to be more than just statistics or bayesian inference. There is reason to believe these models are demonstrating the traits of understanding in some cases. I argue the point in some detail here: https://www.reddit.com/r/naturalism/comments/1236vzf https://www.reddit.com/r/naturalism/comments/1236vzf
    - circuit10 3y ago
      But LLMs use and reason with the data, so the word “know” is probably appropriate. Think of it as a analogy, if you want, but it’s reasonable to draw a comparison with what humans do. Even without the reasoning part I’m sure people already use the word “know” to describe how software is able to perform a task or make use of information. “It has no understanding” I keep seeing this claim, I don’t understand what it’s supposed to mean. Do we humans have this magical “true understanding”? Can we test for it?
      - JohnFen 3y ago
        I would say that "understanding" something means that you have a mental model of that thing. The more accurate and complete that mental model is, the "deeper" the understanding. LLMs, as I (ahem) understand them, do not have models of the things they're writing about. They do have a model of language (or whatever symbolic system they were trained on), but that's where it ends.
        hammyhavoc 3y ago
        I agree! I asked ChatGPT "Does an LLM understand, or is it simply Bayesian inference?", and the most relevant part of its output: "The model's understanding of language is derived from the vast amount of text it has been trained on. By capturing statistical regularities and patterns, an LLM can generate text that appears to be comprehensible and contextually appropriate. However, it's important to note that LLMs do not possess true understanding or consciousness. They lack semantic or conceptual understanding in the way humans do. Overall, an LLM employs a combination of statistical techniques, including Bayesian inference, and pattern recognition to generate text that simulates understanding to a certain extent."
  - emptysongglass 3y ago
    He's a self-appointed expert who is neither an AI researcher, computer scientist, or any other accredited title you'd expect someone loudly yelling, “WE'RE ALL GOING TO DIE” on a technical topic like the fundamentals of AI and its dangers to hold. You and others continue to foist human qualities on LLMs because it looks like magic. To know is to know oneself. LLMs do not know themselves. In fact, there is no “self” to know.
    - circuit10 3y ago
      I don’t know much about him but I thought he was a researcher? There is an “Academic publications” section on https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky and it says “He is a co-founder and research fellow at the Machine Intelligence Research Institute” LLMs are not self aware, I agree, but there is clearly some reasoning going on and you could argue that its model of human behaviour can somewhat simulate self awareness
      - hammyhavoc 3y ago
        > its model of human behaviour What model of human behaviour? I asked ChatGPT if it has a model of human behaviour, and it said "ChatGPT does not have an explicit model of human behavior".
        circuit10 3y ago
        As I just said ChatGPT is not self aware, so it can’t answer questions about it’s own workings like that. (this is somewhat true of humans too though, there is a lot going on in our subconscious, but we are aware of our own conscious thoughts to some extent at least) It just hallucinated an answer If it didn’t have a model of human behaviour, then it wouldn’t work, because the whole idea is it’s simulating what someone acting as a helpful assistant would do.
        hammyhavoc 3y ago
        Self-awareness isn't a requirement for the presentation of information, which LLMs continually prove. Is every answer thus not a hallucination by this logic? If it's trained on external information, why would this information not include observations and information made about it? A human doesn't need to be able to even read to present a book to someone else with contextually relevant information.

damontal 3y ago

It’s not saying it can’t remember. It’s saying it doesn’t have the ability to do what it did.

hammyhavoc 3y ago
> I just had ChatGPT write me a poem, and then in a new session, pasted the poem back in and asked it if it wrote it. > "No, I did not write that poem." Literally what the comment is saying. It wrote something and can't remember that it wrote something. Remembering that it wrote something is fundamental to answering the question of if it wrote something. Otherwise, how would it ascertain the difference between someone whose style mimics that of ChatGPT output and actual ChatGPT output? No memory means no certainty. It's different from asking it if a comment was generated by an LLM, in which the response heavily leaned on the word "appears". However, LLMs have no understanding of what it's given, it's all just tokens and Bayesian inference.

emptysongglass 3y ago

It doesn't "know" either. We continue to anthromopomorphize these models, which put uncredited doomsayers like Eliezer Yudkowsky on a pedestal.

circuit10 3y ago
What’s wrong with the word “know”? And with Eliezer Yudkowsky?
- hammyhavoc 3y ago
  Because it's a word prediction engine. It's just tokens and Bayesian inference. It has no understanding of what it's doing. If a math textbook has sentence x written in it, does the textbook know this information, or does it just have this information stored in it in a human-readable format? LLMs only present their stored data in a human-readable format on the output, the data set itself isn't readable.
  - hackinthebochs 3y ago
    It would be nice if the naysayers could offer an argument in favor of it being "just" bayesian inference, or even explain what it would mean for it to be more than just statistics or bayesian inference. There is reason to believe these models are demonstrating the traits of understanding in some cases. I argue the point in some detail here: https://www.reddit.com/r/naturalism/comments/1236vzf https://www.reddit.com/r/naturalism/comments/1236vzf
  - circuit10 3y ago
    But LLMs use and reason with the data, so the word “know” is probably appropriate. Think of it as a analogy, if you want, but it’s reasonable to draw a comparison with what humans do. Even without the reasoning part I’m sure people already use the word “know” to describe how software is able to perform a task or make use of information. “It has no understanding” I keep seeing this claim, I don’t understand what it’s supposed to mean. Do we humans have this magical “true understanding”? Can we test for it?
    - JohnFen 3y ago
      I would say that "understanding" something means that you have a mental model of that thing. The more accurate and complete that mental model is, the "deeper" the understanding. LLMs, as I (ahem) understand them, do not have models of the things they're writing about. They do have a model of language (or whatever symbolic system they were trained on), but that's where it ends.
      - hammyhavoc 3y ago
        I agree! I asked ChatGPT "Does an LLM understand, or is it simply Bayesian inference?", and the most relevant part of its output: "The model's understanding of language is derived from the vast amount of text it has been trained on. By capturing statistical regularities and patterns, an LLM can generate text that appears to be comprehensible and contextually appropriate. However, it's important to note that LLMs do not possess true understanding or consciousness. They lack semantic or conceptual understanding in the way humans do. Overall, an LLM employs a combination of statistical techniques, including Bayesian inference, and pattern recognition to generate text that simulates understanding to a certain extent."
- emptysongglass 3y ago
  He's a self-appointed expert who is neither an AI researcher, computer scientist, or any other accredited title you'd expect someone loudly yelling, “WE'RE ALL GOING TO DIE” on a technical topic like the fundamentals of AI and its dangers to hold. You and others continue to foist human qualities on LLMs because it looks like magic. To know is to know oneself. LLMs do not know themselves. In fact, there is no “self” to know.
  - circuit10 3y ago
    I don’t know much about him but I thought he was a researcher? There is an “Academic publications” section on https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky https://en.m.wikipedia.org/wiki/Eliezer_Yudkowsky and it says “He is a co-founder and research fellow at the Machine Intelligence Research Institute” LLMs are not self aware, I agree, but there is clearly some reasoning going on and you could argue that its model of human behaviour can somewhat simulate self awareness
    - hammyhavoc 3y ago
      > its model of human behaviour What model of human behaviour? I asked ChatGPT if it has a model of human behaviour, and it said "ChatGPT does not have an explicit model of human behavior".
      - circuit10 3y ago
        As I just said ChatGPT is not self aware, so it can’t answer questions about it’s own workings like that. (this is somewhat true of humans too though, there is a lot going on in our subconscious, but we are aware of our own conscious thoughts to some extent at least) It just hallucinated an answer If it didn’t have a model of human behaviour, then it wouldn’t work, because the whole idea is it’s simulating what someone acting as a helpful assistant would do.
        hammyhavoc 3y ago
        Self-awareness isn't a requirement for the presentation of information, which LLMs continually prove. Is every answer thus not a hallucination by this logic? If it's trained on external information, why would this information not include observations and information made about it? A human doesn't need to be able to even read to present a book to someone else with contextually relevant information.