5 ms·

> to identify that the answer is not stored in source data How would an LLM do that?

jncfhnb 3y ago
They do this already all the time. Probably the majority of the time. The problem is that a minority of the time is still very problematic. How do they do this? The same as they do now. The most likely token is that the bot doesn’t know the answer. Which is a behavior emergent from its tuning. I don’t get how people believe it can parse complex questions to produce novel ideas but can’t defer to saying “idk” when the answer isn’t known.
- jijijijij 3y ago
  So, you are basing your assessment on your gut feel and personal impression with ChatGPT? Maybe you should tone down the spice a bit, then. Unless you can explain how an actual understanding emerges within an LLM, you can't explain how it would answer the question definitely - it doesn't know, if it does, or does not know something. Generally speaking.
  - jncfhnb 3y ago
    I’m basing it on my being a data scientist who does this. > Unless you can explain how an actual understanding emerges within an LLM Tuning creates the contextual framework on which test is mapped to a latent space that encodes the meaning and most likely next sequences of text rather than just raw most likely sequence of text as seen in training data. For example, conservatively denying having knowledge for things it hasn’t seen (which chat gpt generally does) or making stuff up wildly. > you can't explain how it would answer the question definitely Of course not. It’s a random behavior. It has plenty of flaws.
    - jijijijij 3y ago
      > You don’t need proper cognition to identify that the answer is not stored in source data That's the original argument. > Tuning creates the contextual framework on which test is mapped to a latent space that encodes the meaning and most likely next sequences of text rather than just raw most likely sequence of text as seen in training data That's different than understanding, or knowing. The encoded meaning is not accessible to the LLM, but the human it's presented to. An LLM cannot know about things it has or has not stored in source data, because it is not actually informed by the information processed. You do need proper cognition to know if information is in source data, because reasoning about information strictly requires interpretation and understanding intent, otherwise it's just data.
      - jncfhnb 3y ago
        This simply isn’t true. It’s true that they’re not great at doing this, but they can and do do it and easily demonstrated by chat gpt actively telling you about things it does not know. It does not have cognition. And yet it can do this. Ergo it does not need cognition to do this. LLMs have easily demonstrated reasoning capabilities. The encoded meaning is very clearly explored by the model through its tuned framework and I think it’s ridiculous to pretend otherwise. It’s not stepping through reflection steps in a way that is familiar to humans, but it absolutely is running through semantically defined pattern processing steps. And “known” vs “not known” is one such pattern.
        jijijijij 3y ago
        Yes, I am sure they have implemented an heuristic for this. It can't do this in all cases, so ergo it does need cognition for the category of problem. At least by your reasoning. There is a difference between convincing you in interaction and proving theoretical capabilities. You also don't know, if your experience is down to inherent capabilities of the LLM, or manually implemented algorithms, when you use ChatGPT. We're arguing about different things, or about different levels of abstraction. Have fun with ChatGPT.
        jncfhnb 3y ago
        I run mixtral on my own hardware so I know pretty well what’s the output of the LLM. I’m not sure why you felt compelled to add that jab in there either way. Your argument seems to be that cognition is required to do this perfectly even though things with cognition frequently get this wrong and the bar of the conversation was whether it could be done at all. So I think it seems to be a pretty bad argument.

They do this already all the time. Probably the majority of the time. The problem is that a minority of the time is still very problematic. How do they do this? The same as they do now. The most likely token is that the bot doesn’t know the answer. Which is a behavior emergent from its tuning. I don’t get how people believe it can parse complex questions to produce novel ideas but can’t defer to saying “idk” when the answer isn’t known.

jijijijij 3y ago
So, you are basing your assessment on your gut feel and personal impression with ChatGPT? Maybe you should tone down the spice a bit, then. Unless you can explain how an actual understanding emerges within an LLM, you can't explain how it would answer the question definitely - it doesn't know, if it does, or does not know something. Generally speaking.
- jncfhnb 3y ago
  I’m basing it on my being a data scientist who does this. > Unless you can explain how an actual understanding emerges within an LLM Tuning creates the contextual framework on which test is mapped to a latent space that encodes the meaning and most likely next sequences of text rather than just raw most likely sequence of text as seen in training data. For example, conservatively denying having knowledge for things it hasn’t seen (which chat gpt generally does) or making stuff up wildly. > you can't explain how it would answer the question definitely Of course not. It’s a random behavior. It has plenty of flaws.
  - jijijijij 3y ago
    > You don’t need proper cognition to identify that the answer is not stored in source data That's the original argument. > Tuning creates the contextual framework on which test is mapped to a latent space that encodes the meaning and most likely next sequences of text rather than just raw most likely sequence of text as seen in training data That's different than understanding, or knowing. The encoded meaning is not accessible to the LLM, but the human it's presented to. An LLM cannot know about things it has or has not stored in source data, because it is not actually informed by the information processed. You do need proper cognition to know if information is in source data, because reasoning about information strictly requires interpretation and understanding intent, otherwise it's just data.
    - jncfhnb 3y ago
      This simply isn’t true. It’s true that they’re not great at doing this, but they can and do do it and easily demonstrated by chat gpt actively telling you about things it does not know. It does not have cognition. And yet it can do this. Ergo it does not need cognition to do this. LLMs have easily demonstrated reasoning capabilities. The encoded meaning is very clearly explored by the model through its tuned framework and I think it’s ridiculous to pretend otherwise. It’s not stepping through reflection steps in a way that is familiar to humans, but it absolutely is running through semantically defined pattern processing steps. And “known” vs “not known” is one such pattern.
      - jijijijij 3y ago
        Yes, I am sure they have implemented an heuristic for this. It can't do this in all cases, so ergo it does need cognition for the category of problem. At least by your reasoning. There is a difference between convincing you in interaction and proving theoretical capabilities. You also don't know, if your experience is down to inherent capabilities of the LLM, or manually implemented algorithms, when you use ChatGPT. We're arguing about different things, or about different levels of abstraction. Have fun with ChatGPT.
        jncfhnb 3y ago
        I run mixtral on my own hardware so I know pretty well what’s the output of the LLM. I’m not sure why you felt compelled to add that jab in there either way. Your argument seems to be that cognition is required to do this perfectly even though things with cognition frequently get this wrong and the bar of the conversation was whether it could be done at all. So I think it seems to be a pretty bad argument.