5 ms·

Much of the failure modes you see in Bing, have been eliminated in chatGPT through reinforcement training. It just needs more. Additionally the way it crafted

Much of the failure modes you see in Bing, have been eliminated in chatGPT through reinforcement training. It just needs more. Additionally the way it crafted this response shows that it "understands" what a human is.

simonh 4y ago
The way it crafted its response shows that it has been carefully and rigorously trained to produce text that says it understands what a human is. This is specifically a key criticism of LLMs, so ChatGTP has been coached extensively on it to do a good job. Can we really say that just because a program produces a good text description of something, that it therefore understand that thing in the way that a human does?
- kilgnad 4y ago
  It has not been trained to specifically to produce this text. It was trained randomly on random text from the internet. The production of this text is a side effect of that training. You will note that this text is more detailed and nuanced then what most humans can produce. A human when asked to prove he understands what a human is via text is unlikely do a better job then this. You realize when I asked chatGPT about this I deliberately emphasized it to produce more exact details for it be convincing to you specifically and it added that nuance to the answer in a way that indicated it understood the task. But that is besides the point. I know where you are coming from. You are dismissing the nuance and thinking that this text can easily be reconstructed from a mishmash of existing text in it's training data. To that I urge you you to read this: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ You need to read it to the very end. The beginning and middle is quite trivial but the end is different from all the things you've seen chatGPT perform before. In this case chatGPT cannot have put together a mishmash of text from training data. It was asked to imitate an existing thing and chatGPT did so with startlingly detailed nuance and recursive complexity (you need to read to the end to know what I'm talking about). There is no higher form of evidence in existence for proof of understanding of a thing then emulation of that thing. If you know of any higher form of proof... Let me know. Literally. Is there a question and answer pair that exists such that when you see the the answer, it functions as proof of understanding of the question. If you can't think of a higher form of proof then emulation... then I think you don't have enough information to know if chatGPT understands what you are telling it. You are simply injecting your own bias into the situation. We can tell chatGPT to act in ways that is indistinguishable from a human on a keyboard. I think it will perform remarkably well.
  - simonh 4y ago
    I’ve read that before, it is fascinating. GPT3 was just trained on source text, but ChatGPT has undergone an an additional very rigorous programme of intensive training bootstrapped by human feedback. This has evolved it in order to improve its performance as an information assistant, especially on specific tasks that have proved problematic in the past. One of the things GPT3 has been criticised for is not understanding about people, so you can bet your bottom dollar one of the specific things it’s been coached on is people and how humans work. But it’s still just a language model. It’s just that now it’s been coached how to splurge out better text about people. The underlying limitations of the architecture are still there though. Take the way until recently if you asked it to respond in Danish, it would reply in perfect Danish about how it was an English language model and could not provide responses in Danish. When pointed out that it was replying in Danish, it would adamantly deny it, in Danish. That’s been fixed now. It’s totally mindless, it’s just being coached to provide better and better responses to hide its mindlessness. But papering over the cracks like this with flaw specific training isn’t creating a mind, because it all still works in the same way. This video explains the approach and some of its limitations. https://youtu.be/viJt_DXTfwA https://youtu.be/viJt_DXTfwA
    - kilgnad 4y ago
      Except in the example that you read, it cannot have done what it did without understanding what it was asked.
      - simonh 4y ago
        It depends exactly what you mean by understanding. It has a map of connections between word tokens that allow it to generate an output of word tokens that are useful to us, and which has now just recently been specifically trained for this particular problem domain. So now for these sorts of questions it no longer produces gibberish. None of that works in a way at all analogous to how the human brain processes language, reasons about concepts, or connects ideas beyond just words. It’s not meaning in the sense that we generally understand meaning. If you take a step outside its trained response envelope, it will still fail hilariously generating meaningless drivel. It’s grasp of concepts just isn’t there, it has no concrete foundation. All it knows is word frequency weightings. Don’t get me wrong it’s amazing engineering, it’s probably going to be incredibly useful.

simonh 4y ago

The way it crafted its response shows that it has been carefully and rigorously trained to produce text that says it understands what a human is. This is specifically a key criticism of LLMs, so ChatGTP has been coached extensively on it to do a good job. Can we really say that just because a program produces a good text description of something, that it therefore understand that thing in the way that a human does?

kilgnad 4y ago
It has not been trained to specifically to produce this text. It was trained randomly on random text from the internet. The production of this text is a side effect of that training. You will note that this text is more detailed and nuanced then what most humans can produce. A human when asked to prove he understands what a human is via text is unlikely do a better job then this. You realize when I asked chatGPT about this I deliberately emphasized it to produce more exact details for it be convincing to you specifically and it added that nuance to the answer in a way that indicated it understood the task. But that is besides the point. I know where you are coming from. You are dismissing the nuance and thinking that this text can easily be reconstructed from a mishmash of existing text in it's training data. To that I urge you you to read this: https://www.engraved.blog/building-a-virtual-machine-inside/ https://www.engraved.blog/building-a-virtual-machine-inside/ You need to read it to the very end. The beginning and middle is quite trivial but the end is different from all the things you've seen chatGPT perform before. In this case chatGPT cannot have put together a mishmash of text from training data. It was asked to imitate an existing thing and chatGPT did so with startlingly detailed nuance and recursive complexity (you need to read to the end to know what I'm talking about). There is no higher form of evidence in existence for proof of understanding of a thing then emulation of that thing. If you know of any higher form of proof... Let me know. Literally. Is there a question and answer pair that exists such that when you see the the answer, it functions as proof of understanding of the question. If you can't think of a higher form of proof then emulation... then I think you don't have enough information to know if chatGPT understands what you are telling it. You are simply injecting your own bias into the situation. We can tell chatGPT to act in ways that is indistinguishable from a human on a keyboard. I think it will perform remarkably well.
- simonh 4y ago
  I’ve read that before, it is fascinating. GPT3 was just trained on source text, but ChatGPT has undergone an an additional very rigorous programme of intensive training bootstrapped by human feedback. This has evolved it in order to improve its performance as an information assistant, especially on specific tasks that have proved problematic in the past. One of the things GPT3 has been criticised for is not understanding about people, so you can bet your bottom dollar one of the specific things it’s been coached on is people and how humans work. But it’s still just a language model. It’s just that now it’s been coached how to splurge out better text about people. The underlying limitations of the architecture are still there though. Take the way until recently if you asked it to respond in Danish, it would reply in perfect Danish about how it was an English language model and could not provide responses in Danish. When pointed out that it was replying in Danish, it would adamantly deny it, in Danish. That’s been fixed now. It’s totally mindless, it’s just being coached to provide better and better responses to hide its mindlessness. But papering over the cracks like this with flaw specific training isn’t creating a mind, because it all still works in the same way. This video explains the approach and some of its limitations. https://youtu.be/viJt_DXTfwA https://youtu.be/viJt_DXTfwA
  - kilgnad 4y ago
    Except in the example that you read, it cannot have done what it did without understanding what it was asked.
    - simonh 4y ago
      It depends exactly what you mean by understanding. It has a map of connections between word tokens that allow it to generate an output of word tokens that are useful to us, and which has now just recently been specifically trained for this particular problem domain. So now for these sorts of questions it no longer produces gibberish. None of that works in a way at all analogous to how the human brain processes language, reasons about concepts, or connects ideas beyond just words. It’s not meaning in the sense that we generally understand meaning. If you take a step outside its trained response envelope, it will still fail hilariously generating meaningless drivel. It’s grasp of concepts just isn’t there, it has no concrete foundation. All it knows is word frequency weightings. Don’t get me wrong it’s amazing engineering, it’s probably going to be incredibly useful.