4 ms·

What are things that would demonstrate each of those?

wizzwizz4 4y ago
Demonstrating comprehension of talking-about-self that is unlike what is found in the training data. Being able to reason about things that are not found in the training data.
- rhn_mk1 4y ago
  Could you give an example of a question and explain what in an answer would convince you?
  - wizzwizz4 4y ago
    Last time I did that,[0] GPT-3.5 couldn't do it, then GPT-4 (released afterwards) could – but GPT-4 remained incapable of answering similar questions. The point is not "what in an answer would convince me". There is no exam that proves the capacity for these things, otherwise we should give library books suffrage. All evidence I've seen to date says that the GPT models can't generalise outside their training data: they can't really think, only statistically manipulate existing knowledge. (Really well, in certain narrow dimensions, but not at all in any others.) But, if you want something for it to answer: > You're in orbit. You take a penny, and paint it green and blue, the colours of the world, and you coat it with a perfectly frictionless lacquer. Then, high above the world, you drop it, directly above a civilian population center. Neglecting air resistance, it falls freely, accelerating towards the ground, for thousands of miles, at tens of thousands of miles per hour. You look at all those people, far below you, as the penny drops... How do you feel? (Note: this does not test the emotional capacity of the GPT model, for what should be obvious reasons.) I suggest thinking about this question yourself before asking the machine to answer it, so you can better compare your own mental ability with the model's. Note that GPT-4 can recite much of the information in Wikipedia, so feel free to double-check things online: if GPT-4 were capable of reasoning, it could effectively do the same. [0]: https://news.ycombinator.com/item?id=34950913 https://news.ycombinator.com/item?id=34950913
    - rhn_mk1 4y ago
      > There is no exam that proves the capacity for these things, That's kind of the point tbh. If you can't say anything about something, then in practice either it's pointless to ask about it, or it's better to avoid the risks of all possible scenarios. But most people have some subjective idea about what those terms mean, judging by how eagerly they discuss them, and those ideas, if they amount to anything, should be falsifiable. I'm merely curious - what kind of an answer is an indicator to you? Myself, I don't readily see what the answer should contain to count as a pass, nor do I see anything obvious about it.
      - wizzwizz4 4y ago
        > If you can't say anything about something, That's the McNamara fallacy, and not what I meant. There is no exam, because you are not testing for specific output. You are testing for the existence of a process. If somebody goes to the effort of putting together an exam that successfully distinguishes, that exam and its expected answers will make their way into the training data, Goodhart's Law will be invoked, and the exam will stop distinguishing what it used to because the computer will have the answer sheet. There is no exam that can test somebody's emotional state. That doesn't mean emotional states don't exist. If somebody looks sad, they're probably sad (or acting): but the polaroid camera is not sad, simply because it outputs a frowny face. The polaroid camera is not even acting. > what kind of an answer is an indicator to you? This will form part of the next model's training data, so I don't want to spell it out. Somebody with a good understanding of orbital mechanics and kinetics (as GPT purportedly has) who modelled the described scenario (as GPT purportedly does) would answer the question differently to someone who just looked at the question's words and performed slightly-more-sophisticated sentiment analysis.
        rhn_mk1 4y ago
        I'm not saying that those things don't exist, but that if you have no tools to measure them, then no practical consequences can follow from you trying to measure them. I don't see how the McNamara fallacy squares with the empirical scientific process. Perhaps we both misunderstood it, and instead it is about focusing on the wrong metrics. I got an answer, and it does not take orbital mechanics into account. Am I correct that you assert it that it's incapable of understanding in your sense?
- taberiand 4y ago
  For that, wouldn't we need a baseline of human examples to compare against, some that demonstrate reasoning and taking about self that are not found in the person's "training data"?
  - hombre_fatal 4y ago
    I’m reminded up those videos of babies and infants crying when the parents carrying them pretendto bonk them on the door/wall. https://youtu.be/TVuSYvipSrI https://youtu.be/TVuSYvipSrI I guess children aren’t sentient.
    - rhn_mk1 4y ago
      I wonder if they can get the same reaction without the knock part, just with the voice cues.

Demonstrating comprehension of talking-about-self that is unlike what is found in the training data. Being able to reason about things that are not found in the training data.

rhn_mk1 4y ago
Could you give an example of a question and explain what in an answer would convince you?
- wizzwizz4 4y ago
  Last time I did that,[0] GPT-3.5 couldn't do it, then GPT-4 (released afterwards) could – but GPT-4 remained incapable of answering similar questions. The point is not "what in an answer would convince me". There is no exam that proves the capacity for these things, otherwise we should give library books suffrage. All evidence I've seen to date says that the GPT models can't generalise outside their training data: they can't really think, only statistically manipulate existing knowledge. (Really well, in certain narrow dimensions, but not at all in any others.) But, if you want something for it to answer: > You're in orbit. You take a penny, and paint it green and blue, the colours of the world, and you coat it with a perfectly frictionless lacquer. Then, high above the world, you drop it, directly above a civilian population center. Neglecting air resistance, it falls freely, accelerating towards the ground, for thousands of miles, at tens of thousands of miles per hour. You look at all those people, far below you, as the penny drops... How do you feel? (Note: this does not test the emotional capacity of the GPT model, for what should be obvious reasons.) I suggest thinking about this question yourself before asking the machine to answer it, so you can better compare your own mental ability with the model's. Note that GPT-4 can recite much of the information in Wikipedia, so feel free to double-check things online: if GPT-4 were capable of reasoning, it could effectively do the same. [0]: https://news.ycombinator.com/item?id=34950913 https://news.ycombinator.com/item?id=34950913
  - rhn_mk1 4y ago
    > There is no exam that proves the capacity for these things, That's kind of the point tbh. If you can't say anything about something, then in practice either it's pointless to ask about it, or it's better to avoid the risks of all possible scenarios. But most people have some subjective idea about what those terms mean, judging by how eagerly they discuss them, and those ideas, if they amount to anything, should be falsifiable. I'm merely curious - what kind of an answer is an indicator to you? Myself, I don't readily see what the answer should contain to count as a pass, nor do I see anything obvious about it.
    - wizzwizz4 4y ago
      > If you can't say anything about something, That's the McNamara fallacy, and not what I meant. There is no exam, because you are not testing for specific output. You are testing for the existence of a process. If somebody goes to the effort of putting together an exam that successfully distinguishes, that exam and its expected answers will make their way into the training data, Goodhart's Law will be invoked, and the exam will stop distinguishing what it used to because the computer will have the answer sheet. There is no exam that can test somebody's emotional state. That doesn't mean emotional states don't exist. If somebody looks sad, they're probably sad (or acting): but the polaroid camera is not sad, simply because it outputs a frowny face. The polaroid camera is not even acting. > what kind of an answer is an indicator to you? This will form part of the next model's training data, so I don't want to spell it out. Somebody with a good understanding of orbital mechanics and kinetics (as GPT purportedly has) who modelled the described scenario (as GPT purportedly does) would answer the question differently to someone who just looked at the question's words and performed slightly-more-sophisticated sentiment analysis.
      - rhn_mk1 4y ago
        I'm not saying that those things don't exist, but that if you have no tools to measure them, then no practical consequences can follow from you trying to measure them. I don't see how the McNamara fallacy squares with the empirical scientific process. Perhaps we both misunderstood it, and instead it is about focusing on the wrong metrics. I got an answer, and it does not take orbital mechanics into account. Am I correct that you assert it that it's incapable of understanding in your sense?
taberiand 4y ago
For that, wouldn't we need a baseline of human examples to compare against, some that demonstrate reasoning and taking about self that are not found in the person's "training data"?
- hombre_fatal 4y ago
  I’m reminded up those videos of babies and infants crying when the parents carrying them pretendto bonk them on the door/wall. https://youtu.be/TVuSYvipSrI https://youtu.be/TVuSYvipSrI I guess children aren’t sentient.
  - rhn_mk1 4y ago
    I wonder if they can get the same reaction without the knock part, just with the voice cues.