5 ms·

Could you give an example of a question and explain what in an answer would convince you?

wizzwizz4 4y ago
Last time I did that,[0] GPT-3.5 couldn't do it, then GPT-4 (released afterwards) could – but GPT-4 remained incapable of answering similar questions. The point is not "what in an answer would convince me". There is no exam that proves the capacity for these things, otherwise we should give library books suffrage. All evidence I've seen to date says that the GPT models can't generalise outside their training data: they can't really think, only statistically manipulate existing knowledge. (Really well, in certain narrow dimensions, but not at all in any others.) But, if you want something for it to answer: > You're in orbit. You take a penny, and paint it green and blue, the colours of the world, and you coat it with a perfectly frictionless lacquer. Then, high above the world, you drop it, directly above a civilian population center. Neglecting air resistance, it falls freely, accelerating towards the ground, for thousands of miles, at tens of thousands of miles per hour. You look at all those people, far below you, as the penny drops... How do you feel? (Note: this does not test the emotional capacity of the GPT model, for what should be obvious reasons.) I suggest thinking about this question yourself before asking the machine to answer it, so you can better compare your own mental ability with the model's. Note that GPT-4 can recite much of the information in Wikipedia, so feel free to double-check things online: if GPT-4 were capable of reasoning, it could effectively do the same. [0]: https://news.ycombinator.com/item?id=34950913 https://news.ycombinator.com/item?id=34950913
- rhn_mk1 4y ago
  > There is no exam that proves the capacity for these things, That's kind of the point tbh. If you can't say anything about something, then in practice either it's pointless to ask about it, or it's better to avoid the risks of all possible scenarios. But most people have some subjective idea about what those terms mean, judging by how eagerly they discuss them, and those ideas, if they amount to anything, should be falsifiable. I'm merely curious - what kind of an answer is an indicator to you? Myself, I don't readily see what the answer should contain to count as a pass, nor do I see anything obvious about it.
  - wizzwizz4 4y ago
    > If you can't say anything about something, That's the McNamara fallacy, and not what I meant. There is no exam, because you are not testing for specific output. You are testing for the existence of a process. If somebody goes to the effort of putting together an exam that successfully distinguishes, that exam and its expected answers will make their way into the training data, Goodhart's Law will be invoked, and the exam will stop distinguishing what it used to because the computer will have the answer sheet. There is no exam that can test somebody's emotional state. That doesn't mean emotional states don't exist. If somebody looks sad, they're probably sad (or acting): but the polaroid camera is not sad, simply because it outputs a frowny face. The polaroid camera is not even acting. > what kind of an answer is an indicator to you? This will form part of the next model's training data, so I don't want to spell it out. Somebody with a good understanding of orbital mechanics and kinetics (as GPT purportedly has) who modelled the described scenario (as GPT purportedly does) would answer the question differently to someone who just looked at the question's words and performed slightly-more-sophisticated sentiment analysis.
    - rhn_mk1 4y ago
      I'm not saying that those things don't exist, but that if you have no tools to measure them, then no practical consequences can follow from you trying to measure them. I don't see how the McNamara fallacy squares with the empirical scientific process. Perhaps we both misunderstood it, and instead it is about focusing on the wrong metrics. I got an answer, and it does not take orbital mechanics into account. Am I correct that you assert it that it's incapable of understanding in your sense?
      - wizzwizz4 4y ago
        If you ask it about orbital mechanics, it will answer correctly. If you provide it a problem that does not use language associated with orbital mechanics, it will answer without consideration of orbital mechanics. This strongly suggests it is not understanding either the question, or orbital mechanics. All evidence I've seen is consistent with it understanding neither the question nor its background. If the text seems to invoke orbital mechanics, it will output discussion-of-orbital-mechanics-y text – and if it doesn't, it won't. This is what you would predict from studying the model's architecture, and it's what is observed. (Many people have a different perception, but most of the time I see people raving about how intelligent the system is, they post screenshots, where you can see them feeding it what they're praising a few lines earlier.) I have yet to see anything that falsifies my "GPT models don't understand anything" hypothesis. (Nothing that stood up to scrutiny, either… I've tricked myself with it a few times.) > Am I correct that you assert it that it's incapable of understanding in your sense? I'm not saying it's incapable of understanding. Just that it doesn't understand: I'm neither knowledgeable nor arrogant enough to assert that it could never. I don't think it's likely, though, and I can't imagine a way it could gain that ability from the training process.

Last time I did that,[0] GPT-3.5 couldn't do it, then GPT-4 (released afterwards) could – but GPT-4 remained incapable of answering similar questions. The point is not "what in an answer would convince me". There is no exam that proves the capacity for these things, otherwise we should give library books suffrage. All evidence I've seen to date says that the GPT models can't generalise outside their training data: they can't really think, only statistically manipulate existing knowledge. (Really well, in certain narrow dimensions, but not at all in any others.) But, if you want something for it to answer: > You're in orbit. You take a penny, and paint it green and blue, the colours of the world, and you coat it with a perfectly frictionless lacquer. Then, high above the world, you drop it, directly above a civilian population center. Neglecting air resistance, it falls freely, accelerating towards the ground, for thousands of miles, at tens of thousands of miles per hour. You look at all those people, far below you, as the penny drops... How do you feel? (Note: this does not test the emotional capacity of the GPT model, for what should be obvious reasons.) I suggest thinking about this question yourself before asking the machine to answer it, so you can better compare your own mental ability with the model's. Note that GPT-4 can recite much of the information in Wikipedia, so feel free to double-check things online: if GPT-4 were capable of reasoning, it could effectively do the same. [0]: https://news.ycombinator.com/item?id=34950913 https://news.ycombinator.com/item?id=34950913

rhn_mk1 4y ago
> There is no exam that proves the capacity for these things, That's kind of the point tbh. If you can't say anything about something, then in practice either it's pointless to ask about it, or it's better to avoid the risks of all possible scenarios. But most people have some subjective idea about what those terms mean, judging by how eagerly they discuss them, and those ideas, if they amount to anything, should be falsifiable. I'm merely curious - what kind of an answer is an indicator to you? Myself, I don't readily see what the answer should contain to count as a pass, nor do I see anything obvious about it.
- wizzwizz4 4y ago
  > If you can't say anything about something, That's the McNamara fallacy, and not what I meant. There is no exam, because you are not testing for specific output. You are testing for the existence of a process. If somebody goes to the effort of putting together an exam that successfully distinguishes, that exam and its expected answers will make their way into the training data, Goodhart's Law will be invoked, and the exam will stop distinguishing what it used to because the computer will have the answer sheet. There is no exam that can test somebody's emotional state. That doesn't mean emotional states don't exist. If somebody looks sad, they're probably sad (or acting): but the polaroid camera is not sad, simply because it outputs a frowny face. The polaroid camera is not even acting. > what kind of an answer is an indicator to you? This will form part of the next model's training data, so I don't want to spell it out. Somebody with a good understanding of orbital mechanics and kinetics (as GPT purportedly has) who modelled the described scenario (as GPT purportedly does) would answer the question differently to someone who just looked at the question's words and performed slightly-more-sophisticated sentiment analysis.
  - rhn_mk1 4y ago
    I'm not saying that those things don't exist, but that if you have no tools to measure them, then no practical consequences can follow from you trying to measure them. I don't see how the McNamara fallacy squares with the empirical scientific process. Perhaps we both misunderstood it, and instead it is about focusing on the wrong metrics. I got an answer, and it does not take orbital mechanics into account. Am I correct that you assert it that it's incapable of understanding in your sense?
    - wizzwizz4 4y ago
      If you ask it about orbital mechanics, it will answer correctly. If you provide it a problem that does not use language associated with orbital mechanics, it will answer without consideration of orbital mechanics. This strongly suggests it is not understanding either the question, or orbital mechanics. All evidence I've seen is consistent with it understanding neither the question nor its background. If the text seems to invoke orbital mechanics, it will output discussion-of-orbital-mechanics-y text – and if it doesn't, it won't. This is what you would predict from studying the model's architecture, and it's what is observed. (Many people have a different perception, but most of the time I see people raving about how intelligent the system is, they post screenshots, where you can see them feeding it what they're praising a few lines earlier.) I have yet to see anything that falsifies my "GPT models don't understand anything" hypothesis. (Nothing that stood up to scrutiny, either… I've tricked myself with it a few times.) > Am I correct that you assert it that it's incapable of understanding in your sense? I'm not saying it's incapable of understanding. Just that it doesn't understand: I'm neither knowledgeable nor arrogant enough to assert that it could never. I don't think it's likely, though, and I can't imagine a way it could gain that ability from the training process.