5 ms·

It feels like this is optimizing for safety rather than realism. The davinci models today would have a much better shot at passing the Turing test.

by jcims 4y ago

TaylorAlexander 4y ago
As long as you don't ask them what happened in the news last week, or this year.
- minimaxir 4y ago
  Notably, the bot is upfront about its timeliness and lack thereof.
  - speedgoose 4y ago
    It would repeat to me that he doesn't know what happened in 1998 because its training data stopped in 2021.
  - TaylorAlexander 4y ago
    Well to pass a turing test it would have to say "hmm I was in a coma for 18 months and after waking up I ignored all current news and came here to take this test with you." My real point is that large language models lack certain real world capabilities, like internal motivations and a life that advances every day, and this is one way we can tell them apart from a human if we did a real life turing test. You could ask one facts about its dreams and motivations, and where it hopes to be in 5 years, and it could create a plausible story, but it would all have to be made up, and at some point you could uncover inconsistencies. This is just off the top of my head, but I am sure there are other issues. I don't think any of this will be solved until we have some kind of agent with motivations, which only uses a large language model as part of its cognition. Until then they are just repeating plausible sentences, but they are not grounded in a single motivated agent.
    - GaggiX 4y ago
      What is OpenAI is doing right here is more difficult than passing the Turing test, the Turing test rewards machines that are indistinguishable from human beings, that's a free goal when you trained the network on text written by humans, by fitting the distribution it will behave like a human. The more difficult task that OpenAI is trying to solve is to align the NN to acknowledge that it's a GPT model, it has no way to browse the Net and it has limited knowledge about events after 2021, this is not free, only a really limited subset of the dataset was written from this prospective.
      - TaylorAlexander 4y ago
        It feels like you haven't read my comment. You cannot solve a turing test using a large language model on its own. The language model can spit out human-like responses to questions you may ask, but it does not have a genuine internal life experience. What this means is that if you ask it where it hopes to be in five years, it could just as easily say "I hope to be a manager at my engineering firm" and then it could say "I hope to be running my own bakery". But it does not have genuine goals, it's just inventing plausible-sounding goals. You can tell this is happening if you have a long enough conversation, because things will be inconsistent. the language model does not actually want to run a bakery, it merely "wants" to produce plausible sounding text. You literally cannot solve this problem by training on more text or more humanlike text. You need an "agent" that has genuine motivations and some concept of an internal experience. There is a great paper on the limits of large language models that is worth reading if you'd like to learn more: https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922
        jeremyjh 4y ago
        > You need an "agent" that has genuine motivations and some concept of an internal experience. Don’t you just need a more complete model of such an agent? I agree you can’t get such a thing by training on text.
        GaggiX 4y ago
        You can beat the Turing test with a large generative model, you don't need the model to feel emotions, have genuine goals or anything other than being indistinguishable from a human being, If you build an agent that has and quote: "genuine motivations and some concept of an internal experience" than you will fail the Turing test because the agent will probably recognize the fact it's not human, just as a human recognizes the fact that it's not an AI, that's why the Turing test is so criticized, you're testing how human-like the model is, and that literally means how well it fits the distribution it was trained on; the technology is already here, there's no reason to scale the models further, also context windows can big enough to fool humans, we can see this happening also in other fields than text generation like image generation, sometimes it seems impossible to distinguish the output of these big generative models from one created by a human. The actual goal of today research is alignment, an AI that acts like an human is not useful, it should behave like an AI, that's why OpenAI has spent so much time on aligning its AI to behave like one and not like a human.
boredemployee 4y ago
Yep. The answers are looking more like a bot than like a human, which is good or bad depending on the application.

It feels like this is optimizing for safety rather than realism. The davinci models today would have a much better shot at passing the Turing test.

by jcims 4y ago

It feels like this is optimizing for safety rather than realism. The davinci models today would have a much better shot at passing the Turing test.

TaylorAlexander 4y ago

As long as you don't ask them what happened in the news last week, or this year.

minimaxir 4y ago
Notably, the bot is upfront about its timeliness and lack thereof.
- speedgoose 4y ago
  It would repeat to me that he doesn't know what happened in 1998 because its training data stopped in 2021.
- TaylorAlexander 4y ago
  Well to pass a turing test it would have to say "hmm I was in a coma for 18 months and after waking up I ignored all current news and came here to take this test with you." My real point is that large language models lack certain real world capabilities, like internal motivations and a life that advances every day, and this is one way we can tell them apart from a human if we did a real life turing test. You could ask one facts about its dreams and motivations, and where it hopes to be in 5 years, and it could create a plausible story, but it would all have to be made up, and at some point you could uncover inconsistencies. This is just off the top of my head, but I am sure there are other issues. I don't think any of this will be solved until we have some kind of agent with motivations, which only uses a large language model as part of its cognition. Until then they are just repeating plausible sentences, but they are not grounded in a single motivated agent.
  - GaggiX 4y ago
    What is OpenAI is doing right here is more difficult than passing the Turing test, the Turing test rewards machines that are indistinguishable from human beings, that's a free goal when you trained the network on text written by humans, by fitting the distribution it will behave like a human. The more difficult task that OpenAI is trying to solve is to align the NN to acknowledge that it's a GPT model, it has no way to browse the Net and it has limited knowledge about events after 2021, this is not free, only a really limited subset of the dataset was written from this prospective.
    - TaylorAlexander 4y ago
      It feels like you haven't read my comment. You cannot solve a turing test using a large language model on its own. The language model can spit out human-like responses to questions you may ask, but it does not have a genuine internal life experience. What this means is that if you ask it where it hopes to be in five years, it could just as easily say "I hope to be a manager at my engineering firm" and then it could say "I hope to be running my own bakery". But it does not have genuine goals, it's just inventing plausible-sounding goals. You can tell this is happening if you have a long enough conversation, because things will be inconsistent. the language model does not actually want to run a bakery, it merely "wants" to produce plausible sounding text. You literally cannot solve this problem by training on more text or more humanlike text. You need an "agent" that has genuine motivations and some concept of an internal experience. There is a great paper on the limits of large language models that is worth reading if you'd like to learn more: https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922
      - jeremyjh 4y ago
        > You need an "agent" that has genuine motivations and some concept of an internal experience. Don’t you just need a more complete model of such an agent? I agree you can’t get such a thing by training on text.
      - GaggiX 4y ago
        You can beat the Turing test with a large generative model, you don't need the model to feel emotions, have genuine goals or anything other than being indistinguishable from a human being, If you build an agent that has and quote: "genuine motivations and some concept of an internal experience" than you will fail the Turing test because the agent will probably recognize the fact it's not human, just as a human recognizes the fact that it's not an AI, that's why the Turing test is so criticized, you're testing how human-like the model is, and that literally means how well it fits the distribution it was trained on; the technology is already here, there's no reason to scale the models further, also context windows can big enough to fool humans, we can see this happening also in other fields than text generation like image generation, sometimes it seems impossible to distinguish the output of these big generative models from one created by a human. The actual goal of today research is alignment, an AI that acts like an human is not useful, it should behave like an AI, that's why OpenAI has spent so much time on aligning its AI to behave like one and not like a human.

boredemployee 4y ago

Yep. The answers are looking more like a bot than like a human, which is good or bad depending on the application.