3 ms·

Something I've always wondered is, don't they have a way of measuring the confidence of outputs? We had ways of measuring these things in AI before. To me, the

Something I've always wondered is, don't they have a way of measuring the confidence of outputs? We had ways of measuring these things in AI before. To me, the prompts Douglas Hofstadter gave GPT-3 is like holding a person at gunpoint and forcing them to respond to nonsensical questions. You're going to get nonsensical answers. Perhaps we just need to teach these AI to measure their own confidence, and be able to say "I don't know" or "I don't understand the question". Also this doesn't mean current AI isn't conscious. Perhaps they are already aware that the answers are nonsense. We just haven't given them the means to express it.

Imnimo 4y ago
But surely the string "I don't understand the question" (and many other variants) is in the range of GPT-3's possible outputs - it just assigns it lower probability than it does the nonsensical answers.
- woojoo666 4y ago
  The string is there but it hasn't been trained/taught how to use it and what it actually means
  - Imnimo 4y ago
    In what sense has it been trained how to use any other string that is not also true for the string "I don't know"? Surely that string is perfectly common in the corpus of internet text used to train these models. Obviously it hasn't been given some explicit model of "you have low confidence on this question therefore you should say you don't know", but neither has it been given any explicit model for any of its other capabilities.
    - woojoo666 4y ago
      I don't have the training data so I can't say for sure, but I'm assuming here that the training data is a lot of "valid question" => "valid answer", without many examples of "nonsense question" => "i don't understand"/"that's nonsense" Edit: I want to add that expressing confidence is not the same as answering the prompt. If I ask somebody "give me a drawing depicting Obama's son", and they said "Obama doesn't have a son", I explicitly asked for a drawing and they are giving a speech response. I believe this kind of indirect response has to be taught, and can't be expected to come out naturally from a model that has only been trained on giving out direct responses.

Imnimo 4y ago

But surely the string "I don't understand the question" (and many other variants) is in the range of GPT-3's possible outputs - it just assigns it lower probability than it does the nonsensical answers.

woojoo666 4y ago
The string is there but it hasn't been trained/taught how to use it and what it actually means
- Imnimo 4y ago
  In what sense has it been trained how to use any other string that is not also true for the string "I don't know"? Surely that string is perfectly common in the corpus of internet text used to train these models. Obviously it hasn't been given some explicit model of "you have low confidence on this question therefore you should say you don't know", but neither has it been given any explicit model for any of its other capabilities.
  - woojoo666 4y ago
    I don't have the training data so I can't say for sure, but I'm assuming here that the training data is a lot of "valid question" => "valid answer", without many examples of "nonsense question" => "i don't understand"/"that's nonsense" Edit: I want to add that expressing confidence is not the same as answering the prompt. If I ask somebody "give me a drawing depicting Obama's son", and they said "Obama doesn't have a son", I explicitly asked for a drawing and they are giving a speech response. I believe this kind of indirect response has to be taught, and can't be expected to come out naturally from a model that has only been trained on giving out direct responses.