3 ms·

Third time today I've seen someone remark on the _confidence_ of ChatGPT responses. Indeed it is remarkable!

• scrollaway 4y ago
  ChatGPT doesn't really have a concept of confidence. Everything sounds hyper-confident, unless you tell it to sound otherwise. But... I think this is not necessarily an unsolvable problem within GPT itself. Even just with ChatGPT you can try to introduce the concept of confidence and get it to assign confidence ratings to its own answers. I've been experimenting a lot with that. But ChatGPT is crippled from the get-go: its assistant prompt severely pushes it towards confidence, which exacerbates all this.
• PeterisP 4y ago
  I think that this is an artifact of the training data. In general, we train models on publicly available text, which is generally written by people when/if they became sufficiently confident about something; any discussions where people talk about things they don't know (and admit it) are mostly private and thus only a tiny fraction of the available training data. So the model training process is looking at a filtered world in which everybody talks (writes) with confidence all the time unless they are asking a question, and it's hard for it to learn a substantially different mode of talking.