4 ms·

The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are maki

by dirkt 3y ago

The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. And in an honest interaction, they will tell you. ChatGPT etc. does not. So basically it acts like a pathological liar (who happen to be right as long as it has been trained on something that comes close enough).

paulddraper 3y ago
Yes, agreed.
deleted 3y ago
[deleted]
cubefox 3y ago
Animals (including humans) predict experiences, while language models only predict text. Text is not very closely linked to reality, while experiences are. So it is not surprising that we (humans) have a better sense of what we know than language models.
TeMPOraL 3y ago
> The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. Scientist? STEM workers? Maybe. People who value truth / consistent world model for its own sake? Sure. Normies? Not so much. It's not that they can't - I think they never learned to pay enough attention. What I mean is, most people tend to say things in confidence, and maybe even believe them, regardless of how they acquired the information. They don't seem to process the distinction between "I read it in a book", "A colleague told me that their colleague heard on the radio that...", etc. They don't even track provenance of the information, which is a critical skill you need to not end up confidently making things up. It's a learnable skill, and I think it's even quite easy to pick it up from osmosis - but someone has to make the person feel that it's important. > So basically it acts like a pathological liar I think it's more of a bullshitter than a liar, in the sense that it doesn't care about truth value of what it says. Lying involve knowing the truth, or at least knowing that the thing you're saying ain't it. Making a mistake involves you thinking you're saying the truth, but actually being wrong about it. Bullshitting is just saying whatever helps you achieve your goal; the truth value of what you say doesn't even enter the picture.

by dirkt 3y ago

paulddraper 3y ago

Yes, agreed.

deleted 3y ago

[deleted]

cubefox 3y ago

Animals (including humans) predict experiences, while language models only predict text. Text is not very closely linked to reality, while experiences are. So it is not surprising that we (humans) have a better sense of what we know than language models.

TeMPOraL 3y ago

> The difference to a person is that most (though not all) people actually have an understanding if they know something, if they guess something, if they are making something up, or if they are outright lying. Which is about the first thing you train during a scientific education. Scientist? STEM workers? Maybe. People who value truth / consistent world model for its own sake? Sure. Normies? Not so much. It's not that they can't - I think they never learned to pay enough attention. What I mean is, most people tend to say things in confidence, and maybe even believe them, regardless of how they acquired the information. They don't seem to process the distinction between "I read it in a book", "A colleague told me that their colleague heard on the radio that...", etc. They don't even track provenance of the information, which is a critical skill you need to not end up confidently making things up. It's a learnable skill, and I think it's even quite easy to pick it up from osmosis - but someone has to make the person feel that it's important. > So basically it acts like a pathological liar I think it's more of a bullshitter than a liar, in the sense that it doesn't care about truth value of what it says. Lying involve knowing the truth, or at least knowing that the thing you're saying ain't it. Making a mistake involves you thinking you're saying the truth, but actually being wrong about it. Bullshitting is just saying whatever helps you achieve your goal; the truth value of what you say doesn't even enter the picture.