2 ms·

"You will even notice how it neatly covers everything I could remember – giving equal weight to each data point and deftly joining them all together. And again

"You will even notice how it neatly covers everything I could remember – giving equal weight to each data point and deftly joining them all together. And again, what ChatGPT here had to offer was utterly – absolutely – false. Like a fluent and practised (but unwise) liar it had contrived an account that fitted only the available information." A fundamental flaw in modern "AI" is that it has no model of reality against which it is checking its (or anyone else's) words. It isn't even lying; it has no concept of the difference between truth and lies, therefore it cannot be made to stop lying, because it isn't even lying, it's just spewing language that sounds good. This makes it a passable tool for sales, but an extremely poor tool for anything which requires accuracy of any kind. It's not that it is trying to be accurate and failing (in which case further work on it might be expected to improve it); it is not attempting to be accurate; indeed the concept of accuracy is not anywhere in its architecture. Vocabulary, grammar, syntax, yes; accuracy or truth, no. It will not get better at being truthful as it is worked on, it will only get better at being persuasive.

deleted 2y ago
[deleted]
ben_w 2y ago
> A fundamental flaw in modern "AI" is that it has no model of reality against which it is checking its (or anyone else's) words. It isn't even lying; it has no concept of the difference between truth and lies, therefore it cannot be made to stop lying, because it isn't even lying, it's just spewing language that sounds good. For the early ones, the reality against which they were checking their words, was their training corpus. Now they've also got RAG and search. In the context of "find a story that fits this description", even the original training corpus was pretty effective. Not perfect, but pretty good… for stuff that was well represented. If all Transformer models could do was vocabulary, grammar, and syntax, they wouldn't have ever been usable as more than a very light form of autocomplete. Even word2vec got analogies (man is to woman what king is to queen, etc.). > it will only get better at being persuasive I kinda agree, unfortunately: it will *also* keep getting better at being persuasive, and this is indeed a bad thing. "LGTM" is easier to test for than "correct" — but to me, that's an example of humans having limits on how well we can model reality, how well we can differentiate between truth and lies, etc.

deleted 2y ago

[deleted]

ben_w 2y ago

> A fundamental flaw in modern "AI" is that it has no model of reality against which it is checking its (or anyone else's) words. It isn't even lying; it has no concept of the difference between truth and lies, therefore it cannot be made to stop lying, because it isn't even lying, it's just spewing language that sounds good. For the early ones, the reality against which they were checking their words, was their training corpus. Now they've also got RAG and search. In the context of "find a story that fits this description", even the original training corpus was pretty effective. Not perfect, but pretty good… for stuff that was well represented. If all Transformer models could do was vocabulary, grammar, and syntax, they wouldn't have ever been usable as more than a very light form of autocomplete. Even word2vec got analogies (man is to woman what king is to queen, etc.). > it will only get better at being persuasive I kinda agree, unfortunately: it will *also* keep getting better at being persuasive, and this is indeed a bad thing. "LGTM" is easier to test for than "correct" — but to me, that's an example of humans having limits on how well we can model reality, how well we can differentiate between truth and lies, etc.