7 ms·
> A fundamental flaw in modern "AI" is that it has no model of reality against which it is checking its (or anyone else's) words. It isn't even lying; it has no
by ben_w 2y ago
> A fundamental flaw in modern "AI" is that it has no model of reality against which it is checking its (or anyone else's) words. It isn't even lying; it has no concept of the difference between truth and lies, therefore it cannot be made to stop lying, because it isn't even lying, it's just spewing language that sounds good.
For the early ones, the reality against which they were checking their words, was their training corpus. Now they've also got RAG and search.
In the context of "find a story that fits this description", even the original training corpus was pretty effective. Not perfect, but pretty good… for stuff that was well represented.
If all Transformer models could do was vocabulary, grammar, and syntax, they wouldn't have ever been usable as more than a very light form of autocomplete.
Even word2vec got analogies (man is to woman what king is to queen, etc.).
> it will only get better at being persuasive
I kinda agree, unfortunately: it will *also* keep getting better at being persuasive, and this is indeed a bad thing. "LGTM" is easier to test for than "correct" — but to me, that's an example of humans having limits on how well we can model reality, how well we can differentiate between truth and lies, etc.