4 ms·
> These things have no concept of correctness or error. They have no concept of true or false. Indeed, they have no concept of concepts, or indeed of anything e
by xg15 7mo ago
> These things have no concept of correctness or error. They have no concept of true or false. Indeed, they have no concept of concepts, or indeed of anything else.
Is this true? You can question the humanizing term "concept", but the entire process of pretraining and then RLHF optimizing is essentially about establishing a standard of "good" vs "bad" for the model.
- n4r9 7mo agoGood vs bad is an orthogonal spectrum to true vs false. LLMs are trained to be convincing, not correct.
- chromacity 7mo agoThey are trained and evaluated on correctness benchmarks. But correctness on benchmark questions is only loosely coupled to correctness outside the benchmark, in part because LLMs aren't grounded to the same biological reality as humans. You can't easily convince an average person to cut off their own hand and this has little to do with higher-level thought. In contrast, it only takes a bit of creativity to convince an LLM to say or do almost anything.