3 ms·
> I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence. This might be true in a strict sen
by wkirby 2y ago
> I do think that SOTA LLMs like GPT-4o perform about as good as high school graduates in America with average intelligence.
This might be true in a strict sense, but I think it's really, really important to consider the uses of LLMs vs a high-school graduate. LLMs are confidently wrong (and confidently correct) with the exact same measure, and in many ways they are presented to users as unimpeachable.
If I ask an average person to do a medium-complex logic problem, my human brain discounts their answer because I've been socialized to believe that humans are bad at logic. I will take any answer I'm given with usually appropriate skepticism.
LLMs, on the other hand, are on the computer: an interface I've been socialized to believe is always correct on matters of math and logic. That's what it is, a logic machine. Second guessing the computer on matters of logic and arithmetic almost always result in me realizing my puny human mind has done something wrong.
To me, this directly contradicts your conclusion: LLMs are mostly only capable of misleading large portions of the population.
- pishpash 2y agoWould be good to put equivalent grades on LLM's then. Instead of GPT-4o, it's GPT-11th grade.
- 1oooqooq 2y agothat's in the wrong directing of what the comment said
- steveBK123 2y agoThe problem is that some % of the time they answer at an 11th grade level, and some % of the time they answer at a 4th grade level, with no way of knowing beforehand. The only way of knowing is after the fact, when the prompter has the knowledge to discern whether the answer is correct or not. In which case why are we asking questions we already know the answer of, other than these are toys and fun? I really wonder what happen when the novelty wears off in a few years, where these things are actually useful.
- Eisenstein 2y agoThis is not inherent in the LLM though. Society will adjust to it after learning some very predictable (and predicted) lessons, just like it always does.
- wkirby 2y agoWill it? Media literacy is in crisis, and we’ve had the printed word for centuries. I’m much more convinced we will continue to believe “magic box cannot lie” than somehow develop a notion for when to trust the coherent nonsense generator.
- nosianu 2y agoAs the other commenter said, and I would extend that to be even more broad. Humanity has developed few abilities to escape from ancient things like group think or propaganda (since long ago in the form of stories and unproven assertions about one's own group and others). That may very well be because there is no advantage - having a cohesive group is more important than "seeing truth". That would mean that there will be no adjustment in the future either. "Truth" does not appear to be all that important to survival and procreation I think.
- 1986 2y agosociety hasn't even fully adjusted to social media yet