4 ms·
The intractable problem, here, is that “LLMs are good historians” is a nearly useless heuristic. I’m not a historian. I don’t speak old spanish. I am not a dom
by satisfice 2y ago
The intractable problem, here, is that “LLMs are good historians” is a nearly useless heuristic.
I’m not a historian. I don’t speak old spanish. I am not a domain expert at all. I can’t do what the author of this post can do: expertly review the work of an LLM in his field.
My expertise is in software testing, and I can report that LLMs sometimes have reasonable testing ideas— but that doesn’t mean they are safe and effective when used for that purpose by an amateur.
Despite what the author writes, I cannot use an LLM to get good information about history.
- amelius 2y agoYou __can__ get good information from an LLM, however you just have to backtrack every once in a while because the information turned out to be false.
- userbinator 2y agohowever you just have to backtrack every once in a while because the information turned out to be false. The problem is, how do you know? I've seen developers go completely off-course just from bad search engine results and one did admit he felt something wasn't right but kept going because he didn't know better; now imagine he's being told by a very confident but incorrect LLM, and you can see how hazardous that'll be. "You don't know what you don't know."
- sebmellen 2y agoUnless you have a very good understanding of the system you’re working on or the tools you’re using, it’s very possible to get knee deep in crap without knowing it. That’s one of the biggest risks of using LLMs as assistants.
- simonw 2y agoYou need to develop skills like critical thinking, metacognition, analytical reasoning - being able to get to a robust mental model from a bunch of different inputs, some of which may even contradict each other.
- userbinator 2y agoPeople were generally already horrible at that before AI.
- deleted 2y ago[deleted]
- mvdtnz 2y agoAnd therein lies the problem - if you're not already an expert there's no way to tell when is the right moment to backtrack.
- nithril 2y agoThe exact definition of a useful heuristic, "good enough"
- simonw 2y agoThis is similar to the problem with some of the things people have been doing with o1 and o3. I've seen people share "PhD level" results from them... but if I don't have a PhD myself in that subject it's almost impossible for me to evaluate their output and spot if it makes sense or not. I get a ton of value out of LLMs as a programmer partly because I have 20+ years of programming experience, so it's trivial for me to spot when they are doing "good" work as opposed to making dumb mistakes. I can't credibly evaluate their higher level output in other disciplines at all.
- xigency 2y agoThis begs the question, is this wave of LLM AI anything more than a fancy mirror? They're certainly very good at agreeing with people and following along, but, as many have noted, not really useful for anything acting on their own.