4 ms·
When people say "LLMs don't know they don't know" they're referring to some truism: Fundamentally these people believe LLMs are just pattern matching and intell
by raincole 2y ago
When people say "LLMs don't know they don't know" they're referring to some truism: Fundamentally these people believe LLMs are just pattern matching and intelligence is something beyond that.
So when they see LLMs say "I don't know", their reaction is "it doesn't know what it is talking about", not "wow LLMs actually can tell what it knows!"
Conversely, when they observe a human confidently says something wrong, their reaction is "what a cocky guy" but not "oh so humans are not better than LLMs in this regard."
Because they already believe humans are different and superior, whatever they observe doesn't affect their opinion.
- powerhugs 2y ago> Because they already believe humans are different and superior. This is not a matter of belief, it's a matter of observable fact. Humans reason. LLM predict.
- hakre 2y agoA better way to differentiate might be to say how many watts have been used for an answer. > Humans reason Are you sure? They reason in a way to match others predictions, right? The problem of humans transporting information from one individual to another remains. All reason can effectively only be prediction, as it requires two persons at minimum. And there is a lot of noise to filter. > LLM predict. Well, there is a human that needs to reason with what the LLM has predicted. So from that perspective, it should suffice already for the moment.
- thesumofall 2y agoI‘m not sure we know what the difference between these two is
- djantje 2y agoOk, but are humans not using predictions while reasoning? And are LLM's unable to reason?
- raincole 2y agoSee?
- nurettin 2y agoLooks like Hoomans also terse and lazy.
- mdp2021 2y ago> when they observe a human confidently says something wrong For the nth+1: well wired humans have built their systems trained on double checking and distrusting their own ideas.
- viraptor 2y agoAnd unsurprisingly, the reasoning models do better by introducing self-doubt/reflection and you can improve the response quality by getting multiple answers and auto-judging them. Maps pretty well onto the concept you describe.
- mdp2021 2y agoSure! But we have even more work to do. E.g. -- Natural intelligences learn - after reflection, the mind is updated (hopefully upgraded): we need dynamic Neural Networks in the systems. -- Ideas honing, and the perfectioning of the World Model and cognitive systems behind them, should be iterative and continuous...
- viraptor 2y agohttps://arxiv.org/abs/2502.06049 https://arxiv.org/abs/2502.06049 But for most practical use cases we don't want the persistent memory. We already have opportunistic fact extraction like chatgpt profile building if we want that. We also have graph builders. This mostly covers the private usage. Public version was attempted and just ends up like this: https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
- mdp2021 2y agoThank you! This (LM2: Large Memory Models) is one article I will find the time to read properly, analytically. If you like the metaphors (which this article seems to come close to), you can have a sort of "unconscious" and a "conscious" - where the "unconscious" may be somehow similar to Kant's "trascendental" (interpreted as the mind's Operating System before experience). > Tay Well, if the bot becomes "opinionated", it's probably just immature - just like a human. You will tell it to ponder well texts like Adam Rutherford's How to Argue With a Racist... But surely we want an Intelligence that forms its own opinions: we want to understand and manage reality, so we should hope that a super-Chomsky and a super-Ferguson will at some point converge and at some point put their "identity"-related positions into perspective.
- Cascais 2y agoPeople are surprised our brain is just physics/chemistry/etc. while our whole fucking body is just physics/chemistry/etc.
- _1tem 2y ago> LLMs are just pattern matching and intelligence is something beyond that. Human knowledge is ultimately embodied in physical experiences, it is not merely spitting out words. How can an LLM know the taste of a mango? Ability to regurgitate previously written text about mangos is neither knowledge nor intelligence. To know something, one must experience it, or something analogous to it. Even mathematicians "experience" their formulas and proofs in an ineffable way, they don't just produce text. There's an "aha" moment when learners "get it" and they experience knowledge and are then able to produce correct answers and generate new knowledge and discoveries. LLMs just generate text, and have yet to produce any significant new discovery in any field. LLMs are like fake wine sommeliers who have read a lot of lingo about wine and can speak convincingly enough to fool an amateur but would fail a simple taste test. We would say such a person doesn't know what he's talking about. People who think LLMs are intelligent are like a cargo cult, or polytheists who believe their idols are gods. They're anthropomorphizing processes they don't understand. How can a probabilistic next-token word predictor be intelligent? Actual mammal brains are far more complex than that.
- jdenning 2y agoVery well said.
- rowanG077 2y agoThis is just fluff. I can read about something and understand it. I don't need have died in a car crash to know I most certainly don't want to be in one. Until we do actually know how intelligence emerges in the brain it makes zero sense to compare any AI to it.
- _1tem 2y agoYou read something and understand it only if you have had analogous experiences. A person from the 16th century would not be able to understand what the words "Call me on FaceTime" means because they have never had an experience of using a smartphone. You certainly don't know what it means to die. But you know what "car" is and "crash" is because you have had (and maybe witnessed) analogous physical experiences.
- nkrisc 2y ago> Because they already believe humans are different and superior, whatever they observe doesn't affect their opinion. It’s frightening to think the people are already equating LLMs and humans. LLMs use text created by humans to create more text that resembles text written by humans. There is no comparison between the two. Every human is capable of acquiring new knowledge and experiences. An LLM can only acquire what has already been acquired by some human.
- tim333 2y agoThings like Google's project Astra are starting to be able to see stuff https://www.youtube.com/watch?v=bU2lUU5vURg&t=367s https://www.youtube.com/watch?v=bU2lUU5vURg&t=367s Similar tech in the latest Gemini I think.
- lostmsu 2y agoHuh? I haven't seen any humans create text without using text created by other humans.
- bmicraft 2y agoPeople usually get a reality check when blurt out misinformation in conversation - they get tested in school and on the job on whether what they think they've learned is actually true or works. OTOH an LLMs don't currently have any ability to test their "knowledge" and learn from the results. I think that's currently the both biggest problem and the biggest difference.