4 ms·
We really don’t understand the human brain well enough to have confidence that the mechanisms that cause people to respond with “I don’t know” are at all simila
by foldr 1y ago
We really don’t understand the human brain well enough to have confidence that the mechanisms that cause people to respond with “I don’t know” are at all similar to the mechanisms which cause LLMs to give such responses. And there are quite a few prima facie reasons to think that they wouldn’t be the same.
- Sharlin 1y agoThe mechanics don't have to be similar, only analogous, in the morphology sense.
- foldr 1y ago'Analogous in the morphology sense' is actually a more specific concept than 'similar'. But either way, we still don't know if they're analogous, or similar, or whatever term you prefer. Anyone who actually understands both LLMs and the human brain well enough to make confident claims that they basically work the same really ought to put in the effort to write up a paper and get a Nobel prize or two.
- Sharlin 1y agoAnalogous in the morphology sense means having come up with an entirely distinct solution to a common problem. Insect and bird wings have little to do with each other except that both flap to create lift. It explicitly does not imply the solutions are similar in mechanism, although that can be, and often is, a result of convergent evolution, of course. In particular, generally speaking (not claiming that LLMs a road to AGI, which is something I doubt) it's generally not a well-defensible philosophical position that the vertebrate brain (and remember that mammalian, bird and cephalopod brains are very different) is uniquely suited to produce what we call "intelligence". > Anyone who actually understands both LLMs and the human brain well enough to make confident claims that they basically work the same This is a strawman and not my position.
- foldr 1y agoIt was a characterization of the position of the post I was originally responding to, not your position. I don’t think anyone in this discussion has claimed that brains are uniquely suited to producing intelligence. The point was just that we have no idea if there is any interesting correspondence between how LLMs work and how brains work, beyond superficial and obvious analogies.
- ben_w 1y agoFWIW, I'm describing failure modes of a human, not mechanisms. I also think "would" in the comment I'm replying to is closer to "could" than to "does".
- foldr 1y agoCould you expand on that? What failure modes are we talking about exactly?
- ben_w 1y agoDunning Kruger, and also everyone who thinks they know better than domain experts. Humans who are wrong are often completely oblivious to being wrong.
- hglover945 1y ago[dead]
- foldr 1y agoOk, but what does that have to do with LLMS?
- ben_w 1y agoI'm saying both human brains and LLMs are bad at knowing when they're unsure — it may not even be possible to make an LLM that's superhuman in this regard. Most random humans have nearly constant confidence in their answers regardless of how much they know about a topic. -- Hmm. Stream of consciousness thoughts here: Problem is LLMs are very broad "System 1" thinking, no "System 2". Other kinds of model that combine LLM with a more logical module? Perhaps that could work. And if so then I guess you could use the logical module to create tokens for an LLM to learn from? Eventually, but may be inefficient. To work, logic module needs to know what LLM gets wrong, to create tokens for LLM to learn from; if you can do that, but fine tuning is slow and expensive and still errs, so why not keep the logic separate and just have it point out mistakes and have LLM regenerate answers a few times and if that doesn't work then say "IDK"?