3 ms·

I'm always intrigued when phenomena observed in the brain carry across to artificial systems. The idea of multiple modes of thought reminds me of Thinking, Fast

I'm always intrigued when phenomena observed in the brain carry across to artificial systems. The idea of multiple modes of thought reminds me of Thinking, Fast and Slow and the idea of system 1 vs system 2. LLMs typically respond to prompts in a fast, 'instinctive' system 1 way, but you can push an LLM into system 2 by making it go through steps and writing code. Originally this idea was presented as a bit of a workaround for a limitation of LLMs, but perhaps in the grand scheme of things it's actually the smart approach and we should never expect LLMs to consistently perform system 2 style tasks off the cuff?

j_maffe 2y ago
That is exactly how I describe it to others. LLMs have mostly figured out the fast system but it's woefully bad at system 2. I'm surprised this kind of analogy isn't common in discussions about LLMs.
- the_jends 2y ago
  Because LLMs works like how system 1 does, with heuristics such as pattern matching, frequency heuristics, etc. The fall back to coding stuff works not because it's doing system 2, it's because there are many instances of system 2 thinking for this particular problem done by humans so that the instances can be found using the heuristics.
  - j_maffe 2y ago
    Yeah, I think the analogy here would be how an experienced programmer would write code if asked to write on the spot given a specific prompt with no time to think.
butlike 2y ago
The difficulty might be in that much like we partition our brains to prepare for low-effort tasks and high-effort tasks (have you ever hyped yourself up to "finally sit down and get this thing done?"), so too could LLMs partition their compute space. Basically memoizing the low-hanging, easy-to-answer fruit. The difficulty comes in when a system 2 task arises, it's not easily apparent at first what the requester is asking for. Some people are just fine with a single example being shit out, then they copy-and-paste it into their IDE and go from there. Others, want a step-by-step breakdown of the reasoning, which won't be apparent until 2+ prompts in. It's the same as your boss emailing you asking: "What about this report?" It could just be a one-off request for perspective, or it could spider web out into a multi-ticket JIRA epic for a new feature, but you can't really discern the intention from the initial ask. The difference is back-and-forth is more widly accepted, in my experience, interfacing human-to-human than human-to-LLM.
EnergyAmy 2y ago
I get kind of annoyed when seeing people talk about LLMs without understanding this. I think it's a reaction of sorts, people are so busy dunking on any flaws that they're missing the forest for the trees. Chain of thought reasoning (i.e. system 2) looks very interesting and not at all a hack, any more so than a person sitting down to think hard is a hack.
KuriousCat 2y ago
I think this analogy is not right. System 1 vs 2 in biology is in response to the need to preserve the life and run away from threats. For interactive question answer system it makes no sense to resort to system 1 kind of responses.

j_maffe 2y ago

That is exactly how I describe it to others. LLMs have mostly figured out the fast system but it's woefully bad at system 2. I'm surprised this kind of analogy isn't common in discussions about LLMs.

the_jends 2y ago
Because LLMs works like how system 1 does, with heuristics such as pattern matching, frequency heuristics, etc. The fall back to coding stuff works not because it's doing system 2, it's because there are many instances of system 2 thinking for this particular problem done by humans so that the instances can be found using the heuristics.
- j_maffe 2y ago
  Yeah, I think the analogy here would be how an experienced programmer would write code if asked to write on the spot given a specific prompt with no time to think.

butlike 2y ago

The difficulty might be in that much like we partition our brains to prepare for low-effort tasks and high-effort tasks (have you ever hyped yourself up to "finally sit down and get this thing done?"), so too could LLMs partition their compute space. Basically memoizing the low-hanging, easy-to-answer fruit. The difficulty comes in when a system 2 task arises, it's not easily apparent at first what the requester is asking for. Some people are just fine with a single example being shit out, then they copy-and-paste it into their IDE and go from there. Others, want a step-by-step breakdown of the reasoning, which won't be apparent until 2+ prompts in. It's the same as your boss emailing you asking: "What about this report?" It could just be a one-off request for perspective, or it could spider web out into a multi-ticket JIRA epic for a new feature, but you can't really discern the intention from the initial ask. The difference is back-and-forth is more widly accepted, in my experience, interfacing human-to-human than human-to-LLM.

EnergyAmy 2y ago

I get kind of annoyed when seeing people talk about LLMs without understanding this. I think it's a reaction of sorts, people are so busy dunking on any flaws that they're missing the forest for the trees. Chain of thought reasoning (i.e. system 2) looks very interesting and not at all a hack, any more so than a person sitting down to think hard is a hack.

KuriousCat 2y ago

I think this analogy is not right. System 1 vs 2 in biology is in response to the need to preserve the life and run away from threats. For interactive question answer system it makes no sense to resort to system 1 kind of responses.