3 ms·

Among the more mystically inclined there is a new metaphysic of consciousness that is emerging. It is based on the Bicameral Mentality [1] posited by Jaynes in

Among the more mystically inclined there is a new metaphysic of consciousness that is emerging. It is based on the Bicameral Mentality [1] posited by Jaynes in 1976. It is strongly related to the System 1/System 2 theory of the psychologist Kahneman in his 2011 book Thinking, Fast and Slow [2]. Ian McGilchrist has spilled a lot of ink trying to formalize this idea including in his 2009 work The Master and His Emissary. The 10,000ft view of the argument is that the human brain has two simultaneously running processes. One process is interested in fine details, analytic and critical thinking, reductionism, language, intelligence. The other process is interested in high-level details, connections between elements, holistic, symbolism, intuition. The argument further assumes that the systems are (to some degree) independent and cannot be built upon each other. These faculties are often ascribed to the left/right brain. If you subscribe to this idea (and this is a big if which you have to grant for the purpose of this argument), then you could argue that LLMs only capture System 2/analytical/reductionist processes of the brain. In that case, you could claim that "superhuman" is an incorrect way to describe their abilities since it only captures 1 aspect of the bicameral mind. However, this argument is inappropriate for discussions specifically on the topic of AI safety. The most basic response would be to point out that LLMs have or will surpass human capability in System 2 type thinking and thereby deserve the description "superhuman intelligence". This article seems to smuggle in this bicameral distinction as if it is universally agreed upon fact. The author seems to be demanding that Bengio concede to this framing as a basis for any discussion the author wants to have. 1. https://en.wikipedia.org/wiki/Bicameral_mentality https://en.wikipedia.org/wiki/Bicameral_mentality 2. https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow 3. https://en.wikipedia.org/wiki/The_Master_and_His_Emissary https://en.wikipedia.org/wiki/The_Master_and_His_Emissary

SonOfLilit 2y ago
Funny, in my discussions with my friends we all intuitively agree that LLMs are great (I'd say stronger than me in most subjects including my profession of 18 year, programming) at system 1, but can't system 2 for th life of them (except for AlphaProof which came out last week and showed that a very simple, very expensive solution Just Works).
- zoogeny 2y ago
  I would say that the fact that LLMs are language models is what would make one argue that they are left brain (or whatever constellation of traits are associated with that half of the split). In general, the other side is considered non-language oriented (e.g. symbolic, emotional, intuitive). I doubt you or your friends would agree that LLMs are more intuitive or emotional than you. I think the old association with "creativity" with the right brain might be misleading. For example, for some people there is a huge difference between being able to vomit out rhyming couplets in any language and being able to write a poem like Homer's ‎Odyssey. Another distinction one could use is that between "craft" and "art". It isn't that LLMs can't be skillful at manipulating words into pretty patterns, it's that the patterns ultimately have no connection to a larger narrative (unless explicitly directed by human instruction). At any rate, I would find it hard to agree with anyone who suggested that LLMs show any kind of "intuition" but I'd be interested if you can give concrete examples from your own use.
  - SonOfLilit 2y ago
    I don't understand System 1/2 as related to left/right, more to immediate answers vs planning, or say policy vs tree search in AlphaGo. When trying to solve a leetcode problem, I will "come up" (this step is atomic) with a possible solution, then think it through and figure out its weaknesses and come up with an amended solution. GPT4 beats me at the first part, but sucks at the second, even when specifically instructed to examine the wrong output its code gave and reflect on it.
    - zoogeny 2y ago
      I agree that System 1/2 doesn't map directly onto the right/left brain. One of McGilchrist's points is that our society has almost completely turned to relying on analytical and critical thinking. As a programmer, for example, we are often taught to quantify our problems (e.g. gather metrics), and then use reason/logic/rationality to arrive at efficient solutions. For that reason, I think it is hard for us to even understand what is being talked about by "right brain" kind of activities. One example that is helpful is to consider walking down a dark street at night. All of a sudden you feel uneasy and you become alert and you start to carefully examine your surroundings. It is that "feeling of unease" that is being pointed to, the sense of something being brought into your awareness. Another example is how an artist, for example a painter, will stand back and look at a painting in progress and decide "is this done?". If you are at the stage of "solve a leetcode problem" you are already 100% in left brain territory. Perhaps consider moments as a programmer where you get a feeling "something here isn't right". I don't mean, some test is obviously failing, I mean you suddenly feel some aspect of a situation is "off" and you realize you need to direct your attention towards it without any external warning. Then consider: could an LLM have that same kind of intuition?

SonOfLilit 2y ago

Funny, in my discussions with my friends we all intuitively agree that LLMs are great (I'd say stronger than me in most subjects including my profession of 18 year, programming) at system 1, but can't system 2 for th life of them (except for AlphaProof which came out last week and showed that a very simple, very expensive solution Just Works).

zoogeny 2y ago
I would say that the fact that LLMs are language models is what would make one argue that they are left brain (or whatever constellation of traits are associated with that half of the split). In general, the other side is considered non-language oriented (e.g. symbolic, emotional, intuitive). I doubt you or your friends would agree that LLMs are more intuitive or emotional than you. I think the old association with "creativity" with the right brain might be misleading. For example, for some people there is a huge difference between being able to vomit out rhyming couplets in any language and being able to write a poem like Homer's ‎Odyssey. Another distinction one could use is that between "craft" and "art". It isn't that LLMs can't be skillful at manipulating words into pretty patterns, it's that the patterns ultimately have no connection to a larger narrative (unless explicitly directed by human instruction). At any rate, I would find it hard to agree with anyone who suggested that LLMs show any kind of "intuition" but I'd be interested if you can give concrete examples from your own use.
- SonOfLilit 2y ago
  I don't understand System 1/2 as related to left/right, more to immediate answers vs planning, or say policy vs tree search in AlphaGo. When trying to solve a leetcode problem, I will "come up" (this step is atomic) with a possible solution, then think it through and figure out its weaknesses and come up with an amended solution. GPT4 beats me at the first part, but sucks at the second, even when specifically instructed to examine the wrong output its code gave and reflect on it.
  - zoogeny 2y ago
    I agree that System 1/2 doesn't map directly onto the right/left brain. One of McGilchrist's points is that our society has almost completely turned to relying on analytical and critical thinking. As a programmer, for example, we are often taught to quantify our problems (e.g. gather metrics), and then use reason/logic/rationality to arrive at efficient solutions. For that reason, I think it is hard for us to even understand what is being talked about by "right brain" kind of activities. One example that is helpful is to consider walking down a dark street at night. All of a sudden you feel uneasy and you become alert and you start to carefully examine your surroundings. It is that "feeling of unease" that is being pointed to, the sense of something being brought into your awareness. Another example is how an artist, for example a painter, will stand back and look at a painting in progress and decide "is this done?". If you are at the stage of "solve a leetcode problem" you are already 100% in left brain territory. Perhaps consider moments as a programmer where you get a feeling "something here isn't right". I don't mean, some test is obviously failing, I mean you suddenly feel some aspect of a situation is "off" and you realize you need to direct your attention towards it without any external warning. Then consider: could an LLM have that same kind of intuition?