4 ms·

Yeah, this is the question I always find myself asking after anything like this. Without establishing a definition of "intelligence", anything that follows is m

Yeah, this is the question I always find myself asking after anything like this. Without establishing a definition of "intelligence", anything that follows is meaningless. And "intelligence" can be defined. People just think of different things when you use the word unless you define it. Here is how I like to define intelligence: the intersection between knowledge and reason. An example of a system that is highly knowledgeable but lacks intelligence due to its inability to reason is Wikipedia. A system that is highly reasonable but lacks intelligence due to its lack of knowledge is a calculator. It's obvious that language models encode more knowledge than humans do. Even a few GB model that I can download and run on my computer can spit out facts about random things. Way more than any human could. Sometimes they are wrong about facts, but so is any knowledge system, including humans. An LLMs reasoning capability is a bit less clear cut. When you have any understanding of how LLMs work, it seems like they should have no ability to do novel reasoning. Yet when you try to push them, they are often surprisingly good. I think that people who say the best LLMs have worse than human reasoning capabilities probably overestimate the average adults reasoning capability. Plus it is not hard at all to hook up a language model as a component in a greater intelligence system, by adding other components that specialize in reason. For example an environment to run code in.

righthand 2y ago
I will say that LLM reasoning is good because it’s using information that has already been reasoned. Whether that reasoned information is fact or fiction is another case all together, so since LLMs are good at regurgitating information then they are good at regurgitating reasoned information as well. I would expect reasoned regurgitation to inform me at best; I wouldn’t expect something regurgitating to be able to necessarily be able to reason information it can’t regurgitate already.
- Zambyte 2y ago
  Wikipedia is using information that has already been reasoned also. That is just knowledge. I specifically called out "novel reasoning" because you can use examples from your personal life after the model was trained, or completely fabricate scenarios, and it will still be able to produce interesting and useful reasoning about it. Again, it's not the best tool available for training and logic applications, but the fact that it really can demonstrate novel reasoning along with its obvious knowledge means thar it clearly has some non-zero level of intelligence.
  - righthand 2y ago
    I know some dogs that don’t quite understand capitalism but can definitely reason in a novel way. I think intelligence might not just be reasoning but also the ability to seek knowledge and reason. Some form of will to attain higher intelligence. Machines only have human interests of further knowledge in that regard.
    - Zambyte 2y ago
      > I know some dogs that don’t quite understand capitalism but can definitely reason in a novel way I would just describe this as intelligent but less than average human intelligence. > I think intelligence might not just be reasoning but also the ability to seek knowledge and reason. Some form of will to attain higher intelligence. This is a circular definition of intelligence that doesn't really make sense. How can intelligence be the drive for more intelligence? Those have to be two different things. Consider a college educated person working in some field that requires lots of critical thinking. They have been doing their job for 30+ years, and they are very good at it. They are stubborn and set in their ways; not interested in changing anything. Now consider a 3 year old who is curious about everything. Who is more intelligent? The answer seems to obviously be the adult, even though they refuse to learn anything more. I really don't think the willingness or ability to learn is a component of intelligence. A system can be statically intelligent or it can be dynamically intelligent. Dynamically intelligent systems are more exciting and novel, but statically intelligent systems can still be extremely useful. Ability to learn is more interesting in the context of general intelligence (also note general intelligence implies the existence of non-general or domain specific intelligence). In order to be able to act intelligently in any situation (such as: what if my computer suddenly grows legs) it must be able to learn through experimentation.
      - righthand 2y ago
        So what I gather from what you’re saying. A will to learn has nothing to do with intelligence. But then you go on to state that it revolves around general or domain specific intelligence. Then what is general intelligence. You’re moving the goal post back into undefined terms. I am not stating that intelligence is the drive for more intelligence. I am saying that intelligence is that act of collecting more knowledge. Setting bars like: > I would just describe this as intelligent but less than average human intelligence. I know some people who appear dumber than dogs because they believe after schooling is complete they have everything they need to operate in life. Are these people below human intelligence? Intelligence is restrictive to your exposure to knowledge. All knowledge can be aggregated to a domain. You could state “general intelligence” is a primary school education, but not all primary schools deliver the same education. So what is it?

righthand 2y ago

I will say that LLM reasoning is good because it’s using information that has already been reasoned. Whether that reasoned information is fact or fiction is another case all together, so since LLMs are good at regurgitating information then they are good at regurgitating reasoned information as well. I would expect reasoned regurgitation to inform me at best; I wouldn’t expect something regurgitating to be able to necessarily be able to reason information it can’t regurgitate already.

Zambyte 2y ago
Wikipedia is using information that has already been reasoned also. That is just knowledge. I specifically called out "novel reasoning" because you can use examples from your personal life after the model was trained, or completely fabricate scenarios, and it will still be able to produce interesting and useful reasoning about it. Again, it's not the best tool available for training and logic applications, but the fact that it really can demonstrate novel reasoning along with its obvious knowledge means thar it clearly has some non-zero level of intelligence.
- righthand 2y ago
  I know some dogs that don’t quite understand capitalism but can definitely reason in a novel way. I think intelligence might not just be reasoning but also the ability to seek knowledge and reason. Some form of will to attain higher intelligence. Machines only have human interests of further knowledge in that regard.
  - Zambyte 2y ago
    > I know some dogs that don’t quite understand capitalism but can definitely reason in a novel way I would just describe this as intelligent but less than average human intelligence. > I think intelligence might not just be reasoning but also the ability to seek knowledge and reason. Some form of will to attain higher intelligence. This is a circular definition of intelligence that doesn't really make sense. How can intelligence be the drive for more intelligence? Those have to be two different things. Consider a college educated person working in some field that requires lots of critical thinking. They have been doing their job for 30+ years, and they are very good at it. They are stubborn and set in their ways; not interested in changing anything. Now consider a 3 year old who is curious about everything. Who is more intelligent? The answer seems to obviously be the adult, even though they refuse to learn anything more. I really don't think the willingness or ability to learn is a component of intelligence. A system can be statically intelligent or it can be dynamically intelligent. Dynamically intelligent systems are more exciting and novel, but statically intelligent systems can still be extremely useful. Ability to learn is more interesting in the context of general intelligence (also note general intelligence implies the existence of non-general or domain specific intelligence). In order to be able to act intelligently in any situation (such as: what if my computer suddenly grows legs) it must be able to learn through experimentation.
    - righthand 2y ago
      So what I gather from what you’re saying. A will to learn has nothing to do with intelligence. But then you go on to state that it revolves around general or domain specific intelligence. Then what is general intelligence. You’re moving the goal post back into undefined terms. I am not stating that intelligence is the drive for more intelligence. I am saying that intelligence is that act of collecting more knowledge. Setting bars like: > I would just describe this as intelligent but less than average human intelligence. I know some people who appear dumber than dogs because they believe after schooling is complete they have everything they need to operate in life. Are these people below human intelligence? Intelligence is restrictive to your exposure to knowledge. All knowledge can be aggregated to a domain. You could state “general intelligence” is a primary school education, but not all primary schools deliver the same education. So what is it?