6 ms·

Heres a definition. How impressive is the output relative to the input. And by input, I don't just mean the prompt, but all the training data itself. Do you th

Heres a definition. How impressive is the output relative to the input. And by input, I don't just mean the prompt, but all the training data itself. Do you think someone who has only ever studied pre-calc would be able to work through a calculus book if they had sufficient time? how about a multi-variable calc book? How about grad level mathematics? IMO intelligence and thinking is strictly about this ratio; what can you extrapolate from the smallest amount of information possible, and why? From this perspective, I dont think any of our LLMs are remotely intelligent despite what our tech leaders say.

mycall 11mo ago
Animals think but come with instincts which breaks the output relative to the input test you propose. Behaviors are essentially pre-programmed input from millions of years of evolution, stored in the DNA/neurology. The learning thus typically associative and domain-specific, not abstract extrapolation. A crow bending a piece of wire into a hook to retrieve food demonstrates a novel solution extrapolated from minimal, non-instinctive, environmental input. This kind of zero-shot problem-solving aligns better with your definition of intelligence.
kryogen1c 11mo ago
Hear, hear! I have long thought this, but not had as good way to put it as you did. If you think about geniuses like Einstein and ramanujen, they understood things before they had the mathematical language to express them. LLMs are the opposite; they fail to understand things after untold effort, training data, and training. So the question is, how intelligent are LLMs when you reduce their training data and training? Since they rapidly devolve into nonsense, the answer must be that they have no internal intelligence Ever had the experience of helping someone who's chronically doing the wrong thing, to eventually find they had an incorrect assumption, an incorrect reasoning generating deterministic wrong answers? LLMs dont do that; they just lack understanding. They'll hallucinate unrelated things because they dont know what they're talking about - you may have also had this experience with someone :)
- hodgehog11 11mo ago
  > So the question is, how intelligent are LLMs when you reduce their training data and training? Since they rapidly devolve into nonsense, the answer must be that they have no internal intelligence This would be the equivalent of removing all senses of a human from birth and expecting them to somehow learn things. They will not. Therefore humans are not intelligent? > LLMs dont do that; they just lack understanding. You have no idea what they are doing. Since they are smaller than the dataset, they must have learned an internal algorithm. This algorithm is drawing patterns from somewhere - those are its internal, incorrect assumptions. It does not operate in the same way that a human does, but it seems ridiculous to say that it lacks intelligence because of that. It sounds like you've reached a conclusion, that LLMs cannot be intelligent because they have said really weird things before, and are trying to justify it in reverse. Sure, it may not have grasped that particular thing. But are you suggesting that you've never met a human that is feigning understanding in a particular topic say some really weird things akin to an LLM? I'm an educator, and I have heard the strangest things that I just cannot comprehend no matter how much I dig. It really feels like shifting goalposts. We need to do better than that.
  - pka 11mo ago
    > and are trying to justify it in reverse In split-brain experiments this is exactly how one half of the brain retroactively justifies the action of the other half. Maybe it is the case in LLMs that an overpowered latent feature sets the overall direction of the "thought" and then inference just has to make the best of it.
- nsagent 11mo ago
  You might be interested in reading about the minimum description length (MDL) principle [1]. Despite all the dissenters to your argument, what your positing is quite similar to MDL. It's how you can fairly compare models (I did some research in this area for LLMs during my PhD). Simply put, to compare models, you describe both the model and training data using a code (usual reported as number of bits). The trained model that represents the data within the fewest number of bits is the more powerful model. This paper [2] from ICML 2021 shows a practical approach for attempting to estimate MDL for NLP models applied to text datasets. [1]: http://www.modelselection.org/mdl/ http://www.modelselection.org/mdl/ [2]: https://proceedings.mlr.press/v139/perez21a.html https://proceedings.mlr.press/v139/perez21a.html
hodgehog11 11mo ago
Yeah, that's compression. Although your later comments neglect the many years of physical experience that humans have as well as the billions of years of evolution. And yes, by this definition, LLMs pass with flying colours.
- saberience 11mo ago
  I hate when people bring up this “billions of years of evolution” idea. It’s completely wrong and deluded in my opinion. Firstly humans have not been evolving for “billions” of years. Homo sapiens have been around for maybe 300’000 years, and the “homo” genus has been 2/3 million years. Before that we were chimps etc and that’s 6/7 million years ago. If you want to look at the entire brain development, ie from mouse like creatures through to apes and then humans that’s 200M years. If you want to think about generations it’s only 50/75M generations, ie “training loops”. That’s really not very many. Also the bigger point is this, for 99.9999% of that time we had no writing, or any kind of complex thinking required. So our ability to reason about maths, writing, science etc is only in the last 2000-2500 years! Ie only roughly 200 or so generations. Our brain was not “evolved” to do science, maths etc. Most of evolution was us running around just killing stuff and eating and having sex. It’s only a tiny tiny amount of time that we’ve been working on maths, science, literature, philosophy. So actually, these models have a massive, massive amount of training more than humans had to do roughly the same thing but using insane amounts of computing power and energy. Our brains were evolved for a completely different world and environment and daily life that the life we lead now. So yes, LLMs are good, but they have been exposed to more data and training time than any human could have unless we lived for 100000 years and still perform worse than we do in most problems!
  - GoblinSlayer 11mo ago
    >Most of evolution was us running around just killing stuff and eating and having sex. Tell Boston Dynamics how to do that. Mice inherited brain from their ancestors. You might think you don't need a working brain to reason about math, but that's because you don't know how thinking works, it's argument from ignorance.
rolisz 11mo ago
Is the millions of years of evolution part of the training data for humans?
- Rebuff5007 11mo ago
  Millions of years of evolution have clearly equipped our brain with some kind of structure (or "inductive bias") that makes it possible for us to actively build a deep understanding for our world... In the context of AI I think this translates more to representations and architecture than it does with training data.
  - goatlover 11mo ago
    Because genes don't encode the millions of years of experience from ancestors, despite how interesting that is in say the Dune Universe (with help of the spice melange). My understanding is genes don't even specifically encode for the exact structure of the brain. It's more of a recipe that gets generated than a blue print, with young brains doing a lot of pruning as they start experiencing the world. It's a malleable architecture that self-adjusts as needed.
lukebuehler 11mo ago
That an okay-ish definition, but to me this is more about whether this kind of "intelligence" is worth it, not whether it is intelligence itself. The current AI boom clearly thinks it is worth to put that much input to get the current frontier-model-level of output. Also, don't forget the input scales across roughly 1B weekly users at inference time. I would say a good definition has to, minimally, take on the Turing test (even if you disagree, you should say why). Or in current vibe parlance, it does "feel" intelligent to many people--they see intelligence in it. In my book this allows us to call it intelligent, at least loosely.
skeeter2020 11mo ago
This feels too linear. Machines are great at ingesting huge volumes of data, following relatively simple rules and producing optimized output, but are LLMs sufficiently better than humans at finding windy, multi-step connections across seemingly unrelated topics & fields? Have they shown any penchant for novel conclusions from observational science? What I think your ratio misses is the value in making the targeted extrapolation or hypothesis that holds up out of a giant body of knowledge.
- blks 11mo ago
  Are you aware of anything novel, produced by an LLM?
tremon 11mo ago
I'm not sure I understand what you're getting at. You seem to be on purpose comparing apples and oranges here: for an AI, we're supposed to include the entire training set in the definition of its input, but for a human we don't include the entirety of that human's experience and only look at the prompt?
- Rebuff5007 11mo ago
  > but for a human we don't include the entirety of that human's experience and only look at the prompt? When did I say that? Of course you look at a human's experience when you judge the quality of their output. And you also judge their output based on the context they did their work in. Newton wouldn't be Newton if he was the 14th guy to claim that the universe is governed by three laws of motion. Extending the example I used above, I would be more impressed if an art student aced a tough calc test than a math student, given that a math student probably has spent much more time with the material. "Intelligence and "thinking" are abstract concepts, and I'm simply putting forward a way that I think about them. It works very much outside the context of AI too. The "smartest" colleagues I've worked with are somehow able to solve a problem with less information or time than I need. Its usually not because they have more "training data" than me.
jononor 11mo ago
For more on this perspective, see the paper On the measure of intelligence (F. Chollet, 2019). And more recently, the ARC challenge/benchmarks, which are early attempts at using this kind of definition in practice to improve current systems.
fragmede 11mo ago
There are plenty of humans that will never "get" calculus, despite numerous attempts at the class and countless hours of 1:1 tutoring. Are those people not intelligent? Do they not think? We could say yes they aren't, but by the metric of making money, plenty of people are smart enough to be rich, while college math professors aren't. And while that's a facile way of measuring someone's worth or their contribution to society (some might even say "bad"), it remains that even if someone cant understand calculus, some of them are intelligent enough to understand humans enough to be rich through some fashion that wasn't simply handed to them.
- chipsrafferty 11mo ago
  I don't think it's actually true that someone with: 1. A desire to learn calculus 2. A good teacher 3. No mental impairments such as dementia or other major brain drainers could not learn calculus. Most people don't really care to try or don't get good resources. What you see as an intelligent mathematician is almost always someone born with better resources that was also encouraged to pursue math.
  - fragmede 11mo ago
    1 and 3 are loopholes large enough to drive a semi truck through. You could calculate how far the truck traveled if you have its acceleration with a double integral, however.

mycall 11mo ago

Animals think but come with instincts which breaks the output relative to the input test you propose. Behaviors are essentially pre-programmed input from millions of years of evolution, stored in the DNA/neurology. The learning thus typically associative and domain-specific, not abstract extrapolation. A crow bending a piece of wire into a hook to retrieve food demonstrates a novel solution extrapolated from minimal, non-instinctive, environmental input. This kind of zero-shot problem-solving aligns better with your definition of intelligence.

kryogen1c 11mo ago

Hear, hear! I have long thought this, but not had as good way to put it as you did. If you think about geniuses like Einstein and ramanujen, they understood things before they had the mathematical language to express them. LLMs are the opposite; they fail to understand things after untold effort, training data, and training. So the question is, how intelligent are LLMs when you reduce their training data and training? Since they rapidly devolve into nonsense, the answer must be that they have no internal intelligence Ever had the experience of helping someone who's chronically doing the wrong thing, to eventually find they had an incorrect assumption, an incorrect reasoning generating deterministic wrong answers? LLMs dont do that; they just lack understanding. They'll hallucinate unrelated things because they dont know what they're talking about - you may have also had this experience with someone :)

hodgehog11 11mo ago
> So the question is, how intelligent are LLMs when you reduce their training data and training? Since they rapidly devolve into nonsense, the answer must be that they have no internal intelligence This would be the equivalent of removing all senses of a human from birth and expecting them to somehow learn things. They will not. Therefore humans are not intelligent? > LLMs dont do that; they just lack understanding. You have no idea what they are doing. Since they are smaller than the dataset, they must have learned an internal algorithm. This algorithm is drawing patterns from somewhere - those are its internal, incorrect assumptions. It does not operate in the same way that a human does, but it seems ridiculous to say that it lacks intelligence because of that. It sounds like you've reached a conclusion, that LLMs cannot be intelligent because they have said really weird things before, and are trying to justify it in reverse. Sure, it may not have grasped that particular thing. But are you suggesting that you've never met a human that is feigning understanding in a particular topic say some really weird things akin to an LLM? I'm an educator, and I have heard the strangest things that I just cannot comprehend no matter how much I dig. It really feels like shifting goalposts. We need to do better than that.
- pka 11mo ago
  > and are trying to justify it in reverse In split-brain experiments this is exactly how one half of the brain retroactively justifies the action of the other half. Maybe it is the case in LLMs that an overpowered latent feature sets the overall direction of the "thought" and then inference just has to make the best of it.
nsagent 11mo ago
You might be interested in reading about the minimum description length (MDL) principle [1]. Despite all the dissenters to your argument, what your positing is quite similar to MDL. It's how you can fairly compare models (I did some research in this area for LLMs during my PhD). Simply put, to compare models, you describe both the model and training data using a code (usual reported as number of bits). The trained model that represents the data within the fewest number of bits is the more powerful model. This paper [2] from ICML 2021 shows a practical approach for attempting to estimate MDL for NLP models applied to text datasets. [1]: http://www.modelselection.org/mdl/ http://www.modelselection.org/mdl/ [2]: https://proceedings.mlr.press/v139/perez21a.html https://proceedings.mlr.press/v139/perez21a.html

hodgehog11 11mo ago

Yeah, that's compression. Although your later comments neglect the many years of physical experience that humans have as well as the billions of years of evolution. And yes, by this definition, LLMs pass with flying colours.

saberience 11mo ago
I hate when people bring up this “billions of years of evolution” idea. It’s completely wrong and deluded in my opinion. Firstly humans have not been evolving for “billions” of years. Homo sapiens have been around for maybe 300’000 years, and the “homo” genus has been 2/3 million years. Before that we were chimps etc and that’s 6/7 million years ago. If you want to look at the entire brain development, ie from mouse like creatures through to apes and then humans that’s 200M years. If you want to think about generations it’s only 50/75M generations, ie “training loops”. That’s really not very many. Also the bigger point is this, for 99.9999% of that time we had no writing, or any kind of complex thinking required. So our ability to reason about maths, writing, science etc is only in the last 2000-2500 years! Ie only roughly 200 or so generations. Our brain was not “evolved” to do science, maths etc. Most of evolution was us running around just killing stuff and eating and having sex. It’s only a tiny tiny amount of time that we’ve been working on maths, science, literature, philosophy. So actually, these models have a massive, massive amount of training more than humans had to do roughly the same thing but using insane amounts of computing power and energy. Our brains were evolved for a completely different world and environment and daily life that the life we lead now. So yes, LLMs are good, but they have been exposed to more data and training time than any human could have unless we lived for 100000 years and still perform worse than we do in most problems!
- GoblinSlayer 11mo ago
  >Most of evolution was us running around just killing stuff and eating and having sex. Tell Boston Dynamics how to do that. Mice inherited brain from their ancestors. You might think you don't need a working brain to reason about math, but that's because you don't know how thinking works, it's argument from ignorance.

rolisz 11mo ago

Is the millions of years of evolution part of the training data for humans?

Rebuff5007 11mo ago
Millions of years of evolution have clearly equipped our brain with some kind of structure (or "inductive bias") that makes it possible for us to actively build a deep understanding for our world... In the context of AI I think this translates more to representations and architecture than it does with training data.
- goatlover 11mo ago
  Because genes don't encode the millions of years of experience from ancestors, despite how interesting that is in say the Dune Universe (with help of the spice melange). My understanding is genes don't even specifically encode for the exact structure of the brain. It's more of a recipe that gets generated than a blue print, with young brains doing a lot of pruning as they start experiencing the world. It's a malleable architecture that self-adjusts as needed.

lukebuehler 11mo ago

That an okay-ish definition, but to me this is more about whether this kind of "intelligence" is worth it, not whether it is intelligence itself. The current AI boom clearly thinks it is worth to put that much input to get the current frontier-model-level of output. Also, don't forget the input scales across roughly 1B weekly users at inference time. I would say a good definition has to, minimally, take on the Turing test (even if you disagree, you should say why). Or in current vibe parlance, it does "feel" intelligent to many people--they see intelligence in it. In my book this allows us to call it intelligent, at least loosely.

skeeter2020 11mo ago

This feels too linear. Machines are great at ingesting huge volumes of data, following relatively simple rules and producing optimized output, but are LLMs sufficiently better than humans at finding windy, multi-step connections across seemingly unrelated topics & fields? Have they shown any penchant for novel conclusions from observational science? What I think your ratio misses is the value in making the targeted extrapolation or hypothesis that holds up out of a giant body of knowledge.

blks 11mo ago
Are you aware of anything novel, produced by an LLM?

tremon 11mo ago

I'm not sure I understand what you're getting at. You seem to be on purpose comparing apples and oranges here: for an AI, we're supposed to include the entire training set in the definition of its input, but for a human we don't include the entirety of that human's experience and only look at the prompt?

Rebuff5007 11mo ago
> but for a human we don't include the entirety of that human's experience and only look at the prompt? When did I say that? Of course you look at a human's experience when you judge the quality of their output. And you also judge their output based on the context they did their work in. Newton wouldn't be Newton if he was the 14th guy to claim that the universe is governed by three laws of motion. Extending the example I used above, I would be more impressed if an art student aced a tough calc test than a math student, given that a math student probably has spent much more time with the material. "Intelligence and "thinking" are abstract concepts, and I'm simply putting forward a way that I think about them. It works very much outside the context of AI too. The "smartest" colleagues I've worked with are somehow able to solve a problem with less information or time than I need. Its usually not because they have more "training data" than me.

jononor 11mo ago

For more on this perspective, see the paper On the measure of intelligence (F. Chollet, 2019). And more recently, the ARC challenge/benchmarks, which are early attempts at using this kind of definition in practice to improve current systems.

fragmede 11mo ago

There are plenty of humans that will never "get" calculus, despite numerous attempts at the class and countless hours of 1:1 tutoring. Are those people not intelligent? Do they not think? We could say yes they aren't, but by the metric of making money, plenty of people are smart enough to be rich, while college math professors aren't. And while that's a facile way of measuring someone's worth or their contribution to society (some might even say "bad"), it remains that even if someone cant understand calculus, some of them are intelligent enough to understand humans enough to be rich through some fashion that wasn't simply handed to them.

chipsrafferty 11mo ago
I don't think it's actually true that someone with: 1. A desire to learn calculus 2. A good teacher 3. No mental impairments such as dementia or other major brain drainers could not learn calculus. Most people don't really care to try or don't get good resources. What you see as an intelligent mathematician is almost always someone born with better resources that was also encouraged to pursue math.
- fragmede 11mo ago
  1 and 3 are loopholes large enough to drive a semi truck through. You could calculate how far the truck traveled if you have its acceleration with a double integral, however.