4 ms·

Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained. Also how do you prove that GPT is worse at counting? Because

Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained. Also how do you prove that GPT is worse at counting? Because GPT can currently count both apples and oranges.

inimino 3y ago
> Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained. No. Animals evolved and are able to do those things. Evolution is not training, and evolution has approximately zero to do with how transformers work.
- mewpmewp2 3y ago
  What's the difference here? Both seem to have adaptive neural networks where those networks change as time goes on due to a reward - for animals, mutated genes being more likely to be given forward if the change was good. Over millions of generations it's statistically likely that more good genes that caused the neural networks to be in a state that is better able to solve problems within the environment get passed on, eventually resulting in an emerging intelligence. For training you similarly change the state of the neural network depending on whether the answer is good or bad.
  - inimino 3y ago
    Evolution operates on genes, which do not encode synaptic connections for one thing. The analogy you're making here is so stretched it's hard to begin to say what's wrong with it. Backpropagation and natural selection are about as different as two things can be. About the only thing you can say they have in common is that both can be modeled as optimization processes. What's the difference between a star and a bonfire? Both use fuel and produce heat and light.
    - mewpmewp2 3y ago
      I mean the point was about LLMs not truly being problem solvers because they were trained to do so as opposed to having been evolved through evolution. I'm looking for what the difference is specifically within that dimension. Biological pigeons had their own process of evolution how they reached to have the type of neural networks and systems in themselves that gave them the ability to count - but not in all contexts for sure. So yes, my point is that both have an optimisation process that through time lend them those emerging capabilities.
      - inimino 3y ago
        No, the point was that LLMs are not good at problem solving because they are not good at problem solving, not because they were not evolved. We don't understand how we or animals solve problems, which is why we haven't yet succeeded in replicating that in AI. You're the one bringing in these incredibly strained analogies, because you want GPT 4 to be more than it is or something, I'm not sure.
  - nicklecompte 3y ago
    The bigest difference is that training does not change the size or architecture of an artificial neural network, but biological evolution dramatically changes the size and architecture of animals' brains. Your comparison is sincerely vacuous. It vaguely makes sense if you're talking about GPT-3 to GPT-4 (though I don't think it's helpful). It makes no sense if you're talking about training a single neural network.
    - mewpmewp2 3y ago
      I mean that exposure to a lot of training material yielded in the final set of capabilities. Pigeons and their ancestors were exposed to certain situations throughout evolution that yielded in the formation of neural network and its ability to "count". Which I believe is not actual "one, two, three", but just the amount of signals being activated resulting in a certain output from pigeon. There's a difference in how a human counts, except for small numbers which you can intuitively immediately come up with a number. There was training material which were situations to which organisms had to produce output for and if the output was good their genetics survived, eventually forming the neural network that was able to handle this training material well, but similarly producing emergent behaviour like being able to "count". But GPT-Vision can easily do as well what a Pigeon can. What's the exact thing that implies Pigeon is doing it somehow more intelligently? If you ask them on a picture the quantity of something, I'm pretty sure both respond to the amount of this type of signal received either though light waves or pixels encoded for GPT.
      - RaftPeople 3y ago
        > Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained. If you interpret this question at the most abstract level of "aren't both solutions arrived at through training/trial+error method?" - then the answer is probably yes, they are both arrived at in some conceptually similar manner. But they are two very different underlying systems and we don't really understand the biological systems well enough to even be able to truly compare. Beyond that, it seems that humans (switching to humans from pigeons) have some sort of representation/understanding of the world around us such that even if we produce the same result as ChatGPT to a counting question, the information stored within our systems is not equivalent.
nicklecompte 3y ago
> Also how do you prove that GPT is worse at counting? Back in June 2023 GPT-4 was dramatically worse at counting than a pigeon in the sense that it couldn't accurately tell the difference between sentences with 3 words and sentences with 5 words, whereas pigeons can count almost anything up to about 10. It also routinely failed "pick the shorter sentence" tests which I literally took from a test administered to mice. GPT simply doesn't understand what numbers are, whereas pigeons and mice have an intuitive understanding similar to toddlers. You don't need to teach kids what 3 means, you just need to teach them the human symbol for the concept of 3. GPT only has the human symbol and does not seem capable of understanding the concept. In my testing GPT-4 consistently failed counting / pattern-recognition tests even if you used "chain-of-thought" prompting. As far as I could tell its only true understanding of numbers was "one, two, many." This seems reflected in real use cases, where GPT routinely (and hilariously) ignores commands to return 50 words/etc of output. GPT doesn't know what fifty means, it just knows what various documents that say "word count: 50" look like, and tries to imitate the tone. Since transformer neural networks lack recursion I conjecture that GPT will never be able to understand a number larger than 2, even if in specific cases it can solve counting problems up to eleventy billion. This is what I mean by "counting apples, not oranges," its sense of counting is paper-thin and easily fooled by adversarial prompts. It is much harder to fool a mouse or a pigeon. Many of the tests I ran back in April 2023 no longer work. I strongly suspect this is because OpenAI trained GPT to many of the tests that people were throwing at it, and not because GPT actually became "smarter." I stopped messing around with GPT specifically because OpenAI doesn't issue any release notes, making replicability impossible. Mistrial's 77B model was dramatically worse than even GPT-3 at counting, but I doubt they trained it to count. Not sure about LLaMa/etc.
- mewpmewp2 3y ago
  When you are talking about "counting", do you mean the logical process of going "one", "two", "three"... or do you mean the ability to statistically estimate the amount of quantity by the amount of signal you are processing? E.g. are pigeons actually "counting" as in the process how humans calculate to be accurate? Or are they just responding to the signal? Like similar to how a person could tell whether some sound is higher or lower pitch, but they wouldn't be able to actually numerically say the actual exact frequency. Because to me pigeons are just similarly responding to the amount of "signal" they are receiving, not actually doing abstract reasoning. And looking at the science studies, it also seems that they had to train pigeons to be able to count, they weren't able to do it out of the box.
mtlb 3y ago
The biological neural structures that encode behavior are “trained” through evolution, but even the most advanced animals rely mostly on conditioned (= learned during lifetime) reflexes, and not on the ones “hardcoded” evolutionary. Certainly not much evolutionary “training” in the human brain has happened in the last 3000 years, yet advancement in our understanding of the world has been plentiful. But human thinking (including rationality, mathematics, etc.) is on a different level to even learned animalistic behavior. Some great apes were taught language and even showed basic abstract conceptual thinking, but were never able to reach the level of 3-5 years old human kids. The problem with GPTs and other statistical models is that they can learn incredibly complex patterns in anything we can express as bytes, but not learn the simplest concepts of maths despite being trained on the whole corpus of mathematical texts available on the internet, while kids need classes that can be covered in a single textbook to understand them, and adults may need just a textbook for this.
- mewpmewp2 3y ago
  The claim about those models being "statistical". Why wouldn't you consider human brain to be statistical or animal's brains as "statistical"? Because in the end human brains as well as any brains it seems they could be thought of statistical results from long periods of training and producing output from input. Where am I wrong? I assume by statistical you mean that the ending result of state of neurons can be represented as numbers and pathways leading through these as probabilities of going through a certain pathway - but it occurs to me that same is with human brain, no?
  - mtlb 3y ago
    You are not wrong, but too abstract. Saying that brain is statistical and therefore can be represented with statistical model such as artificial neural net is like saying that brain is computational and therefore can be represented with a Turing-complete system, or that brain is physical and can be represented via physical simulation, etc.

inimino 3y ago

> Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained. No. Animals evolved and are able to do those things. Evolution is not training, and evolution has approximately zero to do with how transformers work.

mewpmewp2 3y ago
What's the difference here? Both seem to have adaptive neural networks where those networks change as time goes on due to a reward - for animals, mutated genes being more likely to be given forward if the change was good. Over millions of generations it's statistically likely that more good genes that caused the neural networks to be in a state that is better able to solve problems within the environment get passed on, eventually resulting in an emerging intelligence. For training you similarly change the state of the neural network depending on whether the answer is good or bad.
- inimino 3y ago
  Evolution operates on genes, which do not encode synaptic connections for one thing. The analogy you're making here is so stretched it's hard to begin to say what's wrong with it. Backpropagation and natural selection are about as different as two things can be. About the only thing you can say they have in common is that both can be modeled as optimization processes. What's the difference between a star and a bonfire? Both use fuel and produce heat and light.
  - mewpmewp2 3y ago
    I mean the point was about LLMs not truly being problem solvers because they were trained to do so as opposed to having been evolved through evolution. I'm looking for what the difference is specifically within that dimension. Biological pigeons had their own process of evolution how they reached to have the type of neural networks and systems in themselves that gave them the ability to count - but not in all contexts for sure. So yes, my point is that both have an optimisation process that through time lend them those emerging capabilities.
    - inimino 3y ago
      No, the point was that LLMs are not good at problem solving because they are not good at problem solving, not because they were not evolved. We don't understand how we or animals solve problems, which is why we haven't yet succeeded in replicating that in AI. You're the one bringing in these incredibly strained analogies, because you want GPT 4 to be more than it is or something, I'm not sure.
- nicklecompte 3y ago
  The bigest difference is that training does not change the size or architecture of an artificial neural network, but biological evolution dramatically changes the size and architecture of animals' brains. Your comparison is sincerely vacuous. It vaguely makes sense if you're talking about GPT-3 to GPT-4 (though I don't think it's helpful). It makes no sense if you're talking about training a single neural network.
  - mewpmewp2 3y ago
    I mean that exposure to a lot of training material yielded in the final set of capabilities. Pigeons and their ancestors were exposed to certain situations throughout evolution that yielded in the formation of neural network and its ability to "count". Which I believe is not actual "one, two, three", but just the amount of signals being activated resulting in a certain output from pigeon. There's a difference in how a human counts, except for small numbers which you can intuitively immediately come up with a number. There was training material which were situations to which organisms had to produce output for and if the output was good their genetics survived, eventually forming the neural network that was able to handle this training material well, but similarly producing emergent behaviour like being able to "count". But GPT-Vision can easily do as well what a Pigeon can. What's the exact thing that implies Pigeon is doing it somehow more intelligently? If you ask them on a picture the quantity of something, I'm pretty sure both respond to the amount of this type of signal received either though light waves or pixels encoded for GPT.
    - RaftPeople 3y ago
      > Aren't animals trained to do all of those things through evolution? Similarly how GPT is trained. If you interpret this question at the most abstract level of "aren't both solutions arrived at through training/trial+error method?" - then the answer is probably yes, they are both arrived at in some conceptually similar manner. But they are two very different underlying systems and we don't really understand the biological systems well enough to even be able to truly compare. Beyond that, it seems that humans (switching to humans from pigeons) have some sort of representation/understanding of the world around us such that even if we produce the same result as ChatGPT to a counting question, the information stored within our systems is not equivalent.

nicklecompte 3y ago

> Also how do you prove that GPT is worse at counting? Back in June 2023 GPT-4 was dramatically worse at counting than a pigeon in the sense that it couldn't accurately tell the difference between sentences with 3 words and sentences with 5 words, whereas pigeons can count almost anything up to about 10. It also routinely failed "pick the shorter sentence" tests which I literally took from a test administered to mice. GPT simply doesn't understand what numbers are, whereas pigeons and mice have an intuitive understanding similar to toddlers. You don't need to teach kids what 3 means, you just need to teach them the human symbol for the concept of 3. GPT only has the human symbol and does not seem capable of understanding the concept. In my testing GPT-4 consistently failed counting / pattern-recognition tests even if you used "chain-of-thought" prompting. As far as I could tell its only true understanding of numbers was "one, two, many." This seems reflected in real use cases, where GPT routinely (and hilariously) ignores commands to return 50 words/etc of output. GPT doesn't know what fifty means, it just knows what various documents that say "word count: 50" look like, and tries to imitate the tone. Since transformer neural networks lack recursion I conjecture that GPT will never be able to understand a number larger than 2, even if in specific cases it can solve counting problems up to eleventy billion. This is what I mean by "counting apples, not oranges," its sense of counting is paper-thin and easily fooled by adversarial prompts. It is much harder to fool a mouse or a pigeon. Many of the tests I ran back in April 2023 no longer work. I strongly suspect this is because OpenAI trained GPT to many of the tests that people were throwing at it, and not because GPT actually became "smarter." I stopped messing around with GPT specifically because OpenAI doesn't issue any release notes, making replicability impossible. Mistrial's 77B model was dramatically worse than even GPT-3 at counting, but I doubt they trained it to count. Not sure about LLaMa/etc.

mewpmewp2 3y ago
When you are talking about "counting", do you mean the logical process of going "one", "two", "three"... or do you mean the ability to statistically estimate the amount of quantity by the amount of signal you are processing? E.g. are pigeons actually "counting" as in the process how humans calculate to be accurate? Or are they just responding to the signal? Like similar to how a person could tell whether some sound is higher or lower pitch, but they wouldn't be able to actually numerically say the actual exact frequency. Because to me pigeons are just similarly responding to the amount of "signal" they are receiving, not actually doing abstract reasoning. And looking at the science studies, it also seems that they had to train pigeons to be able to count, they weren't able to do it out of the box.

mtlb 3y ago

The biological neural structures that encode behavior are “trained” through evolution, but even the most advanced animals rely mostly on conditioned (= learned during lifetime) reflexes, and not on the ones “hardcoded” evolutionary. Certainly not much evolutionary “training” in the human brain has happened in the last 3000 years, yet advancement in our understanding of the world has been plentiful. But human thinking (including rationality, mathematics, etc.) is on a different level to even learned animalistic behavior. Some great apes were taught language and even showed basic abstract conceptual thinking, but were never able to reach the level of 3-5 years old human kids. The problem with GPTs and other statistical models is that they can learn incredibly complex patterns in anything we can express as bytes, but not learn the simplest concepts of maths despite being trained on the whole corpus of mathematical texts available on the internet, while kids need classes that can be covered in a single textbook to understand them, and adults may need just a textbook for this.

mewpmewp2 3y ago
The claim about those models being "statistical". Why wouldn't you consider human brain to be statistical or animal's brains as "statistical"? Because in the end human brains as well as any brains it seems they could be thought of statistical results from long periods of training and producing output from input. Where am I wrong? I assume by statistical you mean that the ending result of state of neurons can be represented as numbers and pathways leading through these as probabilities of going through a certain pathway - but it occurs to me that same is with human brain, no?
- mtlb 3y ago
  You are not wrong, but too abstract. Saying that brain is statistical and therefore can be represented with statistical model such as artificial neural net is like saying that brain is computational and therefore can be represented with a Turing-complete system, or that brain is physical and can be represented via physical simulation, etc.