4 ms·

This simply does not pass the smell test. If you want to analogize biological brains to ANNs then clearly evolution refines the architecture of the "natural neu

This simply does not pass the smell test. If you want to analogize biological brains to ANNs then clearly evolution refines the architecture of the "natural neural network," not the data. No ANN training involves adding artificial neurons, defining new types of artificial neurons, etc, but that clearly describes the biological history of the brain. Taking this article literally, the brain hasn't really changed much since nematodes, it's just absorbed a ton of data. That's transparently stupid. All species, including humans and worms, have evolved over billions of years to get where they are today. Why is it that only humans get to access the billions of years of data? I am guessing the author has childish views about humans being "the most evolved" species. But all species are more evolved than they were 1m years ago. This entire article relies on a dumb bait-and-switch. It's an incoherent analogy which seems motivated by a desire to simply excuse away the shortcomings of transformers.

danielmarkbruce 2y ago
> clearly evolution refines the architecture of the "natural neural network," not the data This doesn't pass the smell test. Common sense and experience suggest animals have both built in functionality and a built in ability to learn and they seem to work together. Babies don't learn to cry for food, it works right out of the box. Some animals work with practically zero parental guidance. This is a pretty silly statement overall. Just being alive and paying attention should give someone a sense for how silly it is.
- aithrowawaycomm 2y ago
  I genuinely don't understand your point at all. I am not contesting that instincts exist, and I am not sure what you think the disagreement is. Let me rephrase: There have been two sides to ANN development: architecture and data. If we're analogizing brains to ANNs like the author is doing, then brains also have an "architectural" component and a "data" component. But we need to be clear that evolution (including individual mutations and epigenetics) shapes the architecture, while lived experience shapes the data. The author is claiming that the experience of billions of years of evolution should actually go into the "data" column, somehow, and that human brains at birth have actually learned from the experience of rodent-lizards that lived 200 million years ago. This is just idiotic, and it does not help that the architecture is completely ignored. Again the problem is a fundamental bait-and-switch in the analogy itself. Instinctual behavior is much better-explained by evolution influencing the architecture of the brain, rather than waving your hands and saying "data, lots of it!" The crying baby is much better explained by an ancestor to all birds and mammals who had a neurological mutation that made it whine (perhaps softly) when it was hungry, not because that lizard finally accumulated enough "evolutionary data" to learn how to cry. (What would that data be?) The neurological mutation is of course purely speculative, but it is plausible, which is not the case for what the author is saying. The author claims quite directly that humans learn from billions of years of animal experiences in a very similar way to how GPT learns from billions of lines of text, which is a preposterous claim that requires extraordinary evidence. No evidence is provided. I am confident that this Substack is quackery.
  - danielmarkbruce 2y ago
    There are clearly a lot of things you don't understand. The word "learn" is extremely broadly defined in current use. Evolution is a form of learning, at the population level (given the broad definition of learning). https://chatgpt.com/share/6705bfa0-67e8-8002-b570-010e7a25dbe8 https://chatgpt.com/share/6705bfa0-67e8-8002-b570-010e7a25db... The data is either the genes or the combination of genes and environment/laws of physics. It's laid out in the post. This blog post may or may not be right, but the idea that it's some obviously stupid idea is obviously stupid. It's not even novel - this idea has been around.
jcgrillo 2y ago
> architecture of the "natural neural network" not the data I believe you're right, but to pick a nit it's not just the architecture of the meatNN it's also the sensors that input data to it, the channels those sensors send the data over, and the motor control mechanisms that enable the meatNN to do interesting sensor feedback experiments. It's the whole damn thing that's been optimized to work as a coherent unit by billions of years of evolution.

danielmarkbruce 2y ago

> clearly evolution refines the architecture of the "natural neural network," not the data This doesn't pass the smell test. Common sense and experience suggest animals have both built in functionality and a built in ability to learn and they seem to work together. Babies don't learn to cry for food, it works right out of the box. Some animals work with practically zero parental guidance. This is a pretty silly statement overall. Just being alive and paying attention should give someone a sense for how silly it is.

aithrowawaycomm 2y ago
I genuinely don't understand your point at all. I am not contesting that instincts exist, and I am not sure what you think the disagreement is. Let me rephrase: There have been two sides to ANN development: architecture and data. If we're analogizing brains to ANNs like the author is doing, then brains also have an "architectural" component and a "data" component. But we need to be clear that evolution (including individual mutations and epigenetics) shapes the architecture, while lived experience shapes the data. The author is claiming that the experience of billions of years of evolution should actually go into the "data" column, somehow, and that human brains at birth have actually learned from the experience of rodent-lizards that lived 200 million years ago. This is just idiotic, and it does not help that the architecture is completely ignored. Again the problem is a fundamental bait-and-switch in the analogy itself. Instinctual behavior is much better-explained by evolution influencing the architecture of the brain, rather than waving your hands and saying "data, lots of it!" The crying baby is much better explained by an ancestor to all birds and mammals who had a neurological mutation that made it whine (perhaps softly) when it was hungry, not because that lizard finally accumulated enough "evolutionary data" to learn how to cry. (What would that data be?) The neurological mutation is of course purely speculative, but it is plausible, which is not the case for what the author is saying. The author claims quite directly that humans learn from billions of years of animal experiences in a very similar way to how GPT learns from billions of lines of text, which is a preposterous claim that requires extraordinary evidence. No evidence is provided. I am confident that this Substack is quackery.
- danielmarkbruce 2y ago
  There are clearly a lot of things you don't understand. The word "learn" is extremely broadly defined in current use. Evolution is a form of learning, at the population level (given the broad definition of learning). https://chatgpt.com/share/6705bfa0-67e8-8002-b570-010e7a25dbe8 https://chatgpt.com/share/6705bfa0-67e8-8002-b570-010e7a25db... The data is either the genes or the combination of genes and environment/laws of physics. It's laid out in the post. This blog post may or may not be right, but the idea that it's some obviously stupid idea is obviously stupid. It's not even novel - this idea has been around.

jcgrillo 2y ago

> architecture of the "natural neural network" not the data I believe you're right, but to pick a nit it's not just the architecture of the meatNN it's also the sensors that input data to it, the channels those sensors send the data over, and the motor control mechanisms that enable the meatNN to do interesting sensor feedback experiments. It's the whole damn thing that's been optimized to work as a coherent unit by billions of years of evolution.