9 ms·

> Now, the smart money still says it’s done by a team of researchers, and it’s cobbled together over all that. But my reasoning on this is: If you take your ent

> Now, the smart money still says it’s done by a team of researchers, and it’s cobbled together over all that. But my reasoning on this is: If you take your entire DNA, it’s less than a gigabyte of information. So even your entire human body is not all that much in the instructions, and the brain is this tiny slice of it —like 40 megabytes, and it’s not tightly coded. So, we have our existence proof of humanity: What makes our brain, what makes our intelligence, is not all that much code. Nice thought

jhoelzel 4y ago
Well, we could also simply be a lot of RAM interacting with itself until the power runs out. Following that trail of thought, intelligence is an achievement and not a physicality.
thechao 4y ago
I have a tongue-in-cheek assertion that a single human can be smart — perhaps even intelligent. But, what we're experiencing in our society, right now, is an emergent superintelligence that is the culmination of the interaction of 8 billion "merely smart" monkeys. When I'm feeling especially dark, I just assume that most humans aren't even conscious or usefully sapient most of the time, and that continuity of consciousness & sapience is only possible due to the density of our daily interactions. Humans are simple in this model (just like Carmack asserts) because they aren't properly intelligent, sapient, or conscious 100% of the time.
- henriquez 4y ago
  What is consciousness?
  - thechao 4y ago
    I'm home, alone, so I couldn't possibly tell you!
seanhunter 4y ago
A really good example of a smart person reasoning from completely made up assumptions to a punchy-sounding but almost-certainly wildly wrong conclusion. Just because our DNA can be efficiently encoded doesn't mean that our brain is a tiny proportion of that encoding. Your DNA doesn't change much from when you're born to when you die (random degredation aside) and yet your cognative abilities change beyond all recognition. Why is that? Well maybe there's more to what's in the brain than just what's encoded in your DNA. Secondly, how does he get to the 40Mb number? I don't think we know anywhere near enough to know how much information it would take to encode a brain, but 40Mb seems just made up. For starters, consider the amount of random stuff you can remember from your entire life. Are you saying that all can be encoded in just 40Mb? Seems very unlikely.
- vacuumcl 4y ago
  He is saying that the "base code" / DNA which makes up the brain might be only 40MB (for example). Not that what eventually emerges from that (such as our memories and learned abilities) can be captured with just 40MB. It's similar to how the Game of Life can be implemented in just a few lines, but very complex behavior can emerge from that. The key is to find a sufficiently simple but general model from which intelligence equal to our own can emerge given sufficient training.
  - seanhunter 4y ago
    I understand that, but that is an extremely banal observation if you think about it, because the fact that there is this incredible emergent behavior from a simple starting system is the heart of the mystery here. One of the things that everyone is sort of skipping over is the "sufficient training" part. There is no bootstrap reinforcement learning possible for AGI. You can't alphago this sucker and have it play simulations against itself because the whole idea of generality is that there isn't a simple rule framework in which you could run such a simulation. So training any kind of AGI is a really hard problem.
    - anthonypasq 4y ago
bitshiftfaced 4y ago
We also have to import the "physics" package, so maybe it requires more than the estimated 40mb after all.
- sn9 4y ago
  Nah that's just the standard library. Anyone can access it.
throwaway4aday 4y ago
That's only the code describing the molecular hardware which is mostly just a description of a neuron. We don't as yet know how the neurons decide to specialize and organize themselves into a brain but one guess is that they are exploiting the physical principals that govern atomic and molecular interactions so the "code" for this might not even be present in the DNA itself it would be the physics of "a group of molecules of this shape interacting with another group of molecules of that shape will orient and form a boundary that looks like this and that boundary will do X when another molecule with a different shape comes along" the sheer intricacy of such systems is mind boggling. It's like building a working car out of magnets and bubblegum.
- m3kw9 4y ago
  Yeah some sort of physical optimization that depends on forces definitely come into play in organizing the brain structure but at the end we can mimic that in code
  - throwaway4aday 4y ago
    Maybe, I'm still not convinced that artificial neural networks have all the same capabilities as biological ones.
- faitswulff 4y ago
  Don’t forget that life is a self hosting bootstrapped compiler on the trillionth or so generation. The code itself could be almost incidental.
- Balgair 4y ago
  Neuroscientist-y person here. We know a LOT about how neurons organize. Not even close to everything we 'should' know, but we do know a lot. Most of this is in the development of the brain. How you get from one cell to the trillions that make up a person. The real quick and dirty explanation is that cells follow multiple chemical gradients to find their 'home'. This is a HUGE topic though, and I'm being terse as I have a meeting to get to. How adult cells organize also has a LOT of science behind it. Again, though, it's mostly about chemical gradients, with a bit of electrical stuff thrown in. Again, HUGE topic.
serverholic 4y ago
You can’t just say that a human comes from a gigabyte of information. That info is fed into a massively complex physical system, which he is ignoring.
- marviel 4y ago
  I don't disagree that our culture gives us a lot. To use ML terms -- Humans have "Foundation Models" which are composed of: - Their Biological makeup - The culture into which they are raised
  - serverholic 4y ago
    I'm not even talking about culture. DNA is only useful because the laws of physical reality are built a certain way. Without that context the information within DNA is meaningless.
    - marviel 4y ago
      Well sure but couldn't the same thing be said about a computer program?
      - serverholic 4y ago
        A good analogy is perhaps a config file for a computer program. My interpretation is that Carmack is essentially confusing a config file for the computer program itself, then saying "look how small it is, this shouldn't be that hard to reverse engineer".
- kmoser 4y ago
  Yes, this is where Carmack's optimism goes awry. He says: "...as soon as you’re at the point where you have the equivalent of a toddler—something that is a being, it’s conscious, it’s not Einstein, it can’t even do multiplication—if you’ve got a creature that can learn, you can interact with and teach it things on some level." He's wrong. There is currently no practical way to produce a software system that possesses the ability for human thought, reasoning, and motivation without that system possessing uniquely human (let alone organic) properties: the biological and chemical makeup, plus the physical characteristics, of a human, and the ability to process using human senses. (Hint: a neural net processing video images is a mere shadow of how a human processes things with their senses of sight, sound, and touch.) Carmack thinks humans can be reliably reduced to spherical cows in a vacuum, but that only holds true on paper. A real human being is not merely a meat machine: we are driven largely by emotions and physical desires, none of which exist in a computer except through elaborate simulation thereof. Now, I'm sure over the next couple of decades we will make huge strides in mimicking a human being's ability to learn, i.e. creating ever more complex LLMs and AI models that act increasingly more humanlike, but they will be nothing but more and more elaborate parlor tricks which, when prodded just the right way, will fail completely, revealing that they were never human at all. They will be like Avatars (from the movie): sophisticated simulacra that rely on elaborate behind-the-scenes support systems, without which they are useless.

jhoelzel 4y ago

Well, we could also simply be a lot of RAM interacting with itself until the power runs out. Following that trail of thought, intelligence is an achievement and not a physicality.

thechao 4y ago

I have a tongue-in-cheek assertion that a single human can be smart — perhaps even intelligent. But, what we're experiencing in our society, right now, is an emergent superintelligence that is the culmination of the interaction of 8 billion "merely smart" monkeys. When I'm feeling especially dark, I just assume that most humans aren't even conscious or usefully sapient most of the time, and that continuity of consciousness & sapience is only possible due to the density of our daily interactions. Humans are simple in this model (just like Carmack asserts) because they aren't properly intelligent, sapient, or conscious 100% of the time.

henriquez 4y ago
What is consciousness?
- thechao 4y ago
  I'm home, alone, so I couldn't possibly tell you!

seanhunter 4y ago

A really good example of a smart person reasoning from completely made up assumptions to a punchy-sounding but almost-certainly wildly wrong conclusion. Just because our DNA can be efficiently encoded doesn't mean that our brain is a tiny proportion of that encoding. Your DNA doesn't change much from when you're born to when you die (random degredation aside) and yet your cognative abilities change beyond all recognition. Why is that? Well maybe there's more to what's in the brain than just what's encoded in your DNA. Secondly, how does he get to the 40Mb number? I don't think we know anywhere near enough to know how much information it would take to encode a brain, but 40Mb seems just made up. For starters, consider the amount of random stuff you can remember from your entire life. Are you saying that all can be encoded in just 40Mb? Seems very unlikely.

vacuumcl 4y ago
He is saying that the "base code" / DNA which makes up the brain might be only 40MB (for example). Not that what eventually emerges from that (such as our memories and learned abilities) can be captured with just 40MB. It's similar to how the Game of Life can be implemented in just a few lines, but very complex behavior can emerge from that. The key is to find a sufficiently simple but general model from which intelligence equal to our own can emerge given sufficient training.
- seanhunter 4y ago
  I understand that, but that is an extremely banal observation if you think about it, because the fact that there is this incredible emergent behavior from a simple starting system is the heart of the mystery here. One of the things that everyone is sort of skipping over is the "sufficient training" part. There is no bootstrap reinforcement learning possible for AGI. You can't alphago this sucker and have it play simulations against itself because the whole idea of generality is that there isn't a simple rule framework in which you could run such a simulation. So training any kind of AGI is a really hard problem.
  - anthonypasq 4y ago

bitshiftfaced 4y ago

We also have to import the "physics" package, so maybe it requires more than the estimated 40mb after all.

sn9 4y ago
Nah that's just the standard library. Anyone can access it.

throwaway4aday 4y ago

That's only the code describing the molecular hardware which is mostly just a description of a neuron. We don't as yet know how the neurons decide to specialize and organize themselves into a brain but one guess is that they are exploiting the physical principals that govern atomic and molecular interactions so the "code" for this might not even be present in the DNA itself it would be the physics of "a group of molecules of this shape interacting with another group of molecules of that shape will orient and form a boundary that looks like this and that boundary will do X when another molecule with a different shape comes along" the sheer intricacy of such systems is mind boggling. It's like building a working car out of magnets and bubblegum.

m3kw9 4y ago
Yeah some sort of physical optimization that depends on forces definitely come into play in organizing the brain structure but at the end we can mimic that in code
- throwaway4aday 4y ago
  Maybe, I'm still not convinced that artificial neural networks have all the same capabilities as biological ones.
faitswulff 4y ago
Don’t forget that life is a self hosting bootstrapped compiler on the trillionth or so generation. The code itself could be almost incidental.
Balgair 4y ago
Neuroscientist-y person here. We know a LOT about how neurons organize. Not even close to everything we 'should' know, but we do know a lot. Most of this is in the development of the brain. How you get from one cell to the trillions that make up a person. The real quick and dirty explanation is that cells follow multiple chemical gradients to find their 'home'. This is a HUGE topic though, and I'm being terse as I have a meeting to get to. How adult cells organize also has a LOT of science behind it. Again, though, it's mostly about chemical gradients, with a bit of electrical stuff thrown in. Again, HUGE topic.

serverholic 4y ago

You can’t just say that a human comes from a gigabyte of information. That info is fed into a massively complex physical system, which he is ignoring.

marviel 4y ago
I don't disagree that our culture gives us a lot. To use ML terms -- Humans have "Foundation Models" which are composed of: - Their Biological makeup - The culture into which they are raised
- serverholic 4y ago
  I'm not even talking about culture. DNA is only useful because the laws of physical reality are built a certain way. Without that context the information within DNA is meaningless.
  - marviel 4y ago
    Well sure but couldn't the same thing be said about a computer program?
    - serverholic 4y ago
      A good analogy is perhaps a config file for a computer program. My interpretation is that Carmack is essentially confusing a config file for the computer program itself, then saying "look how small it is, this shouldn't be that hard to reverse engineer".
kmoser 4y ago
Yes, this is where Carmack's optimism goes awry. He says: "...as soon as you’re at the point where you have the equivalent of a toddler—something that is a being, it’s conscious, it’s not Einstein, it can’t even do multiplication—if you’ve got a creature that can learn, you can interact with and teach it things on some level." He's wrong. There is currently no practical way to produce a software system that possesses the ability for human thought, reasoning, and motivation without that system possessing uniquely human (let alone organic) properties: the biological and chemical makeup, plus the physical characteristics, of a human, and the ability to process using human senses. (Hint: a neural net processing video images is a mere shadow of how a human processes things with their senses of sight, sound, and touch.) Carmack thinks humans can be reliably reduced to spherical cows in a vacuum, but that only holds true on paper. A real human being is not merely a meat machine: we are driven largely by emotions and physical desires, none of which exist in a computer except through elaborate simulation thereof. Now, I'm sure over the next couple of decades we will make huge strides in mimicking a human being's ability to learn, i.e. creating ever more complex LLMs and AI models that act increasingly more humanlike, but they will be nothing but more and more elaborate parlor tricks which, when prodded just the right way, will fail completely, revealing that they were never human at all. They will be like Avatars (from the movie): sophisticated simulacra that rely on elaborate behind-the-scenes support systems, without which they are useless.