6 ms·

I'm not so sure about the simulation, but Ray's right that the brain cannot be more complex than the data that specifies it, speaking information-theoretically.

by 10ren 16y ago

I'm not so sure about the simulation, but Ray's right that the brain cannot be more complex than the data that specifies it, speaking information-theoretically. A biologist probably wouldn't get this, because he has too much knowledge of how difficult and complex the actual translation is. It's a mathematical idea, like non-constructive proofs, which freak out sensible folk - and rightfully so. (EDIT no offence intended, they freak me out too) The only other source of information is the non-genomic environment - extra-nucleur DNA like mitochondria, and the womb (which is arguably already specified in the genome, unless mother nature has done a Ken Thompson http://cm.bell-labs.com/who/ken/trust.html http://cm.bell-labs.com/who/ken/trust.html at some point.) But it's weird to claim that 50% of our genome encodes the brain. Really? Perhaps it's just that 50% is required by the brain, much of it being foundational to the whole organism (like standard libraries.)

phreeza 16y ago
I see it as an upper bound for the Kolmogorov Complexity of the brains structure. I am very dubious that the first successful attempts at doing this will be as efficient as the genetic code though...
roadnottaken 16y ago
This strikes me as very similar to a Mandelbrot fractal. The fractal is overwhelmingly complex and intricate, but wholly mathematically described by a very simple formula. The complexity of the brain is similar: A few simple cells and proteins can interact in extraordinarily complex ways. Some biologists understand this just fine, thank you very much.
- roadnottaken 16y ago
  In fact, some biologists have even written books on the subject: http://books.google.com/books?id=lZcSpRJz0dgC&lpg=PP1&dq=isbn%3A0195079515&pg=PP1#v=onepage&q&f=false http://books.google.com/books?id=lZcSpRJz0dgC&lpg=PP1&#3...
aswanson 16y ago
But even looking at it in an information-theoretic way, what is being stated is that the instruction set is far more powerful than the binary analogy Kurzweil is using. The instruction I give: "Cure Cancer" has 11 bytes. But that instruction implies a ridiculously complicated set of sequences that we haven't figured out yet. In this situation, look at the genome as the instruction set for protein construction and folding, an ongoing problem in research we have just begun to investigate. The information contained in the genome is combinatorially descriptive and therefore not as simple as made out to be, if you define information as the amount of "surprise" in the outcome.
- nkassis 16y ago
  I don't think it can be compared to instructions. More like just the startup conditions, like a rom or something. All analogies that compare computers to biological things are terrible. Mine is as much so as any other.
KirinDave 16y ago
> I'm not so sure about the simulation, but Ray's right that the brain cannot be more complex than the data that specifies it Which would be true if the DNA specification for that particular part of the body was the only part of what specifies the brain. Myers is pointing out that assertion is patently false. The environment of developing creature and the interactions between cell types and their environment (and themselves) is a giant information content multiplier, and the DNA need not explicitly specify any of this information for it to exist and be relevant. Bringing this to a familiar compsci example; imagine software for creating neural net recognizers. You can look at the source code for a net and say, "This will take N inputs and produce N outputs" You can look at a finished classifier and say, "Ah, I see what this does! It tells the airbags in this car when to deploy!" But that's as far as you can go without the training data that was used to train the classifier. This is a doubly good example because it's often very difficult to determine HOW a complex neural net is doing what it does, but it's fairly easy to explain how to train one to do that task.
- sown 16y ago
  Is this sort of like having a pre-processor and system code to get a program loaded, linked and running?
  - KirinDave 16y ago
    Perhaps, only the pre-processor would be in a feedback loop with the early parts stages of the program life-cycle. Typically the linker and pre-processor are one way operations. In a developing organism, the environment feeds back on the cells developing and triggers new responses which causes new environmental changes which causes new feedback. Biology is full of bizarre examples of when this process goes awry and our bodies have bizarre features. Dawkins's example of the Laryngeal nerve that makes a crazy loop down into the mammalian chest cavity is the classic example.
  - endtime 16y ago
    Maybe, if you consider the pre-processor to be, well, reality...
- 16y ago
itistoday 16y ago
but Ray's right that the brain cannot be more complex than the data that specifies it, speaking information-theoretically. You're conflating two very different concepts of complexity here. One is the complexity of a static state of information, and the other is the complexity arising from dynamical systems. As roadnottaken pointed out, fractals are perfect examples of systems that are described by very "simple" formulas, yet contain infinite complexity. It could therefore be said that the simple equation of the Mandelbrot set represents infinite complexity. However, if you take a particular iteration of the formula, then you can get a finite concept of its complexity, i.e. how many bits it takes to represent the image you're seeing. So, to say that the "the brain cannot be more complex than the data that specifies it" is true in one sense, but completely useless in another. To put this into terms of the Mandelbrot, you can gather up all the bits that represents some particular iteration of the Mandelbrot, but that doesn't tell you anything about how it works, or even how to generate the next frame. You need the equation for that. That's just one place where Ray fails. The second is the lingering question of whether computers in their current state are even capable of "simulating" a brain. It's still not an answered question of what the role of the non-determinism found in Quantum Mechanics plays in the brain and the interactions of various chemicals. It was recently shown that DNA relies on QM entanglement to "hold it together". If it turns out that non-determinism and QM effects play a crucial role in biology (which they almost certainly do), then the very rigid and deterministic system that is the CPU may simply be incapable of simulating a human brain.
- canadaduane 16y ago
  The objective is to simulate the real-time activities of the brain, not the evolutionary-time activities that were and are acting on the brain. Continuing from your extension of the fractal analogy, the former is more like an "iteration of the Mandelbrot" while the latter is "how to generate the next frame". We do not need to know how our human-precursor brains worked, nor do we need to know what our human-successor brains will be like to successfully simulate current-human-brain intelligence. It seems plausible to me that we will be able to understand how to simulate the functions of the brain without necessarily simulating the physical universe and its remarkable evolutionary unfolding--which seems to be the ultimate level of complexity and one that I agree is far beyond us.
dkarl 16y ago
The only other source of information is the non-genomic environment - extra-nucleur DNA like mitochondria, and the womb (which is arguably already specified in the genome, unless mother nature has done a Ken Thompson http://cm.bell-labs.com/who/ken/trust.html http://cm.bell-labs.com/who/ken/trust.html at some point.) That just gives you a newborn baby's brain, which is a pretty poor standard for displaying "human" intelligence. If we didn't get any smarter than that, we'd be pretty dumb by animal standards. To get to "human" intelligence, you have to be able to simulate a rich environment for the brain to learn from. You also have to model the growth of the brain and its response to stimulus -- the physics, chemistry, and biochemistry of the brain. DNA doesn't have to do that, because it runs on a platform with that functionality built-in (i.e., the real world.)
- jerf 16y ago
  Yes, but it gives you the newborn baby's brain in toto, including its ability to grow up into a normal adult human. If you get to newborn's brain you are 99.9% of the way there from the AI side. By the time we get there, feeding it stimulus will be a relatively simple problem by comparison.
  - dkarl 16y ago
    Not if the world is actually more complicated than a newborn baby's brain. Can we simulate a baby's interaction with its mother without simulating the mother's brain?
    - tbrownaw 16y ago
      Yup, wire it up to a baby-shaped I/O package and ask one of your grad students to take it home with her.
    - jerf 16y ago
      We have a world in hand. We're not trying to build a world simulation, we're trying to build intelligences. By the time we get this far, the infant will probably be embodied, and we can use real "mothers". Some speculate that a non-embodied being can never become intelligent. Bear in mind we are talking about at least 20 years hence, in my mind.
cdavid 16y ago
You are making the assumption that the information quantity is a good measure of the complexity, which is far from obvious. It is also wrong in general for the genetic code (for example, many organisms have a much bigger genetic code than human, even though they are arguably much simpler - e.g. rice has more encoding genes than human). Also, having a few set of simple rules is not enough to understand or reproduce a system in general.
Keyframe 16y ago
> but Ray's right that the brain cannot be more complex than the data that specifies it Indeed, however due to the Kolmogorov complexity http://en.wikipedia.org/wiki/Kolmogorov_complexity http://en.wikipedia.org/wiki/Kolmogorov_complexity one can argue both for DNA being a resource and, with more common sense, that DNA and all appropriate resources during a lifetime up to a point where you take a measure of a whole brain all funnels into a resource from which you can describe a brain. And this is only about Kolmogorov Complexity and under assumption that brain is a discrete set. I am not much well read on biology, but I think there were recent advancements where some organisms also showed quantum effects playing a biological role. Even with disregarding our current lack of knowledge we can still argue if brain is a discrete set or not, proving either would be a major contributing factor in our understanding of it I think.
deleted 16y ago
[deleted]
JabavuAdams 16y ago
Where is the information describing the machine ABI in the following legal C++ program? "#include <iostream>\nusing namespace std; int main() { cout << "Hello, world." << endl; return 0;}\n" Next question: how large is the pre-processor / compiler / linker / runtime, relative to the given program. The point is that this idea that you can just look at the information-content of the DNA is so dumb it's not even wrong. It's funny that you should mention (and dismiss) the Ken Thompson hack, because that seems to be a much better analogy. Even if 100% of our genome encoded the brain, its information-content would be a ridiculous under-estimate of the organism's complexity. Note that I'm not saying anything about whether we actually need to simulate the brain to achieve AI.

by 10ren 16y ago

phreeza 16y ago

I see it as an upper bound for the Kolmogorov Complexity of the brains structure. I am very dubious that the first successful attempts at doing this will be as efficient as the genetic code though...

roadnottaken 16y ago

This strikes me as very similar to a Mandelbrot fractal. The fractal is overwhelmingly complex and intricate, but wholly mathematically described by a very simple formula. The complexity of the brain is similar: A few simple cells and proteins can interact in extraordinarily complex ways. Some biologists understand this just fine, thank you very much.

roadnottaken 16y ago
In fact, some biologists have even written books on the subject: http://books.google.com/books?id=lZcSpRJz0dgC&lpg=PP1&dq=isbn%3A0195079515&pg=PP1#v=onepage&q&f=false http://books.google.com/books?id=lZcSpRJz0dgC&lpg=PP1&#3...

aswanson 16y ago

But even looking at it in an information-theoretic way, what is being stated is that the instruction set is far more powerful than the binary analogy Kurzweil is using. The instruction I give: "Cure Cancer" has 11 bytes. But that instruction implies a ridiculously complicated set of sequences that we haven't figured out yet. In this situation, look at the genome as the instruction set for protein construction and folding, an ongoing problem in research we have just begun to investigate. The information contained in the genome is combinatorially descriptive and therefore not as simple as made out to be, if you define information as the amount of "surprise" in the outcome.

nkassis 16y ago
I don't think it can be compared to instructions. More like just the startup conditions, like a rom or something. All analogies that compare computers to biological things are terrible. Mine is as much so as any other.

KirinDave 16y ago

> I'm not so sure about the simulation, but Ray's right that the brain cannot be more complex than the data that specifies it Which would be true if the DNA specification for that particular part of the body was the only part of what specifies the brain. Myers is pointing out that assertion is patently false. The environment of developing creature and the interactions between cell types and their environment (and themselves) is a giant information content multiplier, and the DNA need not explicitly specify any of this information for it to exist and be relevant. Bringing this to a familiar compsci example; imagine software for creating neural net recognizers. You can look at the source code for a net and say, "This will take N inputs and produce N outputs" You can look at a finished classifier and say, "Ah, I see what this does! It tells the airbags in this car when to deploy!" But that's as far as you can go without the training data that was used to train the classifier. This is a doubly good example because it's often very difficult to determine HOW a complex neural net is doing what it does, but it's fairly easy to explain how to train one to do that task.

sown 16y ago
Is this sort of like having a pre-processor and system code to get a program loaded, linked and running?
- KirinDave 16y ago
  Perhaps, only the pre-processor would be in a feedback loop with the early parts stages of the program life-cycle. Typically the linker and pre-processor are one way operations. In a developing organism, the environment feeds back on the cells developing and triggers new responses which causes new environmental changes which causes new feedback. Biology is full of bizarre examples of when this process goes awry and our bodies have bizarre features. Dawkins's example of the Laryngeal nerve that makes a crazy loop down into the mammalian chest cavity is the classic example.
- endtime 16y ago
  Maybe, if you consider the pre-processor to be, well, reality...
16y ago

itistoday 16y ago

but Ray's right that the brain cannot be more complex than the data that specifies it, speaking information-theoretically. You're conflating two very different concepts of complexity here. One is the complexity of a static state of information, and the other is the complexity arising from dynamical systems. As roadnottaken pointed out, fractals are perfect examples of systems that are described by very "simple" formulas, yet contain infinite complexity. It could therefore be said that the simple equation of the Mandelbrot set represents infinite complexity. However, if you take a particular iteration of the formula, then you can get a finite concept of its complexity, i.e. how many bits it takes to represent the image you're seeing. So, to say that the "the brain cannot be more complex than the data that specifies it" is true in one sense, but completely useless in another. To put this into terms of the Mandelbrot, you can gather up all the bits that represents some particular iteration of the Mandelbrot, but that doesn't tell you anything about how it works, or even how to generate the next frame. You need the equation for that. That's just one place where Ray fails. The second is the lingering question of whether computers in their current state are even capable of "simulating" a brain. It's still not an answered question of what the role of the non-determinism found in Quantum Mechanics plays in the brain and the interactions of various chemicals. It was recently shown that DNA relies on QM entanglement to "hold it together". If it turns out that non-determinism and QM effects play a crucial role in biology (which they almost certainly do), then the very rigid and deterministic system that is the CPU may simply be incapable of simulating a human brain.

canadaduane 16y ago
The objective is to simulate the real-time activities of the brain, not the evolutionary-time activities that were and are acting on the brain. Continuing from your extension of the fractal analogy, the former is more like an "iteration of the Mandelbrot" while the latter is "how to generate the next frame". We do not need to know how our human-precursor brains worked, nor do we need to know what our human-successor brains will be like to successfully simulate current-human-brain intelligence. It seems plausible to me that we will be able to understand how to simulate the functions of the brain without necessarily simulating the physical universe and its remarkable evolutionary unfolding--which seems to be the ultimate level of complexity and one that I agree is far beyond us.

dkarl 16y ago

The only other source of information is the non-genomic environment - extra-nucleur DNA like mitochondria, and the womb (which is arguably already specified in the genome, unless mother nature has done a Ken Thompson http://cm.bell-labs.com/who/ken/trust.html http://cm.bell-labs.com/who/ken/trust.html at some point.) That just gives you a newborn baby's brain, which is a pretty poor standard for displaying "human" intelligence. If we didn't get any smarter than that, we'd be pretty dumb by animal standards. To get to "human" intelligence, you have to be able to simulate a rich environment for the brain to learn from. You also have to model the growth of the brain and its response to stimulus -- the physics, chemistry, and biochemistry of the brain. DNA doesn't have to do that, because it runs on a platform with that functionality built-in (i.e., the real world.)

jerf 16y ago
Yes, but it gives you the newborn baby's brain in toto, including its ability to grow up into a normal adult human. If you get to newborn's brain you are 99.9% of the way there from the AI side. By the time we get there, feeding it stimulus will be a relatively simple problem by comparison.
- dkarl 16y ago
  Not if the world is actually more complicated than a newborn baby's brain. Can we simulate a baby's interaction with its mother without simulating the mother's brain?
  - tbrownaw 16y ago
    Yup, wire it up to a baby-shaped I/O package and ask one of your grad students to take it home with her.
  - jerf 16y ago
    We have a world in hand. We're not trying to build a world simulation, we're trying to build intelligences. By the time we get this far, the infant will probably be embodied, and we can use real "mothers". Some speculate that a non-embodied being can never become intelligent. Bear in mind we are talking about at least 20 years hence, in my mind.

cdavid 16y ago

You are making the assumption that the information quantity is a good measure of the complexity, which is far from obvious. It is also wrong in general for the genetic code (for example, many organisms have a much bigger genetic code than human, even though they are arguably much simpler - e.g. rice has more encoding genes than human). Also, having a few set of simple rules is not enough to understand or reproduce a system in general.

Keyframe 16y ago

> but Ray's right that the brain cannot be more complex than the data that specifies it Indeed, however due to the Kolmogorov complexity http://en.wikipedia.org/wiki/Kolmogorov_complexity http://en.wikipedia.org/wiki/Kolmogorov_complexity one can argue both for DNA being a resource and, with more common sense, that DNA and all appropriate resources during a lifetime up to a point where you take a measure of a whole brain all funnels into a resource from which you can describe a brain. And this is only about Kolmogorov Complexity and under assumption that brain is a discrete set. I am not much well read on biology, but I think there were recent advancements where some organisms also showed quantum effects playing a biological role. Even with disregarding our current lack of knowledge we can still argue if brain is a discrete set or not, proving either would be a major contributing factor in our understanding of it I think.

deleted 16y ago

[deleted]

JabavuAdams 16y ago

Where is the information describing the machine ABI in the following legal C++ program? "#include <iostream>\nusing namespace std; int main() { cout << "Hello, world." << endl; return 0;}\n" Next question: how large is the pre-processor / compiler / linker / runtime, relative to the given program. The point is that this idea that you can just look at the information-content of the DNA is so dumb it's not even wrong. It's funny that you should mention (and dismiss) the Ken Thompson hack, because that seems to be a much better analogy. Even if 100% of our genome encoded the brain, its information-content would be a ridiculous under-estimate of the organism's complexity. Note that I'm not saying anything about whether we actually need to simulate the brain to achieve AI.