4 ms·

The argument Myers is making is that while the DNA might be the input to the system, the total amount of data in the system is that input plus the rules around

by akeefer 16y ago

The argument Myers is making is that while the DNA might be the input to the system, the total amount of data in the system is that input plus the rules around how that input is interpreted/works. Those rules (for example around protein folding) are currently encoded in biological systems as the laws of physics, more or less, but they're insanely complicated and currently unknown. So the point is that perhaps if you had a system that simulated all of the laws of physics exactly correctly such that proteins folded and interacted exactly right, only then could you get away with an amount of input equivalent to the amount of information encoded in the part of our DNA related to the brain. Actually encoding those rules is probably the harder part of the problem, and could easily take several orders of magnitude more work. (10x? 1,000,000x? Who even knows).

ewjordan 16y ago
1) An estimate of complexity is what it is - an estimate of complexity. It's not a claim that the way to achieve AI is to figure out the details of the particular encoding that nature ended up using, so the precise nature of those rules is not something we care about. 2) While it's true that those runtime rules (which we can kind of consider as the "interpreter" for our DNA) are extremely complex, this has almost zero bearing on the informational content in our DNA that is put towards creating the "intelligence algorithm", whatever that is. Sure, there's probably a bit of extra compression based on the fact that the physics allows some actions to be "built in", but unless you believe that DNA is physically optimized to make intelligent computer construction very concise, the logical content of these computations is probably explicitly "written" in. And it's hard to believe that DNA is somehow specifically optimized for intelligence, because it was first used in completely unintelligent creatures and appears in exactly the same form now. Now, it may be the case that DNA's physics are tailor-made to efficiently code for useful physical structures. But intelligence is a level of abstraction above that, and we're all but guaranteed that very little compressibility exists in the "language" for such higher level constructs. What would an argument be without a strained analogy: if you're writing a complex web application, the size of your application is roughly independent (within an order of magnitude, for sure) of the architecture that it will ultimately run on (where by "size of application" I mean the size of everything that it takes to run it, interpreters, frameworks, etc.). Sure, the binary size might be slightly different depending on whether you're writing it for ARM, PPC, x86, etc., but not hugely different. We would be extremely surprised if on three platforms your executable weighed in at 10 mb and on a fourth (which had a few different machine level instructions) it compiled down to 10 kb - the only way we could imagine that happening is if someone somehow "cheated" and embedded large parts of your actual application logic into the processor, adding specialized Ruby on Rails instructions to the machine code, or something like that. :) Encoding and dynamics details may make differences in compressibility, but past an order of magnitude, you're really talking about "cheating", and it's an Occam's Razor problem to assume that nature optimized in such a way for intelligence...
- akeefer 16y ago
  I think the article's original point (and mine as well) was that considering the size of the code (i.e. the DNA) as a measure of the complexity of the task is totally disingenuous when you don't have (to use your analogy) the web server, the libraries you're calling, the parser/compiler/linker for the language, the operating system for the server along with its drivers/TCP stack/etc., the processor it runs on, the mother board, or the storage. In order to turn 10,000 lines of code into a web application, you need millions of lines of code (and Verilog or whathaveyou) in terms of infrastructure. The problem for AI is not just encoding the DNA, as it were, it's in building all those other pieces around it. Estimating the complexity of building a software brain based on the amount of information in DNA is like estimating the complexity of building a web application using 1950's hardware. "It's only 10,000 lines of code! How hard can that be? All we have to do is write the code, plus the frameworks, programming language, and operating system, plus do all the hardware design."
  - rayval 16y ago
    Good point. Not only that, but a case can be made that one cannot build such as system as an end stage. Instead, it appears that ontogeny must recapitulate phylogeny. The system must develop over time as a result of inputs (and the remembered collection of past inputs encoded in the DNA). It would be as if in order to build Twitter with Ruby on Rails, you first had to program a tax calculation application in Cobol on a 1950s mainframe.
  - ewjordan 16y ago
    It's only 10,000 lines of code! How hard can that be? All we have to do is write the code, plus the frameworks, programming language, and operating system, plus do all the hardware design. Except that DNA doesn't even come close to being a high level language, since the low level details were not specifically designed for compressibility of the code (in fact, the low level details, the "bare metal ops", are pretty much fixed by the for-all-intents-and-purposes random laws of physics, which means we shouldn't assume that they enable any particularly high compressibility ratios for anything). So a more apt comparison would be if we saw an assembly language program in some strange incomprehensible assembly language and said "It's only 10,000 lines of operations on the bare metal! Now all we have to do is figure out how the hell the system this runs on works, and how we can translate that code into a more sensible (and probably vastly more compact) form." ...which might even be a harder problem, to be fair. Kurzweil's essentially proposed evidence of existence of an algorithm of length N that does whatever it is we mean by intelligence. Which is fine, and I think is probably correct (IMO, even his estimate about the minimal amount of code it would take is probably too high, though that's another story). But he's overlooking the fact that the mere existence of such a compact algorithm doesn't help us find it at all, and I think a lot of the complaints others have made about his statements are more aimed at that leap of logic, not the existence claim itself. I completely agree that even brain scanning tech might not help us simulate the important bits very well, even if we did have access to that tech and computers fast enough to run the sims.
10ren 16y ago
So it's a bit like a program, with sequences also selecting a different Turing machine? (which determines how that subsequence is interpreted.) Because the Turing machine is selected entirely by the sequence (the protein folding caused by the laws of physics is selected entirely by the sequence), the number of possible results (the number of different shapes that could result) is limited to the number of different sequences. That is, the information in the phenome seems to be limited by the information in the genome. If you think of it as a two part message, with the first part encoding a model, and the second part configuring it, then the DNA can be seen as the configuration, and the laws of physics as the model (which isn't actually coded anywhere like DNA - we'd have to write that ourselves.) This model is constant over all life, so that DNA from all species (plants and animals) share the same "model" (laws of physics that cause protein folding etc.) Another example of a two-part message is that the first part is a programming language, and the second part is a program written in that language. For a high level language (esp with libraries), it's obvious that a very short program might do an awful lot; but the true information content is not that program alone, but the total including the language and libraries it uses. However, and this is my point, I don't believe that the laws of physics have been constructed so conveniently that provide as much assistance as a high level language with libraries. At most, nature may have stumbled onto hacks in physics (like surface tension, interfaces and gradients) and exploited them. Actually, given how long it took to get life started, perhaps it had to find a whole bunch of clever hacks (randomly recombining for billions of years over a whole planet) before it came up with a workable model (that is the model that DNA configure.) hmmm... we might be able to estimate the information content of the 'model' by how many tries it took to come across it.
- jacquesm 16y ago
  > That is, the information in the phenome seems to be limited by the information in the genome. I think that is a very original use of the word 'limited', limited in this case holds enough room for random chance to come up with human beings. For all practical purposes that 'limited' might as well be unlimited.
  - 10ren 16y ago

by akeefer 16y ago

ewjordan 16y ago

1) An estimate of complexity is what it is - an estimate of complexity. It's not a claim that the way to achieve AI is to figure out the details of the particular encoding that nature ended up using, so the precise nature of those rules is not something we care about. 2) While it's true that those runtime rules (which we can kind of consider as the "interpreter" for our DNA) are extremely complex, this has almost zero bearing on the informational content in our DNA that is put towards creating the "intelligence algorithm", whatever that is. Sure, there's probably a bit of extra compression based on the fact that the physics allows some actions to be "built in", but unless you believe that DNA is physically optimized to make intelligent computer construction very concise, the logical content of these computations is probably explicitly "written" in. And it's hard to believe that DNA is somehow specifically optimized for intelligence, because it was first used in completely unintelligent creatures and appears in exactly the same form now. Now, it may be the case that DNA's physics are tailor-made to efficiently code for useful physical structures. But intelligence is a level of abstraction above that, and we're all but guaranteed that very little compressibility exists in the "language" for such higher level constructs. What would an argument be without a strained analogy: if you're writing a complex web application, the size of your application is roughly independent (within an order of magnitude, for sure) of the architecture that it will ultimately run on (where by "size of application" I mean the size of everything that it takes to run it, interpreters, frameworks, etc.). Sure, the binary size might be slightly different depending on whether you're writing it for ARM, PPC, x86, etc., but not hugely different. We would be extremely surprised if on three platforms your executable weighed in at 10 mb and on a fourth (which had a few different machine level instructions) it compiled down to 10 kb - the only way we could imagine that happening is if someone somehow "cheated" and embedded large parts of your actual application logic into the processor, adding specialized Ruby on Rails instructions to the machine code, or something like that. :) Encoding and dynamics details may make differences in compressibility, but past an order of magnitude, you're really talking about "cheating", and it's an Occam's Razor problem to assume that nature optimized in such a way for intelligence...

akeefer 16y ago
I think the article's original point (and mine as well) was that considering the size of the code (i.e. the DNA) as a measure of the complexity of the task is totally disingenuous when you don't have (to use your analogy) the web server, the libraries you're calling, the parser/compiler/linker for the language, the operating system for the server along with its drivers/TCP stack/etc., the processor it runs on, the mother board, or the storage. In order to turn 10,000 lines of code into a web application, you need millions of lines of code (and Verilog or whathaveyou) in terms of infrastructure. The problem for AI is not just encoding the DNA, as it were, it's in building all those other pieces around it. Estimating the complexity of building a software brain based on the amount of information in DNA is like estimating the complexity of building a web application using 1950's hardware. "It's only 10,000 lines of code! How hard can that be? All we have to do is write the code, plus the frameworks, programming language, and operating system, plus do all the hardware design."
- rayval 16y ago
  Good point. Not only that, but a case can be made that one cannot build such as system as an end stage. Instead, it appears that ontogeny must recapitulate phylogeny. The system must develop over time as a result of inputs (and the remembered collection of past inputs encoded in the DNA). It would be as if in order to build Twitter with Ruby on Rails, you first had to program a tax calculation application in Cobol on a 1950s mainframe.
- ewjordan 16y ago
  It's only 10,000 lines of code! How hard can that be? All we have to do is write the code, plus the frameworks, programming language, and operating system, plus do all the hardware design. Except that DNA doesn't even come close to being a high level language, since the low level details were not specifically designed for compressibility of the code (in fact, the low level details, the "bare metal ops", are pretty much fixed by the for-all-intents-and-purposes random laws of physics, which means we shouldn't assume that they enable any particularly high compressibility ratios for anything). So a more apt comparison would be if we saw an assembly language program in some strange incomprehensible assembly language and said "It's only 10,000 lines of operations on the bare metal! Now all we have to do is figure out how the hell the system this runs on works, and how we can translate that code into a more sensible (and probably vastly more compact) form." ...which might even be a harder problem, to be fair. Kurzweil's essentially proposed evidence of existence of an algorithm of length N that does whatever it is we mean by intelligence. Which is fine, and I think is probably correct (IMO, even his estimate about the minimal amount of code it would take is probably too high, though that's another story). But he's overlooking the fact that the mere existence of such a compact algorithm doesn't help us find it at all, and I think a lot of the complaints others have made about his statements are more aimed at that leap of logic, not the existence claim itself. I completely agree that even brain scanning tech might not help us simulate the important bits very well, even if we did have access to that tech and computers fast enough to run the sims.

10ren 16y ago

So it's a bit like a program, with sequences also selecting a different Turing machine? (which determines how that subsequence is interpreted.) Because the Turing machine is selected entirely by the sequence (the protein folding caused by the laws of physics is selected entirely by the sequence), the number of possible results (the number of different shapes that could result) is limited to the number of different sequences. That is, the information in the phenome seems to be limited by the information in the genome. If you think of it as a two part message, with the first part encoding a model, and the second part configuring it, then the DNA can be seen as the configuration, and the laws of physics as the model (which isn't actually coded anywhere like DNA - we'd have to write that ourselves.) This model is constant over all life, so that DNA from all species (plants and animals) share the same "model" (laws of physics that cause protein folding etc.) Another example of a two-part message is that the first part is a programming language, and the second part is a program written in that language. For a high level language (esp with libraries), it's obvious that a very short program might do an awful lot; but the true information content is not that program alone, but the total including the language and libraries it uses. However, and this is my point, I don't believe that the laws of physics have been constructed so conveniently that provide as much assistance as a high level language with libraries. At most, nature may have stumbled onto hacks in physics (like surface tension, interfaces and gradients) and exploited them. Actually, given how long it took to get life started, perhaps it had to find a whole bunch of clever hacks (randomly recombining for billions of years over a whole planet) before it came up with a workable model (that is the model that DNA configure.) hmmm... we might be able to estimate the information content of the 'model' by how many tries it took to come across it.

jacquesm 16y ago
> That is, the information in the phenome seems to be limited by the information in the genome. I think that is a very original use of the word 'limited', limited in this case holds enough room for random chance to come up with human beings. For all practical purposes that 'limited' might as well be unlimited.
- 10ren 16y ago