5 ms·
upper bounds on the program size on machine tell you nothing about the the upper bounds on the other machine.
Not true! Worst case scenario, one machine can e
upper bounds on the program size on machine tell you nothing about the the upper bounds on the other machine.
Not true! Worst case scenario, one machine can emulate the other machine, running that input. Therefore an upper bound on one machine can be used to derive an upper bound on the other. These upper bounds are not perfect, but they are indicative.
Indicative is good enough for the argument that Kurzweil is attempting. His argument has other flaws, but this is not one of them.
That is why, for example, when Chaitin lectures about Omega he is always mumbling "relative to some choice of turing machine" (at least initially, until it is then understood to apply throughout the talk).
Not so fast. Omega is the probability of a randomly chosen machine halting. Suppose that an upper bound on machine A predicted that for machine B within just 50 bytes of data. Then all of the programs of length n on machine A get translated on machine B to a list of programs that is 2400 times bigger, and the rest have unknown behavior. In this case Omega for machine A says very, very little about Omega for machine B.
Therefore "indicative" is very much not good enough for what Gregory Chaitin is talking about.
>Not true! Worst case scenario, one machine can emulate the other machine, running that input.
"If you wish to create a brain in 50 MB of code, you must first implement the universe"?
You already have a universe to run your machine in.
It is, of course, conceivable that the brain contains some kind of computational processes that can be implemented/emulated much more quickly on a biochemical substrate than in doped semiconductors — some kind of quantum computation or something. However, all the evidence we have at the moment points in the opposite direction, and there's no particular reason to believe that the universe is designed to give biochemical systems such an advantage.
That was a flippant Sagan reference.
http://www.youtube.com/watch?v=7s664NsLeFM http://www.youtube.com/watch?v=7s664NsLeFM
>You already have a universe to run your machine in.
The CPU may be physical, but software on it runs on the abstraction which is the CPU's instruction set. The underlying physical nature of the processor is irrelevant.
He was talking about having one machine emulate another - I was pointing out that that means your i7 emulating all of physics.
Worst case scenario, one machine can emulate the other machine, running that input. Therefore an upper bound on one machine can be used to derive an upper bound on the other.
Very nice try. (I mean that sincerely.) However, you must also have a proof about the complexity of the emulation and the encoding of programs to emulate. Without that proof - which basically means you are no longer talking about two arbitrary machines but rather about two which are deeply related - the mere possibility of emulation tells you nothing other than that one machine is of equal or greater computational power than the other.
[Edit: in context - sure, our fancy big super computer from IBM can emulate the zygote (let's say) but.. what's that gonna take? Trillions of lines of code? 10 lines?]
And, maybe I need another cup of coffee but I am actually having trouble parsing the syntax of your comment about Omega.
However, you must also have a proof about the complexity of the emulation and the encoding of programs to emulate. Without that proof - which basically means you are no longer talking about two arbitrary machines but rather about two which are deeply related - the mere possibility of emulation tells you nothing other than that one machine is of equal or greater computational power than the other.
If we're talking about Turing machines, all of the machines are of equal computational power, and are also able to emulate each other.
If we're talking about real machines, the take away is that moving from one machine to the other doesn't require much code, but does require that the other machine be powerful enough. Thus leading to Ray Kurzweil's argument that the real challenge is getting a powerful enough machine, and not in the amount of code needed to run on that machine. (He further argues that Moore's law will provide us with that machine.)
And, maybe I need another cup of coffee but I am actually having trouble parsing the syntax of your comment about Omega.
Basically I was saying that if you offer some facility to translate programs from machine A to machine B, only a very small portion of the programs on machine B will turn out to be programs that were translated from machine A. Therefore the probability of programs halting on machine A tells us nothing about what the majority of programs on machine B are likely to do, and therefore tells us very little about the probability of a random program on machine B halting.
Does that make more sense?
If we're talking about Turing machines, all of the machines are of equal computational power, and are also able to emulate each other.
We aren't necessarily talking about Turing machines but let's suppose that we are. We're talking about machines powerful enough to compute anything that any Turing machine can compute. You are correct that any one of these machines can emulate any of the others.
What's relevant to Kurzweil's argument is two-fold:
If I tell you about Turing-class machine A and give you Turing-class machine B, we know right away that you can write an A emulator that runs on B. We know that because they are both Turing class. What we don't know is how big that emulator program is. It might take a trillion lines of code just to write the emulator.
Next, I have told you about machine A but now I also give you a 50 Megabyte program (called "GENOME") that runs on machine A. Now you must write your emulator, encode GENOME for your emulator, and run it on B. We know you can do that since these are both Turing class machines. What we do not know is how many bits it will take to encode GENOME. It could be 50 Megabytes. Or it could be 50 Terabytes. You just can't tell without proving things about the specific details of machines A and B and your emulator.
Kurzweil makes an argument analogous to "Hey, GENOME only takes 50MB on machine A. There's no way it takes 50TB on machine B. Information theory at work, man! Use your head!" Problem is, "information theory" says no such thing. Actually, it says "If that's all we've been told, hell it could easily take 50TB on machine B. It could take 5 bytes. Who knows? There's not enough said about machines A and B or the program GENOME to tell!"
Basically I was saying that if you offer some facility to translate programs from machine A to machine B, only a very small portion of the programs on machine B will turn out to be programs that were translated from machine A. Therefore the probability of programs halting on machine A tells us nothing about what the majority of programs on machine B are likely to do, and therefore tells us very little about the probability of a random program on machine B halting.
Yes. You are correctly observing that Omega is not really a constant - it is a class of constants. For machine A there is Omega_A. For machine B, Omega_B. That is why Chaitin lectures about Omega but only after several reminders that it's really "Omega_X for some specific machine X -- but I'm mostly just going to say Omega and let X be understood."
So, now, think that through with Kurzweil goggles on. In Kurzweil's world, a program of 50MB on one machine can't be 50TB on the next machine -- that's just silly, right? Well, if he were correct then, in general, Omega_A and Omega_B, for any machines A and B, would have to be somewhat close in value. We know that Omega_A and Omega_B don't have to be close (other than that they both come from the same (0,1) interval). By contradiction we conclude that we don't live in the world Kurzweil is describing.
On standard machine architectures
Yes, right there you are demonstrating a deep misunderstanding of what you are replying to or how it relates to Kurzweil's arguments.
On Omega, you say: all that is necessary is that the programs from A have to be lost in a much larger sea of valid programs in B
You left out that we're comparing sets of A and B programs of similar length -- the "sea" is not so large. Actually you kind of acknowledge this but then say bogus things about what it implies.
You are confused or else just b.s.'ing on purpose. It's been fun. We're done, please.
I'm sick and tired of the downvote barrage lately. That aside, let me try a completely different tack, maybe that sheds some light on the problem.
The complexity of a Turing machine is a function of the number of states that it can encode, in programming languages we call that 'expressiveness'. You can write a smaller program in a language that is more expressive. A Turing machine that allows more states is going to require a shorter tape (fewer cells) to do its work but will be more complex.
The argument I think that Kurzweil makes with his 50 MB figure is that once you take out all the redundancy you've taken out the expressiveness factor. Whatever emulator you use to interpret that 50 MB you'll end up with some 3D structure (if you simulate the embryology properly) that will resemble a brain.
The part where the analogy goes wrong according to me is by postulating a machine that this stuff should run on. The 'emulator' is nothing but the endless repetition of ribosomes threading RNA in order to produce more polypeptide chains which then fold up in order to become proteins.
Your 'emulator' in this case is a physics program, and not even a very complicated one, albeit with an absolutely astronomical data set for any size organism.
In order to avoid having to deal with that data set we try to achieve a higher level model of what the brain does, instead of trying to model what it is.
But that does not change the fact that the principle of the emulator (a 'mere' matter of programming) and the code that governs the construction are what gives rise to the brain and through there to intelligence.
So the upper bound of the problem in terms of complexity is the 50 M + whatever you need in terms of physics to (theoretically) run the simulation of the single fertilized ovum from start to finish, or as much as you need to verify that you have indeed succeeded in your simulation goal.
It can't get more complex than that, even if the generated structure is orders of magnitude more complex at first glance than the input data + the program. Think of it as 'output'. Thought is not the output of that program. Thought is the state changes within the produced (or simulated) brain.
The organic soup and all the proteins arising from it are the nuts and bolts the machinery is made of. That doesn't mean that we can't make similar machinery without those particular nuts and bolts.
I hope that all makes sense.
Right. And this gets at what I see as Kurzweil's flaw. The emulator you need for his argument to work is the physics engine, a specification of the cell, and the necessary imitation of the surrounding emvironment. This is far, far more information than GENOME. And we know that some of this complexity matters.
For instance http://www.nature.com/nature/journal/v441/n7092/full/nature04674.html http://www.nature.com/nature/journal/v441/n7092/full/nature0... has an example of RNA-based inheritance that is not in the genome. Also http://en.wikipedia.org/wiki/Genomic_imprinting http://en.wikipedia.org/wiki/Genomic_imprinting shows that what genes you have and what genes you express are two different stories entirely.
That said, the effort of transcribing the 50 MB of genomic information into the final program is simply not a problem. And if we wanted to transcribe a much larger genome, the additional effort for that is again bounded.
This is all, of course, ignoring the excellent point that one would never want to actually do this given how slow this emulator would have to be. If we create AI in the next few decades, I can guarantee that it won't be a physics emulation of a human brain.
You are apparently talking about "Turing machines" by which you mean finite automata connected to a 2-d tape or things very close to that. That class of machines is a tiny subset of turing complete machines and that class of machines you refer to is irrelevant to the arguments to which you are responding.
When you argue that a simple (just a few rules of physics) emulator could, in principle, do the job -- quite right. Unfortunately for your point, Kurzweil is making a much, much stronger claim. For, surely, you agree that that physics emulation isn't going to result in a practical program in 20 years, yes?
The way you are using "upper bound of the problem in terms of complexity" there... I could use your same logic to argue that the design of a brain can be encoded in exactly 1 bit.
The design of a brain, by your logic, contains 1 bit of information. (In particular, that 50 MB of compressed genome data? I'll give you a better compression algorithm for this purpose and it will, indeed, compress a particular human genome down to 1 bit.)
> You are apparently talking about "Turing machines" by which you mean finite automata connected to a 2-d tape or things very close to that. That class of machines is a tiny subset of turing complete machines and that class of machines you refer to is irrelevant to the arguments to which you are responding.
Since all Turing complete machines are functionally equivalent I don't think it is quite that irrelevant.
if the brain can be simulated by any Turing machine at all it figures that a 2-d tape Turing machine can do the same thing.
The question then becomes how big is the total information content of both Turing machine + tape.
The rules of physics are Turing complete, we use them to build computers with and nature uses them to build the physical universe, including computers and brains.
Kurzweil is not making such a strong claim at all I think, he merely argues that there must be some upper boundary to the starting configuration from which you should be able to arrive at a working brain.
And much as I would like to I can't find fault with that other than that the embryotic development is chaotic in nature and that a very small change in input conditions can easily produce nothing of interest on the output side.
The conclusion that we can draw from that is that the brain (and in fact probably every cell in our bodies) must contain a great deal of machinery to make sure that the process stays on track and does not veer off in to chaos and unpredictable outcomes. For the most part, it works, most of the fetuses carried to term are able to think.
> For, surely, you agree that that physics emulation isn't going to result in a practical program in 20 years, yes?
Some people claim even shorter times than that (which I think is highly irresponsible):
http://www.dailymail.co.uk/health/article-1201724/Artificial-brain-reality-just-10-years-scientists-claim.html http://www.dailymail.co.uk/health/article-1201724/Artificial...
Personally I would put the time required for such a project in the multiple decades, probably 50 years at a minimum, maybe much more assuming that it can be done and that there is no quantum component (as proposed by Penrose, rather weakly) to intelligence.
As for your 1 bit description, you are not taking in to account the information content of the guts of the compression algorithm, and you'd find that algo+compressed data would work out to a nearly constant amount of information.
That 50MB has been created in a way that the size of the algorithm required to compress it and decompress it is relatively small compared to the size of the data.
As soon as you start embedding codebooks inside your algorithm the size of the data will go down but the size of the algorithm goes up correspondingly. Also the algorithm loses its universal character, it's ability to encode multiple data sets with roughly equal characteristics in terms of information content in to roughly equally sized output.