9 ms·

Ray Kurzweil is wrong: The Singularity is not near

25 points by ruchir_21hj 13y ago

girvo 13y ago
See, I'm seriously not convinced about the whole singularity thing. This article sort of touches on why I can't wrap my head around it, but it basically comes down to this: if I "upload" my "consciousness" -- how is it, well, me? Now, I'm fully aware that's a massive can of philosophical worms, but I simply can't reconcile it, personally. I see the future hyperintelligent AI as something more like Iain M Banks "Culture" Minds, but hey, maybe I just can't comprehend it with my puny brain :)
- AndrewDucker 13y ago
  How about if we extend your brain by hooking it up to a memory augmentation device. Still you? And then extend it again by hooking it up to a co-processor? Still you? And then keep extending it, until the meat bits are only 5% of the structure, and not necessary for the running of the whole? Still you? Is You-ness (or identity) fluid and variable, or is it binary - something is either "you" or "not you"?
  - davidw 13y ago
    > memory augmentation device. Still you? Yeah. > co-processor? What's that mean, even?
    - SiVal 13y ago
      A simple calculator, for example. It could feel as if you had all answers to numerical calculations somehow memorized. You might wonder what the twelfth root of two was, and you would just suddenly "remember" the answer.
    - masklinn 13y ago
      > What's that mean, even? A directly accessible computer hooked to your brain, to augment your processing capabilities rather than just memory.
  - vincentperes 13y ago
    Nobody can answer those questions, because we don't understand much of the brain yet. Beyond philosophy, you need to understand what is really YOU.
    - TeMPOraL 13y ago
      > what is really YOU I think it's a wrong question. "You" is in the map, not in the territory. It's how algorithms in our brain factorize the world model.
  - Ygg2 13y ago
    We know so little about our brain I don't think that is really, possible, not without some major breakthroughs in how brain functions. I think Penrose was on right track with his ideas about brain using quantum effects (our words and comprehension follows rules that are eerily familiar of some quantum effects). With that in mind, I don't think it's possible to do an easy copy of it without disrupting the way it works.
  - parasight 13y ago
    > How about if we extend your brain by hooking it up to a memory augmentation device. Still you? Probably "still me" in the same sense as the person I was five minutes ago is the same person as the one I'll be in 5 minutes.
    - CmonDev 13y ago
      Only suddenly you start memorizing much more.
  - RyanZAG 13y ago
    This question is even deeper than just a you or not you. It's been a tough question for a very, very long time. [1] "The ship of Theseus, also known as Theseus's paradox, is a thought experiment that raises the question of whether an object which has had all its components replaced remains fundamentally the same object. The paradox is most notably recorded by Plutarch in Life of Theseus from the late 1st century. Plutarch asked whether a ship which was restored by replacing each and every one of its wooden parts, remained the same ship." [1] http://en.wikipedia.org/wiki/Ship_of_Theseus http://en.wikipedia.org/wiki/Ship_of_Theseus
    - AndrewDucker 13y ago
      Oh yes. See also The Tin Man in The Wizard of Oz, who replaced each part of himself as he chopped them off. There's an early AI researcher who brought up the neuron-by-neuron replacement too, but I can't remember who off-hand (might have been Moravec)
      - andyjohnson0 13y ago
        Sometimes called the Moravec Procedure: http://www.ibiblio.org/jstrout/uploading/moravec.html http://www.ibiblio.org/jstrout/uploading/moravec.html
        kybernetikos 13y ago
        I'd be reasonably happy with the Moravec Procedure where I would be much less happy with most other ways of uploading (the whole thing is based on my irrational but hard to shake belief in my own persistence over time). Of course, the question then of who to trust to be your cloud provider running your conciousness-as-a-service process becomes a very serious one. I don't think there's a single company around today that I would trust to run me.
        andyjohnson0 13y ago
        Of course, the question then of who to trust to be your cloud provider running your conciousness-as-a-service process becomes a very serious one. The main problem is the lawyers. https://www.youtube.com/watch?v=IFe9wiDfb0E https://www.youtube.com/watch?v=IFe9wiDfb0E
  - richardjdare 13y ago
    Perhaps the person's experience diminishes slowly as each part is replaced, until he is finally snuffed out without anyone noticing :) reminds me of "how to get ahead in advertising", or William Burroughs' "Did I ever tell you about the man who taught his asshole to talk" heh.. It's a fascinating question.
  - andyjohnson0 13y ago
    And then you clone the uploaded conciousness 10^6 times, with each clone having continuity of experience back to your earliest memory. Still you? (And before you can answer, the clones all respond "yes").
  - Houshalter 13y ago
    It's a bad argument because it can work the other way around. If I kill a single neuron from your brain are you still you? If I keep killing them 1 by 1, at no point is the transition very big, but at some point you are certainly not "you" anymore, maybe not even conscious.
- kowdermeister 13y ago
  Try imagine it like you fall asleep and when you wake up, your physical body is gone and what you operate on is a virtual body, just like a virtual machine for OS-es.
  - onion2k 13y ago
    That is only really possible if the human mind is a giant chemical-driven deterministic state-machine that could be replicated using current technology on a massive scale. Most neuroscientists believe that there's much more to the human brain than that - there are things going on in our brains like interactions between close synapses using quantum effects. Perhaps we could model that if we understood it better, but the reality is we don't, so we can't really predict whether or not it's something we can copy.
    - kowdermeister 13y ago
      You described the challenge to solve, I think quantum effects does take place in the brain. However, there are numerous experiments on quantum teleportation. In the next 30-50 years technology could advance significantly.
  - rainforest 13y ago
    Externally yes, but what if you can't hot swap consciousness? What if "you" remain in your body and all that happens is a clone of you is produced? Externally, both you and the clone behave identically and assert that you're the same person.
    - girvo 13y ago
      Exactly, bang on!
    - kowdermeister 13y ago
      I thought about the same question before and the solution could be that uploading would work as quantum entanglement, a shared consciousness between your brain and the machine hardware. Once you are half inside, you can command to shut your organic body down.
- sergiosgc 13y ago
  The article is right in the observation that Kurzweil's singularity paper conflates a set of notions. Some are easily testable, some are more difficult to digest. The notion that evolution is exponential is, to me at least, very well supported. This article criticizes the fact that Kurzweil crosses the barrier between biological evolution and technological evolution without mentioning it, and that the utility function used to plot the exponential across evolution barriers is ill-defined. Both observations are true, but I don't think they shake the observation of exponential behaviour in these evolution functions. The conclusion that this will all end in our minds being uploaded to the ether is more difficult to swallow, I concede. It is a philosophical can of worms, and it is a question that will be left unanswered, even after we can replicate human consciousness in a computer. However, if you assume there is nothing magical about our brains, that our consciousness is merely the result of (biological) computation, then @AndrewDucker's approach [1] is an easy way to wrap your head around the concept. Instead of thinking about a clicking moment when you get transferred into silicon, think of a continual transferral process, where you get silicon additions, while your organic brain continuously dies. Imagine this process takes ten years. When do "you" stop being "you"? I'd wager: never. BTW, the same problems you have with brain uploading, I have with star-trek teleporters. If the same machine that can teleport me can copy me, then it is killing me and creating an identical twin. I'd rather not die in transportation, thank you. [1] https://news.ycombinator.com/item?id=7176674 https://news.ycombinator.com/item?id=7176674
  - RivieraKid 13y ago
    > even after we can replicate human consciousness in a computer What exactly do you mean by consciousness? (I personally don't believe that a turing machine can be consciouss.)
    - XorNot 13y ago
      The real question is why you think the universe, or the planet at least, isn't essentially a Turing machine.
      - RivieraKid 13y ago
- rsaarelm 13y ago
  It might not matter that much. If we ever get something like uploading working, the people who don't care that much whether their upload is really them or not are going to have quite a bit of selection pressure going for them.
- RivieraKid 13y ago
  Singularity is an idea that once you have an AI slightly smarter than humans, it will be able to create a better version of itself, the improved version would do the same, etc. leading to an intelligence "explosion". Consiousness is a different problem. I personally came to the conclusion that a deterministic machine cannot be consciouss.
  - sergiosgc 13y ago
    > I personally came to the conclusion that a deterministic machine cannot be consciouss. Is the difference between being conscious or not defined by reading from /dev/random? Most theories of consciousness that point to quantum theory read like that. Unless you assume quantum randomness is god's invisible hand, and then religion infiltrates the argument. (not that God isn't in /dev/random anyhow)
    - RivieraKid 13y ago
      The way I view consciousness is that it's outside of the physical world, but it can directly interact with something physical – the brain. The interaction can go both ways. Physical world can obviously affect consciousness, if I put alcohol into my brain, my consciousness changes. But consciousness can also "push" physical particles in the brain. One possibility is that this happens through quantum phenomena that are supposedly random according to current theory. Or perhaps the brain just pushes atoms. Either way, this should be testable in the future. If you accept this view on consciousness, you can conclude that a deterministic system cannot be consciouss. Of course, you'll probably argue that my view on consciousness is ridiculously unfounded. I disagree, I think there are very strong arguments for it :) A very brief version: (1) There are good reasons to believe that consciousness is more than a complex interaction of physical particles. But physical particles can affect consicousness, so the mind is at least a passive observer. (2) It's actually an active observer, otherwise I wouldn't be writing this. (3) If it's an active observer, the mind must have a way to affect brain's matter.
- wladimir 13y ago
  I've always thought of mind uploading as a comforting lie. Like what you tell a small child about what happened to the old dog. "He's in a better place now". The idea of the singularity implies that the complexity of technology moves above anything humanity can understand. It is unclear whether there still be any use for our kind of individual conscious mind. At what scale will cognition happen? Will the world itself turn into a kind of brain? But I'm with you in hoping that the Minds will be like the culture's and keep us around as pets in a sort of enlightened hedonism :-)
  - XorNot 13y ago
    Does a cell in your body lack an independent existence? You can culture them by themselves, they evolved by themselves. Algae actually does exist as both single cells and collective colonies of the same organism. There's nothing inherently contradictory of a higher intelligence being composed of many active smaller conscious experiences - arguably this is what the internet already is, just a very slow version of it. We need a better interconnect (and some more durable subprocessors).
    - wladimir 13y ago
      Cells certainly have a kind of independent existence (consciousness is a different matter...). But our body needs its cells. It's not simulating cells in another substrate. The 'cells' in the case of the hypothetical technological singularity will be computer hardware, not organic cells or human brains. There is the tacit assumption that silicon-based AI tech is somehow superior to our own carbon-based 'tech' and will displace it. Human brains forming into a kind of super-organism would be a different phenomenon. It wouldn't even need AI, or much more advanced technology. It could indeed be argued that this is the case already with the internet and even printing press before it.
      - TeMPOraL 13y ago
        > There is the tacit assumption that silicon-based AI tech is somehow superior to our own carbon-based 'tech' and will displace it. I doubt it's superior or will replace carbon-based tech (no quotes here; biology is but nanotechnology that was not made by us), but there are two particular improvements introducing human-designed technology could bring: - electrical signals in the brain run at 200Hz; minds could in principle work much faster, were they made of something different - human-designed technology is easier to modify, reprogram and adapt (mostly becasue we built it, so we know how to do it); the idea is that a mind built with it could be made to be able to rewrite/rebuild itself, with luck launching into recursive improvement loop, where a mind redesigns itself to be better, which then goes on to improve itself more, etc.
- khafra 13y ago
  > if I "upload" my "consciousness" -- how is it, well, me? If you were going to be sedated in a blue-painted room, and then an exact copy of you would be made in a red-painted room, what color of wall would you expect to see when you woke up?
- noonespecial 13y ago
  If your consciousness got uploaded how would it know that its not you?
- cgrand-net 13y ago
  > if I "upload" my "consciousness" -- how is it, well, me? It won't be you but you'll be the only one to tell the difference. Thinking of the social pressure (from family, work etc.) of "uploading" yourself is more fun.
- Houshalter 13y ago
  Here is a good explanation on the subject: http://lesswrong.com/lw/pm/identity_isnt_in_specific_atoms/ http://lesswrong.com/lw/pm/identity_isnt_in_specific_atoms/ But the singularity is more than just mind uploading. Technology could advance and people just choose not to upload. You could probably live at least a few thousand years with advanced nanotechnology repairing your brain and body.
skrebbel 13y ago
Oh come on guys. It's much more fun to assume that he's right! This is why you know his name at all: it's a fascinating idea. This is why I believe the singularity will happen in my lifetime. I honestly, genuinely, have no idea whether it really will, but believing that it does makes for excellent ideas, great conversations and a whole lot of excitement (and Judgment Days) in general. I want to believe it. This might be difficult for a certain type of rationalist (that many HN'ers are) to appreciate, but if you really want to believe something very much, the step to actually believing it is very small. Even if it's a conscious process. In general though, no need to squeeze those butt cheeks so much, this is not the stock market. Fun predictions are fun!
- davidw 13y ago
  So how is that not basically a religion?
  - elorant 13y ago
    Because a) that belief is based on scientific facts although some of them are highly exaggerated and b) basically you’re not bashing/attacking/humiliating anyone who tells you otherwise. Personally I don’t believe that Singularity will be achieved in the time frame Kurzwell predicts. Especially regarding AI progress I think that we need at least half a dozen paradigm shifts in order to get to the milestone of a truly self sustained AI entity. But you know in some sense believing it gives you an extra motive on pursuing some of these goals and I think that’s the real benefit of this idea. A few visionaries out there like Kurzwell could push science to new frontiers.
    - acqq 13y ago
      Religion by definition is an acceptance of the claims based on the faith instead of critical thinking and the observation of reality. If the claims use scientific-looking statements which can be proved to not match with the reality or logic they are the "dogma." Of course it's not an "organized religion" but the beliefs are religiously accepted by those that promote it.
acro 13y ago
Still, the performance of computers is increasing exponentially. The arguments in this article seemed a bit weak. http://top500.org/statistics/perfdevel/ http://top500.org/statistics/perfdevel/
- Ygg2 13y ago
  And we all know that calculation speed is the main quality of IQ.
  - acro 13y ago
    What happens when performance approaches infinity? Many AI problems have been solved because there is now simply more processing power available. For example computer Go using Monte Carlo methods. Just a couple of years ago people thought it was impossible for computers to beat human professionals.
    - sanxiyn 13y ago
      As far as I know no computer Go program has ever won against human Go professionals on the even game of 19x19 Go. Do you know otherwise? My information is from http://computer-go.info/h-c/ http://computer-go.info/h-c/
      - acro 13y ago
        I stand corrected, no official even game wins against professionals. Maybe I remember seeing unofficial games on KGS. The progress is impressive anyway, just a while ago the best computer programs were around 8-10 kyu level.
- rainforest 13y ago
  Isn't that part of the author's point? That Kurzweil is picking and choosing exponential developments because they fit his narrative? Since none of us know what the requirements of the Singularity are, it seems odd to extrapolate from a single abstract indicator and assume that it will happen "somewhere" on that line.
pyalot2 13y ago
If Kurzweil is right or wrong and by how much is completely irrelevant. What is relevant, and this is the only relevant thing about the entire discussion is: 1) When will we reach the ability to take a snapshot of an entire organism with a nervous system up to a resolution sufficient to satisfy all expected simulation parameters? 2) When will sufficient computing and storage capacity be available to run a simulation off the data captured in step #1? 3) Will the simulation performed in step #2 adequately match the organism as it exists in real life to declare that the simulation a success. 4) When will we be able to perform steps #1, #2 and #3 for a human sized organism? By the nature of such an experiment, a negative result does not indicate the absence of a positive in the future. And certainly, and inference on a negative outcome at any step cannot be valid without performing the steps, that's just guesswork. It's also worth noting that inherently estimates on timelines of this are nearly impossible because one cannot exclude the possibility of radical advances in scanning, computing and storage hardware. Or, if you prefer to have a TL;DR explanation for geeks. Unless you can write and run a program, it's nearly impossible to debug it or make statements about how useful it'll be.
- panacea 13y ago
  Kurz-goo-weil-le is a grade A bullshit artist.
- circlefavshape 13y ago
  > 1) When will we reach the ability to take a snapshot of an entire organism with a nervous system up to a resolution sufficient to satisfy all expected simulation parameters? Perhaps never. The major thing Kurzweil is ignoring is that this may actually be impossible.
  - pyalot2 13y ago
    It may be impossible. However, medical scanning technology today already does things considered impossible 10 years ago. While in no way sufficient today by any means, you can't exclude (at this time) the possibility that adequate technology might exist in the not so distant future.
tlarkworthy 13y ago
Incorporation is giving a company the legal status of a biological entity. Within 25 years I think we will have the first company staffed entirely by AI machine (trading bitcoins?). If machines ever get the "inventive spark", its seems clear to me there will be rapid expansion of machine influence on the universe. Computation is just silicon (rocks) after all. So that's the big IF. Can we invent inventive machines? Machine learning is certainly going at rapid pace, NELL etc.
Shinkei 13y ago
We've got quite a few years to go... but I firmly believe that we will actually beat his prediction. Artificial intelligence surpassing that of humans is an inevitability. If we accept that the 'mind' is the sum of the parts of the 'brain,' then we can safely say that simulation of a brain should produce a mind. Besides this line of attack, we could independently create a strong AI based on other methods. Just think of the myriad AI we have experienced in the past few years that people 20 or 30 years ago would've laughed at as ridiculous--Watson, Google car, and Deep Blue.
andyjohnson0 13y ago
Its worth pointing out that this is a commentary on Kurzweil's particular view of technological singularity and how it might happen. Picking holes in his proposed theory of mind for AIs, or his relating biological evolution with technological "progress", doesn't necessarily invalidate the general idea of exponentially accelerating technology. Kurzweil is just one advocate. In fact, the idea [1] originated with Von Neumann in the 50s and was popularised by Vernor Vinge in the 90s, and its worth reading Vinge's paper "The Coming Technological Singularity: How to Survive in the Post-Human Era" [2]. Bruce Sterling gave a interesting (and sceptical) talk [3] on the subject a while back. For what its worth, I'm personally not convinced that we'll ever see a technological singularity because I don't think our culture generates technology that has the required attributes. Siri or Google Now don't need to be self-aware in order to do what they do, and there is no apparent commercial pressure to make them so. And since people build technology for human purposes, the presence of people who sleep, get bored, etc. will act as a drag on any tendency for runaway acceleration. [1] http://en.wikipedia.org/wiki/Technological_singularity#History_of_the_idea http://en.wikipedia.org/wiki/Technological_singularity#Histo... [2] http://www-rohan.sdsu.edu/faculty/vinge/misc/singularity.html http://www-rohan.sdsu.edu/faculty/vinge/misc/singularity.htm... [3] http://longnow.org/seminars/02004/jun/11/the-singularity-your-future-as-a-black-hole/ http://longnow.org/seminars/02004/jun/11/the-singularity-you...
- hershel 13y ago
  I'm not sure why awareness is critical to a singularity.Requiring machine intelligence to have a similar consciousness to humans is like asking submarines to swim like a fish, and anyways computer already have self awareness(reflection in programming languages, and some robots who use reflection to learn how to optimize movement) - it's useful but doesn't seem critical AFAIK.
  - andyjohnson0 13y ago
    Good point. The development of some kind of strong AI is often proposed as a catalyst for a technological singularity, and I was kind of following that lead. While I agree that self-awareness may not be needed, it is the only thing I can think of that would motivate an AI to enhance itself up the exponential curve. Relying oh humans to do this probably wouldn't work (past some initial pump-priming) as they tend to get distracted by other human-scale things. On the other hand, speculating about motivation in relation to an AI may be misleading anyway...
pedalpete 13y ago
I find so much of the arguments the article lists as 'far-fetched' are actually not far from where we are today. If you really think about it how far are we from there being ' no distinction... between human and machine or between physical and virtual'. I specifically think of the 'physical and virtual' stuff. I'm currently having a virtual 'conversation' regarding this topic. None of you who may respond are with me, we're doing this all virtually, and I doubt the time is far off where somebody can have a machine respond coherently with their general thoughts on a subject. When the author talks about Kurzweil not differentiating between biological evolution and mechanical evolution, I think that our biological evolution has in many ways been a result of our technological evolution, would one exist without the other? Isn't out technological evolution in the areas of food production resulting in our genetic evolution as larger (both taller and fatter) people? I know I'm picking some pretty minor things here and that means the singularity is still a long way off, but compare the last 40 years to the next 40 years. Not everything Kurzweil describes to be part of the Singularity has to happen exactly as he describes it. It's a theory, and (from getting only about 1/3 through the book) an interesting one. I guess haters gonna hate.
Udo 13y ago
Ray Kurzweil is the wrong person to publicly personify transhumanism (I'm not opportunistically jumping on the bandwagon, I've been saying this for a long time now). I realized this when I read about his idea to resurrect his dead father by feeding some left-over data into an AI tasked with imitating his father. While I do think it's unethical to slave a sentient AI algorithm to this charade to begin with, the larger problem is that he apparently believes this will bring a dead person back to life. That's a very shallow idea for crossing over to a new substrate, clearly a lot more is required to port over a person. While it's good to have a quasi-official spokesperson who can convey some of the concepts to a broader audience, things also got simplified in a way that seriously distorts them. It's not about arbitrary singular points during our development, it's about the breathtaking directions these developments are destined to go in. That's a much harder concept to sell than, say, a magical tipping point where history ends, but I'd say it's well worthwhile. The standard singularity talk doesn't reflect what's really happening to us - nevertheless, it's important to open people up to the prospect of continued technological development. It sounds so simple, but this is really not an obvious path to people who believe the future will mainly contain ever-lighter iPads and will otherwise be the same: there will be a couple of points in history where technology reaches the potential to fundamentally change who we are, what it means to be alive, and what our goals are as a civilization. Technology will make it possible to alter some very fundamental realities humans had to live with for a long time. The impact of this is enormous beyond words. We get to change things in a very big way, including some options that would be hugely unpopular even for the tech-savvy people on HN - like getting rid of material scarcity, for example, or choosing radical life extension. It's clear that this will be upsetting to a lot of people, and suddenly we need a marketing person like Kurzweil. Personally, I'd prefer Eliezer Yudkowsky would take that role, but that's probably not a good choice for "lay people".
- Houshalter 13y ago
  I think it's creepy and ridiculous, but you could rebuild most of a person that way. At least create a new person that would have the same personality, maybe memories, and be completely indistinguishable from the original (that is, you wouldn't be able to recognize any difference at all or do any test, especially if it's reconstructed from your own memories.) As for whether it's moral, well it depends if the copy is actually sentient or just a really advanced chatbot. And if it is sentient, well we create sentient beings all the time; children. But I do agree it isn't a solution to death.
finishingmove 13y ago
Is the Singularity compatible with feudalism?
joesmo 13y ago
Singularity cannot exist until at least one machine is intelligent in some manner, something that is further away than most people think.

25 points by ruchir_21hj 13y ago

girvo 13y ago

See, I'm seriously not convinced about the whole singularity thing. This article sort of touches on why I can't wrap my head around it, but it basically comes down to this: if I "upload" my "consciousness" -- how is it, well, me? Now, I'm fully aware that's a massive can of philosophical worms, but I simply can't reconcile it, personally. I see the future hyperintelligent AI as something more like Iain M Banks "Culture" Minds, but hey, maybe I just can't comprehend it with my puny brain :)

AndrewDucker 13y ago
How about if we extend your brain by hooking it up to a memory augmentation device. Still you? And then extend it again by hooking it up to a co-processor? Still you? And then keep extending it, until the meat bits are only 5% of the structure, and not necessary for the running of the whole? Still you? Is You-ness (or identity) fluid and variable, or is it binary - something is either "you" or "not you"?
- davidw 13y ago
  > memory augmentation device. Still you? Yeah. > co-processor? What's that mean, even?
  - SiVal 13y ago
    A simple calculator, for example. It could feel as if you had all answers to numerical calculations somehow memorized. You might wonder what the twelfth root of two was, and you would just suddenly "remember" the answer.
  - masklinn 13y ago
    > What's that mean, even? A directly accessible computer hooked to your brain, to augment your processing capabilities rather than just memory.
- vincentperes 13y ago
  Nobody can answer those questions, because we don't understand much of the brain yet. Beyond philosophy, you need to understand what is really YOU.
  - TeMPOraL 13y ago
    > what is really YOU I think it's a wrong question. "You" is in the map, not in the territory. It's how algorithms in our brain factorize the world model.
- Ygg2 13y ago
  We know so little about our brain I don't think that is really, possible, not without some major breakthroughs in how brain functions. I think Penrose was on right track with his ideas about brain using quantum effects (our words and comprehension follows rules that are eerily familiar of some quantum effects). With that in mind, I don't think it's possible to do an easy copy of it without disrupting the way it works.
- parasight 13y ago
  > How about if we extend your brain by hooking it up to a memory augmentation device. Still you? Probably "still me" in the same sense as the person I was five minutes ago is the same person as the one I'll be in 5 minutes.
  - CmonDev 13y ago
    Only suddenly you start memorizing much more.
- RyanZAG 13y ago
  This question is even deeper than just a you or not you. It's been a tough question for a very, very long time. [1] "The ship of Theseus, also known as Theseus's paradox, is a thought experiment that raises the question of whether an object which has had all its components replaced remains fundamentally the same object. The paradox is most notably recorded by Plutarch in Life of Theseus from the late 1st century. Plutarch asked whether a ship which was restored by replacing each and every one of its wooden parts, remained the same ship." [1] http://en.wikipedia.org/wiki/Ship_of_Theseus http://en.wikipedia.org/wiki/Ship_of_Theseus
  - AndrewDucker 13y ago
    Oh yes. See also The Tin Man in The Wizard of Oz, who replaced each part of himself as he chopped them off. There's an early AI researcher who brought up the neuron-by-neuron replacement too, but I can't remember who off-hand (might have been Moravec)
    - andyjohnson0 13y ago
      Sometimes called the Moravec Procedure: http://www.ibiblio.org/jstrout/uploading/moravec.html http://www.ibiblio.org/jstrout/uploading/moravec.html
      - kybernetikos 13y ago
        I'd be reasonably happy with the Moravec Procedure where I would be much less happy with most other ways of uploading (the whole thing is based on my irrational but hard to shake belief in my own persistence over time). Of course, the question then of who to trust to be your cloud provider running your conciousness-as-a-service process becomes a very serious one. I don't think there's a single company around today that I would trust to run me.
        andyjohnson0 13y ago
        Of course, the question then of who to trust to be your cloud provider running your conciousness-as-a-service process becomes a very serious one. The main problem is the lawyers. https://www.youtube.com/watch?v=IFe9wiDfb0E https://www.youtube.com/watch?v=IFe9wiDfb0E
- richardjdare 13y ago
  Perhaps the person's experience diminishes slowly as each part is replaced, until he is finally snuffed out without anyone noticing :) reminds me of "how to get ahead in advertising", or William Burroughs' "Did I ever tell you about the man who taught his asshole to talk" heh.. It's a fascinating question.
- andyjohnson0 13y ago
  And then you clone the uploaded conciousness 10^6 times, with each clone having continuity of experience back to your earliest memory. Still you? (And before you can answer, the clones all respond "yes").
- Houshalter 13y ago
  It's a bad argument because it can work the other way around. If I kill a single neuron from your brain are you still you? If I keep killing them 1 by 1, at no point is the transition very big, but at some point you are certainly not "you" anymore, maybe not even conscious.
kowdermeister 13y ago
Try imagine it like you fall asleep and when you wake up, your physical body is gone and what you operate on is a virtual body, just like a virtual machine for OS-es.
- onion2k 13y ago
  That is only really possible if the human mind is a giant chemical-driven deterministic state-machine that could be replicated using current technology on a massive scale. Most neuroscientists believe that there's much more to the human brain than that - there are things going on in our brains like interactions between close synapses using quantum effects. Perhaps we could model that if we understood it better, but the reality is we don't, so we can't really predict whether or not it's something we can copy.
  - kowdermeister 13y ago
    You described the challenge to solve, I think quantum effects does take place in the brain. However, there are numerous experiments on quantum teleportation. In the next 30-50 years technology could advance significantly.
- rainforest 13y ago
  Externally yes, but what if you can't hot swap consciousness? What if "you" remain in your body and all that happens is a clone of you is produced? Externally, both you and the clone behave identically and assert that you're the same person.
  - girvo 13y ago
    Exactly, bang on!
  - kowdermeister 13y ago
    I thought about the same question before and the solution could be that uploading would work as quantum entanglement, a shared consciousness between your brain and the machine hardware. Once you are half inside, you can command to shut your organic body down.
sergiosgc 13y ago
The article is right in the observation that Kurzweil's singularity paper conflates a set of notions. Some are easily testable, some are more difficult to digest. The notion that evolution is exponential is, to me at least, very well supported. This article criticizes the fact that Kurzweil crosses the barrier between biological evolution and technological evolution without mentioning it, and that the utility function used to plot the exponential across evolution barriers is ill-defined. Both observations are true, but I don't think they shake the observation of exponential behaviour in these evolution functions. The conclusion that this will all end in our minds being uploaded to the ether is more difficult to swallow, I concede. It is a philosophical can of worms, and it is a question that will be left unanswered, even after we can replicate human consciousness in a computer. However, if you assume there is nothing magical about our brains, that our consciousness is merely the result of (biological) computation, then @AndrewDucker's approach [1] is an easy way to wrap your head around the concept. Instead of thinking about a clicking moment when you get transferred into silicon, think of a continual transferral process, where you get silicon additions, while your organic brain continuously dies. Imagine this process takes ten years. When do "you" stop being "you"? I'd wager: never. BTW, the same problems you have with brain uploading, I have with star-trek teleporters. If the same machine that can teleport me can copy me, then it is killing me and creating an identical twin. I'd rather not die in transportation, thank you. [1] https://news.ycombinator.com/item?id=7176674 https://news.ycombinator.com/item?id=7176674
- RivieraKid 13y ago
  > even after we can replicate human consciousness in a computer What exactly do you mean by consciousness? (I personally don't believe that a turing machine can be consciouss.)
  - XorNot 13y ago
    The real question is why you think the universe, or the planet at least, isn't essentially a Turing machine.
    - RivieraKid 13y ago
rsaarelm 13y ago
It might not matter that much. If we ever get something like uploading working, the people who don't care that much whether their upload is really them or not are going to have quite a bit of selection pressure going for them.
RivieraKid 13y ago
Singularity is an idea that once you have an AI slightly smarter than humans, it will be able to create a better version of itself, the improved version would do the same, etc. leading to an intelligence "explosion". Consiousness is a different problem. I personally came to the conclusion that a deterministic machine cannot be consciouss.
- sergiosgc 13y ago
  > I personally came to the conclusion that a deterministic machine cannot be consciouss. Is the difference between being conscious or not defined by reading from /dev/random? Most theories of consciousness that point to quantum theory read like that. Unless you assume quantum randomness is god's invisible hand, and then religion infiltrates the argument. (not that God isn't in /dev/random anyhow)
  - RivieraKid 13y ago
    The way I view consciousness is that it's outside of the physical world, but it can directly interact with something physical – the brain. The interaction can go both ways. Physical world can obviously affect consciousness, if I put alcohol into my brain, my consciousness changes. But consciousness can also "push" physical particles in the brain. One possibility is that this happens through quantum phenomena that are supposedly random according to current theory. Or perhaps the brain just pushes atoms. Either way, this should be testable in the future. If you accept this view on consciousness, you can conclude that a deterministic system cannot be consciouss. Of course, you'll probably argue that my view on consciousness is ridiculously unfounded. I disagree, I think there are very strong arguments for it :) A very brief version: (1) There are good reasons to believe that consciousness is more than a complex interaction of physical particles. But physical particles can affect consicousness, so the mind is at least a passive observer. (2) It's actually an active observer, otherwise I wouldn't be writing this. (3) If it's an active observer, the mind must have a way to affect brain's matter.
wladimir 13y ago
I've always thought of mind uploading as a comforting lie. Like what you tell a small child about what happened to the old dog. "He's in a better place now". The idea of the singularity implies that the complexity of technology moves above anything humanity can understand. It is unclear whether there still be any use for our kind of individual conscious mind. At what scale will cognition happen? Will the world itself turn into a kind of brain? But I'm with you in hoping that the Minds will be like the culture's and keep us around as pets in a sort of enlightened hedonism :-)
- XorNot 13y ago
  Does a cell in your body lack an independent existence? You can culture them by themselves, they evolved by themselves. Algae actually does exist as both single cells and collective colonies of the same organism. There's nothing inherently contradictory of a higher intelligence being composed of many active smaller conscious experiences - arguably this is what the internet already is, just a very slow version of it. We need a better interconnect (and some more durable subprocessors).
  - wladimir 13y ago
    Cells certainly have a kind of independent existence (consciousness is a different matter...). But our body needs its cells. It's not simulating cells in another substrate. The 'cells' in the case of the hypothetical technological singularity will be computer hardware, not organic cells or human brains. There is the tacit assumption that silicon-based AI tech is somehow superior to our own carbon-based 'tech' and will displace it. Human brains forming into a kind of super-organism would be a different phenomenon. It wouldn't even need AI, or much more advanced technology. It could indeed be argued that this is the case already with the internet and even printing press before it.
    - TeMPOraL 13y ago
      > There is the tacit assumption that silicon-based AI tech is somehow superior to our own carbon-based 'tech' and will displace it. I doubt it's superior or will replace carbon-based tech (no quotes here; biology is but nanotechnology that was not made by us), but there are two particular improvements introducing human-designed technology could bring: - electrical signals in the brain run at 200Hz; minds could in principle work much faster, were they made of something different - human-designed technology is easier to modify, reprogram and adapt (mostly becasue we built it, so we know how to do it); the idea is that a mind built with it could be made to be able to rewrite/rebuild itself, with luck launching into recursive improvement loop, where a mind redesigns itself to be better, which then goes on to improve itself more, etc.
khafra 13y ago
> if I "upload" my "consciousness" -- how is it, well, me? If you were going to be sedated in a blue-painted room, and then an exact copy of you would be made in a red-painted room, what color of wall would you expect to see when you woke up?
noonespecial 13y ago
If your consciousness got uploaded how would it know that its not you?
cgrand-net 13y ago
> if I "upload" my "consciousness" -- how is it, well, me? It won't be you but you'll be the only one to tell the difference. Thinking of the social pressure (from family, work etc.) of "uploading" yourself is more fun.
Houshalter 13y ago
Here is a good explanation on the subject: http://lesswrong.com/lw/pm/identity_isnt_in_specific_atoms/ http://lesswrong.com/lw/pm/identity_isnt_in_specific_atoms/ But the singularity is more than just mind uploading. Technology could advance and people just choose not to upload. You could probably live at least a few thousand years with advanced nanotechnology repairing your brain and body.

skrebbel 13y ago

Oh come on guys. It's much more fun to assume that he's right! This is why you know his name at all: it's a fascinating idea. This is why I believe the singularity will happen in my lifetime. I honestly, genuinely, have no idea whether it really will, but believing that it does makes for excellent ideas, great conversations and a whole lot of excitement (and Judgment Days) in general. I want to believe it. This might be difficult for a certain type of rationalist (that many HN'ers are) to appreciate, but if you really want to believe something very much, the step to actually believing it is very small. Even if it's a conscious process. In general though, no need to squeeze those butt cheeks so much, this is not the stock market. Fun predictions are fun!

davidw 13y ago
So how is that not basically a religion?
- elorant 13y ago
  Because a) that belief is based on scientific facts although some of them are highly exaggerated and b) basically you’re not bashing/attacking/humiliating anyone who tells you otherwise. Personally I don’t believe that Singularity will be achieved in the time frame Kurzwell predicts. Especially regarding AI progress I think that we need at least half a dozen paradigm shifts in order to get to the milestone of a truly self sustained AI entity. But you know in some sense believing it gives you an extra motive on pursuing some of these goals and I think that’s the real benefit of this idea. A few visionaries out there like Kurzwell could push science to new frontiers.
  - acqq 13y ago
    Religion by definition is an acceptance of the claims based on the faith instead of critical thinking and the observation of reality. If the claims use scientific-looking statements which can be proved to not match with the reality or logic they are the "dogma." Of course it's not an "organized religion" but the beliefs are religiously accepted by those that promote it.

acro 13y ago

Still, the performance of computers is increasing exponentially. The arguments in this article seemed a bit weak. http://top500.org/statistics/perfdevel/ http://top500.org/statistics/perfdevel/

Ygg2 13y ago
And we all know that calculation speed is the main quality of IQ.
- acro 13y ago
  What happens when performance approaches infinity? Many AI problems have been solved because there is now simply more processing power available. For example computer Go using Monte Carlo methods. Just a couple of years ago people thought it was impossible for computers to beat human professionals.
  - sanxiyn 13y ago
    As far as I know no computer Go program has ever won against human Go professionals on the even game of 19x19 Go. Do you know otherwise? My information is from http://computer-go.info/h-c/ http://computer-go.info/h-c/
    - acro 13y ago
      I stand corrected, no official even game wins against professionals. Maybe I remember seeing unofficial games on KGS. The progress is impressive anyway, just a while ago the best computer programs were around 8-10 kyu level.
rainforest 13y ago
Isn't that part of the author's point? That Kurzweil is picking and choosing exponential developments because they fit his narrative? Since none of us know what the requirements of the Singularity are, it seems odd to extrapolate from a single abstract indicator and assume that it will happen "somewhere" on that line.

pyalot2 13y ago

If Kurzweil is right or wrong and by how much is completely irrelevant. What is relevant, and this is the only relevant thing about the entire discussion is: 1) When will we reach the ability to take a snapshot of an entire organism with a nervous system up to a resolution sufficient to satisfy all expected simulation parameters? 2) When will sufficient computing and storage capacity be available to run a simulation off the data captured in step #1? 3) Will the simulation performed in step #2 adequately match the organism as it exists in real life to declare that the simulation a success. 4) When will we be able to perform steps #1, #2 and #3 for a human sized organism? By the nature of such an experiment, a negative result does not indicate the absence of a positive in the future. And certainly, and inference on a negative outcome at any step cannot be valid without performing the steps, that's just guesswork. It's also worth noting that inherently estimates on timelines of this are nearly impossible because one cannot exclude the possibility of radical advances in scanning, computing and storage hardware. Or, if you prefer to have a TL;DR explanation for geeks. Unless you can write and run a program, it's nearly impossible to debug it or make statements about how useful it'll be.

panacea 13y ago
Kurz-goo-weil-le is a grade A bullshit artist.
circlefavshape 13y ago
> 1) When will we reach the ability to take a snapshot of an entire organism with a nervous system up to a resolution sufficient to satisfy all expected simulation parameters? Perhaps never. The major thing Kurzweil is ignoring is that this may actually be impossible.
- pyalot2 13y ago
  It may be impossible. However, medical scanning technology today already does things considered impossible 10 years ago. While in no way sufficient today by any means, you can't exclude (at this time) the possibility that adequate technology might exist in the not so distant future.

tlarkworthy 13y ago

Incorporation is giving a company the legal status of a biological entity. Within 25 years I think we will have the first company staffed entirely by AI machine (trading bitcoins?). If machines ever get the "inventive spark", its seems clear to me there will be rapid expansion of machine influence on the universe. Computation is just silicon (rocks) after all. So that's the big IF. Can we invent inventive machines? Machine learning is certainly going at rapid pace, NELL etc.

Shinkei 13y ago

We've got quite a few years to go... but I firmly believe that we will actually beat his prediction. Artificial intelligence surpassing that of humans is an inevitability. If we accept that the 'mind' is the sum of the parts of the 'brain,' then we can safely say that simulation of a brain should produce a mind. Besides this line of attack, we could independently create a strong AI based on other methods. Just think of the myriad AI we have experienced in the past few years that people 20 or 30 years ago would've laughed at as ridiculous--Watson, Google car, and Deep Blue.

andyjohnson0 13y ago

Its worth pointing out that this is a commentary on Kurzweil's particular view of technological singularity and how it might happen. Picking holes in his proposed theory of mind for AIs, or his relating biological evolution with technological "progress", doesn't necessarily invalidate the general idea of exponentially accelerating technology. Kurzweil is just one advocate. In fact, the idea [1] originated with Von Neumann in the 50s and was popularised by Vernor Vinge in the 90s, and its worth reading Vinge's paper "The Coming Technological Singularity: How to Survive in the Post-Human Era" [2]. Bruce Sterling gave a interesting (and sceptical) talk [3] on the subject a while back. For what its worth, I'm personally not convinced that we'll ever see a technological singularity because I don't think our culture generates technology that has the required attributes. Siri or Google Now don't need to be self-aware in order to do what they do, and there is no apparent commercial pressure to make them so. And since people build technology for human purposes, the presence of people who sleep, get bored, etc. will act as a drag on any tendency for runaway acceleration. [1] http://en.wikipedia.org/wiki/Technological_singularity#History_of_the_idea http://en.wikipedia.org/wiki/Technological_singularity#Histo... [2] http://www-rohan.sdsu.edu/faculty/vinge/misc/singularity.html http://www-rohan.sdsu.edu/faculty/vinge/misc/singularity.htm... [3] http://longnow.org/seminars/02004/jun/11/the-singularity-your-future-as-a-black-hole/ http://longnow.org/seminars/02004/jun/11/the-singularity-you...

hershel 13y ago
I'm not sure why awareness is critical to a singularity.Requiring machine intelligence to have a similar consciousness to humans is like asking submarines to swim like a fish, and anyways computer already have self awareness(reflection in programming languages, and some robots who use reflection to learn how to optimize movement) - it's useful but doesn't seem critical AFAIK.
- andyjohnson0 13y ago
  Good point. The development of some kind of strong AI is often proposed as a catalyst for a technological singularity, and I was kind of following that lead. While I agree that self-awareness may not be needed, it is the only thing I can think of that would motivate an AI to enhance itself up the exponential curve. Relying oh humans to do this probably wouldn't work (past some initial pump-priming) as they tend to get distracted by other human-scale things. On the other hand, speculating about motivation in relation to an AI may be misleading anyway...

pedalpete 13y ago

I find so much of the arguments the article lists as 'far-fetched' are actually not far from where we are today. If you really think about it how far are we from there being ' no distinction... between human and machine or between physical and virtual'. I specifically think of the 'physical and virtual' stuff. I'm currently having a virtual 'conversation' regarding this topic. None of you who may respond are with me, we're doing this all virtually, and I doubt the time is far off where somebody can have a machine respond coherently with their general thoughts on a subject. When the author talks about Kurzweil not differentiating between biological evolution and mechanical evolution, I think that our biological evolution has in many ways been a result of our technological evolution, would one exist without the other? Isn't out technological evolution in the areas of food production resulting in our genetic evolution as larger (both taller and fatter) people? I know I'm picking some pretty minor things here and that means the singularity is still a long way off, but compare the last 40 years to the next 40 years. Not everything Kurzweil describes to be part of the Singularity has to happen exactly as he describes it. It's a theory, and (from getting only about 1/3 through the book) an interesting one. I guess haters gonna hate.

Udo 13y ago

Ray Kurzweil is the wrong person to publicly personify transhumanism (I'm not opportunistically jumping on the bandwagon, I've been saying this for a long time now). I realized this when I read about his idea to resurrect his dead father by feeding some left-over data into an AI tasked with imitating his father. While I do think it's unethical to slave a sentient AI algorithm to this charade to begin with, the larger problem is that he apparently believes this will bring a dead person back to life. That's a very shallow idea for crossing over to a new substrate, clearly a lot more is required to port over a person. While it's good to have a quasi-official spokesperson who can convey some of the concepts to a broader audience, things also got simplified in a way that seriously distorts them. It's not about arbitrary singular points during our development, it's about the breathtaking directions these developments are destined to go in. That's a much harder concept to sell than, say, a magical tipping point where history ends, but I'd say it's well worthwhile. The standard singularity talk doesn't reflect what's really happening to us - nevertheless, it's important to open people up to the prospect of continued technological development. It sounds so simple, but this is really not an obvious path to people who believe the future will mainly contain ever-lighter iPads and will otherwise be the same: there will be a couple of points in history where technology reaches the potential to fundamentally change who we are, what it means to be alive, and what our goals are as a civilization. Technology will make it possible to alter some very fundamental realities humans had to live with for a long time. The impact of this is enormous beyond words. We get to change things in a very big way, including some options that would be hugely unpopular even for the tech-savvy people on HN - like getting rid of material scarcity, for example, or choosing radical life extension. It's clear that this will be upsetting to a lot of people, and suddenly we need a marketing person like Kurzweil. Personally, I'd prefer Eliezer Yudkowsky would take that role, but that's probably not a good choice for "lay people".

Houshalter 13y ago
I think it's creepy and ridiculous, but you could rebuild most of a person that way. At least create a new person that would have the same personality, maybe memories, and be completely indistinguishable from the original (that is, you wouldn't be able to recognize any difference at all or do any test, especially if it's reconstructed from your own memories.) As for whether it's moral, well it depends if the copy is actually sentient or just a really advanced chatbot. And if it is sentient, well we create sentient beings all the time; children. But I do agree it isn't a solution to death.

finishingmove 13y ago

Is the Singularity compatible with feudalism?

joesmo 13y ago

Singularity cannot exist until at least one machine is intelligent in some manner, something that is further away than most people think.