13 ms·

Kurzweil's rebuttal to Paul Allen

102 points by ca98am79 15y ago

Bishop6 15y ago
Kurzweil makes some good points here. He doesn't address each and every criticism with his view of AI progress, but he does a good job calling out Paul Allen on not doing the homework.
- DanielStraight 15y ago
  Calling out someone for not doing their homework seems somewhere in the DH1-2 range: http://www.paulgraham.com/disagree.html http://www.paulgraham.com/disagree.html There are some good points in Kurzweil's response, but the ones about Paul Allen are definitely not them. I think his best point was about extrapolating function from individual cells or structures, without needing to understand every single cell or structure individually.
  - jerf 15y ago
    I disagree. When one "addresses" a long-standing, well-thought out argument with off-the-cuff snap statement or something only slightly more thought-out, it is perfectly permissible to call them out as not being serious. It's like going into a serious religion debate with "Evil exists, therefore God does not. Ah-ha, you are defeated!", or "God must exist because something must have started it all, so there!" as if in the thousands of years of years the debate has been raging on nobody has ever thought of those things, or addressed them at length, in both directions. If you're going to debate the singularity here, maybe you can get by with just stating "I don't believe it's possible" without citing any logic, which as of this writing there's at least two people in this comment set already who have simply stated that without defense, but if you're going to debate one of the leaders of the field it would help if you would at least grant your opponent the courtesy of thinking that just maybe over the course of the decades he's been thinking about this, the obvious objections that you thought up in five seconds just might have been addressed at some point. You may not think they've been adequately or correctly addressed, but don't pretend they haven't been addressed at all. Personally I'm not completely sold on the matter for a variety of reasons myself, but the usual logic given for why you should be skeptical about it is terrible. The interesting questions are a great deal more complicated than something that can be dismissed with something that generally boils down to "Look, I just can't imagine the world changing that much, so it won't".
    - ambler0 15y ago
      I think your comparison to arguments about religion is apt. Many of the opponents to Kurzweil's ideas remind me of those whose opposition to the possibility of a godless universe amounts to, "I can't imagine it, so I don't believe it."
      - john_b 15y ago
        On the other hand, Kurzweil (at least in his essays and articles) often ignores the question of what a fair null hypothesis is for the possibility of the singularity. I think his gift for creating a compelling vision tends to make people forget that the null hypothesis for a scientific assertion is doubt. Kurzweil provides both high level general evidence (like improvements in computation) and low level, domain-specific evidence (like the discussion about the pancreas) to support his claims, but none of that justifies the use of the word "law" in "law of accelerating returns". He attempts an analogy with thermodynamic laws and how they are derived from underlying statistical principles, but there are no underlying fundamental principles of human innovation and progress that are in any way comparable to the certainty and universality of physical laws. This, I think, is why a lot of people (myself included) have a hard time taking him seriously. He tries to apply the same kind of formal analysis that works well in science to human beings and the complex, highly non-scientific processes that underly innovation today. The bottom line is that, until the singularity occurs, human beings will still be needed to build ever more complex and powerful systems, but human beings do not progress at anything close to an exponential rate.
    - DanielStraight 15y ago
      I guess my point was just that calling him out, while strictly correct, is doing nothing to refute whatever statements he may have made on the singularity, nor is it doing anything to promote the idea of the singularity. Replying to "evil exists, therefore God does not," with, "that argument has been made before," doesn't say anything about whether God does or doesn't exist, and it doesn't do anything to refute the argument. The point of PG's How To Disagree is that the point of disagreeing is to address the truth or falsehood of a proposition, not the way the proposition is presented. The DH# levels (at least DH0-2) can be restated something like this: DH0: I don't like the author, therefore his argument is false. DH1: The author has a relevant fault or weakness, therefore his argument is false. DH2: The author didn't present his argument well, therefore his argument is false. While it's nothing against him personally, the first three paragraphs of Kurzweil's reply read like "the author didn't present his argument well, therefore it is false." He recovers later, sure. He easily reaches 5-6 on the DH scale, but the rebuttal would be stronger without the first three paragraphs, and they certainly aren't the best, or most praiseworthy, part of it.
  - coffeeaddicted 15y ago
    Allen attacks Kurzweil rather directly in his essay and the essay title is even "The Singularity Isn't Near". So in that case I think it's fair game if Kurzweil calls him out for not even regarding the arguments he made in "The Singularity is near".
- bh42222 15y ago
  Neither Kurzweil nor Allen have done their homework. This is a disappointingly informal argument if you're looking for hard scientific facts.
shin_lao 15y ago
Science progress is not just a question of intelligence. Therefore, even if computers become more intelligent than humans, it is doubtful a "singularity" will occur.
- loup-vaillant 15y ago
  If not, then what? Even while I agree with Alan Kay when he says that IQ << knowledge << outlook. But all three happen in the brain regardless. Imagine we manage to build a machine that produce more insights than Newton. That particular form of intelligence would be quite likely to trigger a singularity, don't you think?
  - shin_lao 15y ago
    No I don't, because you still need time and a certain amount of randomnesses to make discoveries. Another way to put it is that even if you are twice as intelligent as Newton, you won't discover twice as many things or discover things twice faster.
    - weaponofchoice 15y ago
      Sifting through the day's top-list on the AI appstore... WTH's this? "Newton AI. The power of a 1000 research assistants at the click of a button, and they'll run all day tirelessly.". "99c launch deal, just for today -- get it now!" Hmm. Click. You're now waiting for it to download to your little AiPod that'll beam 'brain bits' to your home-bots, that're now busy sketching out the next monalisa onto a couple of shiny new dreamPads. Why wouldn't those startup dudes down the street try and build a beefier/faster "runs at 50x universe speed" AiPod for the AI platform you just bought your Newton AI app for? ------- Why wouldn't AI be able to simulate 'regular universe time/ human time' faster? Why couldn't AI have stronger, more varied randomness? The bottleneck would be interactions that require a peek into regular universe time: live human input (phone calls, emails), weather, biological data etc.
politician 15y ago
The Singularity concept strikes me as a sort of wishful thinking. Technology advancing so fast that we no longer can control or understand it? Yeah, that already happened to my parent's generation with AOL, yet here I am texting this on my iPhone. New generations understand intuitively what the previous generation understood theoretically. Even so, I fully expect memristors to deliver strong AI.
- 0x12 15y ago
  > Even so, I fully expect memristors to deliver strong AI. Why would they? Is strong AI a function of storage capacity or speed? An AI running at 1/100th of what a future AI may be capable of is still an AI and I can't see how a mere improvement of a couple of orders of magnitude would do what decades of Moore's law have failed to do so far. If strong AI was just a matter of speed then we could theoretically take any of the large clusters available today and run it at an appreciable fraction of its normal speed leading at a minimum to a validation of the fact that it is indeed a strong AI that's been created. The barrier seems to be more that we don't know how to go about building one from a software perspective than that we wouldn't have the capability to design the hardware. So how would an advance in hardware suddenly fix that?
  - politician 15y ago
    Memristors are the silicon equivalent of neurons -- a time-dependent function with state. In mammals, the connectivity of millions of neurons enable the emergence of intelligence. I don't see any reason why a silicon-based neuron network of sufficient size isn't capable of the same. Now, that's entirely different from purposeful design of an AI such that it speaks English and knows what I like for breakfast. I don't think we'll ever know how to sit down and write one in Notepad. Instead, memristor-based AIs will be evolved using genetic algorithms or other evolutionary approaches. Yeah, maybe that's wishful thinking too, but that's how I see it playing out.
    - 0x12 15y ago
      A memristor can be simulated by a couple of formulas and a few variables. (just like neural networks can be simulated). In spite of our ability to simulate this sort of process for decades we have not succeeded in building a strong AI, the difference in having a hardware version is one of performance (just like Neural Network chips are typically a lot faster than their software simulated counterparts). There is no real difference in capability here, just a (possibly very large) speedup. Now, I'm not ruling out that such a speedup will cause us to be able to create things that so far were not possible but I have a hard time convincing myself that this will almost certainly be the case.
suivix 15y ago
It's startling to me that everything I do comes from 50 megabytes of source code.
- natch 15y ago
  Well of course that's just the substrate. A lot of what you do comes from what you've learned and continue to learn from your environment.
- archgoon 15y ago
  Running on a very complicated instruction set.
  - suivix 15y ago
    Same as the comment above, is it different from a worm's instruction set?
  - bermanoid 15y ago
    Let's say that you saw an executable file weighing in at 25 megabytes of hand-coded assembly running on a modern day Core i7 chip. Would you really assume that the source code that it would take to write such a program would be over an order of magnitude different if you were targeting a RISC processor instead? Now give yourself access to an expressive compiled language, and estimate how much code it would take. Does the fact that you're targeting RISC even matter anymore, algorithmically? Unless you're assuming that the biological "instruction set" has some hard coded primitives that make AI an easy problem, it literally doesn't matter at all that the instruction set is complicated (or rather, it matters up to a small constant factor), given that our programming constructs are vastly more powerful than those available to neurons. It's the connectivity algorithms that are important, and biology has absolutely no advantage there.
- sp332 15y ago
  Don't forget the runtime environment though :)
  - 15y ago
losvedir 15y ago
Ugh, not this again. That would mean that the design of the brain would require hundreds of trillions of bytes of information. Yet the design of the brain (like the rest of the body) is contained in the genome. I believe it was on HN that this discussion came up before, but that's a short sighted way of looking at it. Basically, it doesn't take into account all the interactions of the environment required to turn that "source code" into a person. Sure, the DNA would be sufficient if you were able to accurately simulate cellular actions, protein folding, and physics in general, but we just can't do that yet, and it doesn't look like we'll be able to any time soon.
- fl3tch 15y ago
  Exactly. Distributed computing grids still have trouble folding single peptides with reasonable accuracy. We have the source code, but we don't have the compiler.
  - rsl7 15y ago
    Worse, there are about to be seven billion brains on the planet. Not merely one.. and human brains aren't so great in isolation.
    - wanorris 15y ago
      Why would it need to be in isolation? Machine vision, voice recognition, speech synthesis, robotics -- there's no obvious reason why if we could build such a brain we couldn't find a way for it to interact with people.
      - rsl7 15y ago
        I agree, but the devil is in those details. Look at the variations among actual humans.
- bermanoid 15y ago
  Basically, it doesn't take into account all the interactions of the environment required to turn that "source code" into a person. This is an extremely common, and completely incorrect criticism as applied to AGI complexity estimates. Here's the way to think of it: the entirety of the information content required to move from non-intelligence to intelligence has to have been figured out some time between single-celled organisms and humans, because the substrate on which our intelligence is implemented literally didn't exist at that point. Which means that any part of the biological machinery that was in place when single-celled organisms ruled the planet does not count towards the complexity of the "intelligence algorithm" itself - it's irrelevant, accidental complexity, not information content that is required to get from "working computer" to "working intelligent computer". You'll find that almost all of the cellular actions, protein folding, and physics were already working just fine when the single-celled ickies were evolving, so it's all complexity that we can safely ignore, which means we can start the complexity count with DNA. Apart from (IMO) extremely minor epigenetic contributions, the pure-DNA information estimates should provide extremely hard upper bounds on the difficulty of the problem, estimates that we'll probably blow through quite easily once we know what we're doing - evolution rarely finds the optimal solutions to problems, I see no reason to assume that it stumbled across one in this case...
bh42222 15y ago
Allen writes that "the Law of Accelerating Returns (LOAR). . . is not a physical law." I would point out that most scientific laws are not physical laws, but result from the emergent properties of a large number of events at a finer level. A classical example is the laws of thermodynamics (LOT). If you look at the mathematics underlying the LOT, they model each particle as following a random walk. Oh that's a terrible point! Thermodynamics laws are nothing like predictions about the future. I would have thought linguistic slight of hand like this is beneath Kurzweil. Allen's statement that every structure and neural circuit is unique is simply impossible. That would mean that the design of the brain would require hundreds of trillions of bytes of information. Yet the design of the brain (like the rest of the body) is contained in the genome. The design of the human brain is not entirely contained in the genome! As soon as we mapped the human genome we were faced with a paradox. How come the complexity difference between us and mice for example, is NOT proportional to the difference in our genomes? Here's an article form 2002 "Just 2.5% of DNA turns mice into men": http://www.newscientist.com/article/dn2352-just-25-of-dna-turns-mice-into-men.html http://www.newscientist.com/article/dn2352-just-25-of-dna-tu... In other words, if you look at just how the genomes are different then humans and mice ought be a lot more similar than we are. We have since come to find out just what a huge role the feedback-interactions of DNA and its products, like proteins and all kinds of RNA, play in the development of life. This staggeringly complex feedback mechanism is why despite the mapping of the human genome, medical progress still remains excruciatingly slow. Much, much faster the before! But not nearly as fast as we had hopped when the human genome was first mapped. Note that epigenetic information (such as the peptides controlling gene expression) do not appreciably add to the amount of information in the genome. This is true in that they don't add much to the genome. But it is profoundly wrong in that they do add hugely to the actual resulting phenotype. Kurzweil continues in this same vane for a while. I don't know if he has just never bothered to look into the latest research or if his understandably strong desire to not die has resulted in a huge confirmation bias. When Kurzweil talks about the general trend of scientific progress I tend to agree with him. But neither Paul Allen nor anyone else disagrees with the notion that we will reach the singularity at some point in the future. The argument is about the timing. And timing the future, is like timing the stock market, something I don't care to try to do. But when Kurzweil attempts to convince the reader that the singularity is near, by using specific examples, that's when I start do disagree with him. Because once he starts being specific, it becomes easy for me to see where he is wrong, factually, objectively wrong.
api 15y ago
What about energy? It's true that if you look at most areas of technology they are advancing rapidly. Except energy. Energy has stagnated since the 1950s. I'm on the fence on this issue, but there are many very intelligent and knowledgeable people who are predicting a kind of anti-singularity: in the 21st century, fossil fuel depletion will send us way back, perhaps even de-industrialize most societies. Is our civilization simply a machine that is transferring the order (low entropy state) in fossil fuels into order within itself (technology and economic complexity), and when those fossil fuels run out will this ordering process cease? The lack of major breakthroughs in energy in the past 50 years is pretty dramatic. Nuclear looked like an energy panacea once, but it's turned out to be clunky and hard to scale. Solar panels and wind turbines are interesting, but the problem with those is that we basically can't store energy. Energy storage is either super-expensive per kilowatt-hour and not scalable (e.g. Li-Ion batteries) or very inefficient (e.g. water electrolysis to hydrogen). Without a breakthrough on the order of cheap ultra-capacitors or fusion, I'm afraid we'll be seeing peak everything pretty soon, including technological complexity. The thing is: all the technologies of the "singularity" are energy consumers. Where are the producers? What is going to power the singularity? Then there's another area that makes me horribly pessimistic: politics. Most of our societies are degenerating to banana republic levels of corruption. Even if the energy problem is technically solvable, it seems to me that our political systems may be set up to do the absolute worst possible thing in this area: ride the fossil fuel crash into the ground in an orgy of war and despotism.
- lupatus 15y ago
  Note: I made a similar comment a few days ago. It seems that you might be slightly misinformed about energy. Since the 1950s, there have been a number of advancements in energy production. Some examples are efficient shale oil extraction, tar sands oil extraction, horizontal drilling, deep water drilling, and high Arctic drilling. As The Futurist discusses in depth[1], annual world oil consumption has been hovering around 32 billion barrels since about 1982. That means oil consumption, at $100/barrel, is $3.2 trillion, or 5% of nominal world GDP. My take away from that is technology has made the oil supply a non-issue. And, it will continue not being a problem for the foreseeable future. It is not a hard resource limit. Also, check out the the research going into liquid fluoride thorium fusion. It seems to have potential as a way to do cheap, safe, "easy" nuclear fusion. Personally, I think that the problem is human willpower and chicken-little attitudes. To increase my technology optimism, I read nextbigfuture.com. Nearly daily, I see an article posted there that makes me go, "Holy crap! We can do that now??!" [1] http://www.singularity2050.com/2011/07/the-end-of-petrotyrany.html http://www.singularity2050.com/2011/07/the-end-of-petrotyran...
api 15y ago
Another response: Kurzweil also ignores economics. The advance of technology is driven in part by economic forces. Computing power may stagnate not because we have reached physical limits but because present-day computers are good enough for what 98% of the market wants. I see this trend developing. If anything, the trend in consumer computing is toward less powerful but lower-power and more portable computing devices. My current laptop -- a Macbook Air -- is actually slower than my previous laptop. But it is more portable and uses less energy. And it does everything I want. I don't need more power right now. The only areas driving the performance end are gaming, high performance computing, and high-capacity data centers. How long will those go until they too are basically satiated? We've seen this in other areas. The envelope for aviation maxed out in the 1970s with things like the U2. Space flight seems to just now be emerging from a long coma with things like SpaceX, but on closer examination SpaceX is just reviving 1960s ideas and doing them at a lower cost with modern control systems and materials technology. My other reply about energy deals with supply-side limits to growth. This response deals with demand limits to growth.
- lupatus 15y ago
  What the market wants is every map, book, song, movie, game, and poem ever made to be instantly accessible, searchable, and reviewable. They want their work to be autosyncing, autobackedup, and to follow them from device to device. They want to be able to securely talk with friends and family at any time, to publically talk with friends and family at any time, to discover new friends and family, and to be able to completely disconnect from friends and family at will. They want intelligent tools that keep them from making dumb decisions, tools to help them make even better good decisions, and tools that won't get in the way of them making dumb decisions. I think that the market for computing power has a looong way to go before it taps-out what the market demands.
  - api 15y ago
    But how much do they want all this, and does this require major improvements in computing power? It looks to me that almost everything you listed could be done on the computers of five years ago. It's all nothing but software improvements. Existing hardware is good enough.
kenjackson 15y ago
What are the big advances in linear programming that happened since 1988? As api mentions in the comments here on HN there are areas of work where progress stopped. For example passenger jet speed was once thought to continue at a rapid rate such that LA to Europe flights would be a few hours. Skyscraper height was thought to continue, with advances in various technologies and engineering methods making it desirable. Both hit realities that signficantly slowed down their progress. I tend to side with Allen on this. While we're bright people, I don't know if I see us able to keep increasing computing power while keeping actual power consumption reasonably low.
- api 15y ago
  What passenger jet speed and skyscrapers really hit is economics: demand limits to growth. Most super-tall skyscrapers are economic disasters. There seems to be a maximum economically rational height to a skyscraper, and it's already been reached. You can build higher, but if you do you're wasting your money. A human-level or beyond AI would probably be like the Burj Khalifa: an economic disaster. Why build it when screwing and popping out babies is far cheaper and already works? If you want to exceed human intelligence, it would be a lot cheaper to augment human brains with external digital assistants (like what you're using now) or implants than to re-engineer an entirely new embodiment.
  - eavc 15y ago
    >Why build it when screwing and popping out babies is far cheaper and already works? Why build a word processor when pencil, paper, and a scribe is far cheaper and already works? It's about scale.
  - cadalac 15y ago
    The super tall skyscrapers are indeed an economical disaster in them selves, but they skyrocket the value of the surrounding land. The Saudi Prince who's building the Kingdom Tower owns a considerable amount of land around the building site so he'll make a lot of money off the land he sells. It kind of reminds me of how Google makes money with adverting on their free products.
- 15y ago
hugh3 15y ago
I grow tired of Kurzweil's vague arguments against people who disagree with his vague predictions. What I think Kurzweil doesn't understand is that in any argument about what's going to happen in the future, the onus of proof inevitably lies with the guy saying "This is what's going to happen", not the guy saying "Ehh, maybe not". I don't know what's going to happen in the future, and I don't pretend to know what's going to happen in the future, but whatever happens either (a) I'll find out eventually or (b) it'll happen after I'm dead anyway. But john_b's point about Kurzweil's lack of a null hypothesis is a good one. So my question for Kurzweil is this: what will the world look like if you're wrong? What possibilities are your predictions excluding? If I'm still alive in 2060, and I look around at the world around me, under precisely what conditions am I entitled to say "Well whaddya know, looks like Kurzweil was wrong about that Singularity thing after all"?
- api 15y ago
  IMHO, the future cannot be predicted. Period. I'm not talking impossible-as-in-hard. I am talking impossible-as-in-perpetual-motion. Nice well-behaved linear Newtonian systems can be modeled and predicted. There are systems that are chaotic but that can be modeled in the aggregate very well too, like thermodynamic systems and certain kinds of fluid flow. Life isn't like any of that. Life is complex, chaotic, computationally irreducible, and full of feedback loops on top of feedback loops. Even worse: predictions often create economic incentives to prove them wrong. Take a position on the stock market and you have created an incentive for your prediction to not come true. People have always wanted to deny the fundamental unpredictability of history, and have always clung to woo-woo prophecy superstitions toward this end. The ancients had Tarot cards and pig entrails. We have graphs and computer models.
  - hugh3 15y ago
    Of course, predicting the future with 100% certainty is impossible. But that doesn't mean that making predictions is a mug's game. Predicting the future, with appropriate levels of uncertainty, is a very sensible thing to do with your time. I, for instance, predict that if I wander down to Cheeseboard in twenty minutes I'll find that they're selling delicious pizza. And I predict that if I eat that pizza, then it won't poison me. These are all useful predictions, which may be wrong, but are useful nonetheless. It's only when you start sticking inappropriate error bars on your predictions that it becomes a problem. Kurzweil predicts things which are unlikely or perhaps impossible as having probabilities near 100%.
cpeterso 15y ago
Here is a rough transcript of a Long Now Foundation talk by SF author Vernor Vinge (and coiner of the term "singularity") entitled "What If the Singularity Does NOT Happen?". He sees: * Scenario 1: A Return to MADness (nuclear war) * Scenario 2: The Golden Age (peace and prosperity) * Scenario 3: The Wheel of Time (catastrophic natural disaster) http://www-rohan.sdsu.edu/faculty/vinge/longnow/ http://www-rohan.sdsu.edu/faculty/vinge/longnow/
super_mario 15y ago
String AI is not a hardware problem. It's not a matter of lack of computational power. It is a software and modeling problem. If you had an AI algorithm and a model, you could still run it on any Turning machine. It would just take a lot longer (perhaps years or decades or more) to compute a single thought on current hardware instead of real time or faster than real time on some super fast future hardware. There are people (like Roger Penrose) who argued that intelligence and consciousness are not computational in nature (and hence no algorithm can be conscious). Penrose goes all the way down to quantum mechanical effects in the brain. I have not really followed developments on this and where Penrose's argument currently stands.
- spot 15y ago
  Nobody takes him seriously. He is just a Christian apologist who wraps it up with quantum who haa.

102 points by ca98am79 15y ago

Kurzweil makes some good points here. He doesn't address each and every criticism with his view of AI progress, but he does a good job calling out Paul Allen on not doing the homework.

DanielStraight 15y ago
Calling out someone for not doing their homework seems somewhere in the DH1-2 range: http://www.paulgraham.com/disagree.html http://www.paulgraham.com/disagree.html There are some good points in Kurzweil's response, but the ones about Paul Allen are definitely not them. I think his best point was about extrapolating function from individual cells or structures, without needing to understand every single cell or structure individually.
- jerf 15y ago
  I disagree. When one "addresses" a long-standing, well-thought out argument with off-the-cuff snap statement or something only slightly more thought-out, it is perfectly permissible to call them out as not being serious. It's like going into a serious religion debate with "Evil exists, therefore God does not. Ah-ha, you are defeated!", or "God must exist because something must have started it all, so there!" as if in the thousands of years of years the debate has been raging on nobody has ever thought of those things, or addressed them at length, in both directions. If you're going to debate the singularity here, maybe you can get by with just stating "I don't believe it's possible" without citing any logic, which as of this writing there's at least two people in this comment set already who have simply stated that without defense, but if you're going to debate one of the leaders of the field it would help if you would at least grant your opponent the courtesy of thinking that just maybe over the course of the decades he's been thinking about this, the obvious objections that you thought up in five seconds just might have been addressed at some point. You may not think they've been adequately or correctly addressed, but don't pretend they haven't been addressed at all. Personally I'm not completely sold on the matter for a variety of reasons myself, but the usual logic given for why you should be skeptical about it is terrible. The interesting questions are a great deal more complicated than something that can be dismissed with something that generally boils down to "Look, I just can't imagine the world changing that much, so it won't".
  - ambler0 15y ago
    I think your comparison to arguments about religion is apt. Many of the opponents to Kurzweil's ideas remind me of those whose opposition to the possibility of a godless universe amounts to, "I can't imagine it, so I don't believe it."
    - john_b 15y ago
      On the other hand, Kurzweil (at least in his essays and articles) often ignores the question of what a fair null hypothesis is for the possibility of the singularity. I think his gift for creating a compelling vision tends to make people forget that the null hypothesis for a scientific assertion is doubt. Kurzweil provides both high level general evidence (like improvements in computation) and low level, domain-specific evidence (like the discussion about the pancreas) to support his claims, but none of that justifies the use of the word "law" in "law of accelerating returns". He attempts an analogy with thermodynamic laws and how they are derived from underlying statistical principles, but there are no underlying fundamental principles of human innovation and progress that are in any way comparable to the certainty and universality of physical laws. This, I think, is why a lot of people (myself included) have a hard time taking him seriously. He tries to apply the same kind of formal analysis that works well in science to human beings and the complex, highly non-scientific processes that underly innovation today. The bottom line is that, until the singularity occurs, human beings will still be needed to build ever more complex and powerful systems, but human beings do not progress at anything close to an exponential rate.
  - DanielStraight 15y ago
    I guess my point was just that calling him out, while strictly correct, is doing nothing to refute whatever statements he may have made on the singularity, nor is it doing anything to promote the idea of the singularity. Replying to "evil exists, therefore God does not," with, "that argument has been made before," doesn't say anything about whether God does or doesn't exist, and it doesn't do anything to refute the argument. The point of PG's How To Disagree is that the point of disagreeing is to address the truth or falsehood of a proposition, not the way the proposition is presented. The DH# levels (at least DH0-2) can be restated something like this: DH0: I don't like the author, therefore his argument is false. DH1: The author has a relevant fault or weakness, therefore his argument is false. DH2: The author didn't present his argument well, therefore his argument is false. While it's nothing against him personally, the first three paragraphs of Kurzweil's reply read like "the author didn't present his argument well, therefore it is false." He recovers later, sure. He easily reaches 5-6 on the DH scale, but the rebuttal would be stronger without the first three paragraphs, and they certainly aren't the best, or most praiseworthy, part of it.
- coffeeaddicted 15y ago
  Allen attacks Kurzweil rather directly in his essay and the essay title is even "The Singularity Isn't Near". So in that case I think it's fair game if Kurzweil calls him out for not even regarding the arguments he made in "The Singularity is near".
bh42222 15y ago
Neither Kurzweil nor Allen have done their homework. This is a disappointingly informal argument if you're looking for hard scientific facts.

shin_lao 15y ago

Science progress is not just a question of intelligence. Therefore, even if computers become more intelligent than humans, it is doubtful a "singularity" will occur.

loup-vaillant 15y ago
If not, then what? Even while I agree with Alan Kay when he says that IQ << knowledge << outlook. But all three happen in the brain regardless. Imagine we manage to build a machine that produce more insights than Newton. That particular form of intelligence would be quite likely to trigger a singularity, don't you think?
- shin_lao 15y ago
  No I don't, because you still need time and a certain amount of randomnesses to make discoveries. Another way to put it is that even if you are twice as intelligent as Newton, you won't discover twice as many things or discover things twice faster.
  - weaponofchoice 15y ago
    Sifting through the day's top-list on the AI appstore... WTH's this? "Newton AI. The power of a 1000 research assistants at the click of a button, and they'll run all day tirelessly.". "99c launch deal, just for today -- get it now!" Hmm. Click. You're now waiting for it to download to your little AiPod that'll beam 'brain bits' to your home-bots, that're now busy sketching out the next monalisa onto a couple of shiny new dreamPads. Why wouldn't those startup dudes down the street try and build a beefier/faster "runs at 50x universe speed" AiPod for the AI platform you just bought your Newton AI app for? ------- Why wouldn't AI be able to simulate 'regular universe time/ human time' faster? Why couldn't AI have stronger, more varied randomness? The bottleneck would be interactions that require a peek into regular universe time: live human input (phone calls, emails), weather, biological data etc.

politician 15y ago

The Singularity concept strikes me as a sort of wishful thinking. Technology advancing so fast that we no longer can control or understand it? Yeah, that already happened to my parent's generation with AOL, yet here I am texting this on my iPhone. New generations understand intuitively what the previous generation understood theoretically. Even so, I fully expect memristors to deliver strong AI.

0x12 15y ago
> Even so, I fully expect memristors to deliver strong AI. Why would they? Is strong AI a function of storage capacity or speed? An AI running at 1/100th of what a future AI may be capable of is still an AI and I can't see how a mere improvement of a couple of orders of magnitude would do what decades of Moore's law have failed to do so far. If strong AI was just a matter of speed then we could theoretically take any of the large clusters available today and run it at an appreciable fraction of its normal speed leading at a minimum to a validation of the fact that it is indeed a strong AI that's been created. The barrier seems to be more that we don't know how to go about building one from a software perspective than that we wouldn't have the capability to design the hardware. So how would an advance in hardware suddenly fix that?
- politician 15y ago
  Memristors are the silicon equivalent of neurons -- a time-dependent function with state. In mammals, the connectivity of millions of neurons enable the emergence of intelligence. I don't see any reason why a silicon-based neuron network of sufficient size isn't capable of the same. Now, that's entirely different from purposeful design of an AI such that it speaks English and knows what I like for breakfast. I don't think we'll ever know how to sit down and write one in Notepad. Instead, memristor-based AIs will be evolved using genetic algorithms or other evolutionary approaches. Yeah, maybe that's wishful thinking too, but that's how I see it playing out.
  - 0x12 15y ago
    A memristor can be simulated by a couple of formulas and a few variables. (just like neural networks can be simulated). In spite of our ability to simulate this sort of process for decades we have not succeeded in building a strong AI, the difference in having a hardware version is one of performance (just like Neural Network chips are typically a lot faster than their software simulated counterparts). There is no real difference in capability here, just a (possibly very large) speedup. Now, I'm not ruling out that such a speedup will cause us to be able to create things that so far were not possible but I have a hard time convincing myself that this will almost certainly be the case.

suivix 15y ago

It's startling to me that everything I do comes from 50 megabytes of source code.

natch 15y ago
Well of course that's just the substrate. A lot of what you do comes from what you've learned and continue to learn from your environment.
archgoon 15y ago
Running on a very complicated instruction set.
- suivix 15y ago
  Same as the comment above, is it different from a worm's instruction set?
- bermanoid 15y ago
  Let's say that you saw an executable file weighing in at 25 megabytes of hand-coded assembly running on a modern day Core i7 chip. Would you really assume that the source code that it would take to write such a program would be over an order of magnitude different if you were targeting a RISC processor instead? Now give yourself access to an expressive compiled language, and estimate how much code it would take. Does the fact that you're targeting RISC even matter anymore, algorithmically? Unless you're assuming that the biological "instruction set" has some hard coded primitives that make AI an easy problem, it literally doesn't matter at all that the instruction set is complicated (or rather, it matters up to a small constant factor), given that our programming constructs are vastly more powerful than those available to neurons. It's the connectivity algorithms that are important, and biology has absolutely no advantage there.
sp332 15y ago
Don't forget the runtime environment though :)
- 15y ago

losvedir 15y ago

Ugh, not this again. That would mean that the design of the brain would require hundreds of trillions of bytes of information. Yet the design of the brain (like the rest of the body) is contained in the genome. I believe it was on HN that this discussion came up before, but that's a short sighted way of looking at it. Basically, it doesn't take into account all the interactions of the environment required to turn that "source code" into a person. Sure, the DNA would be sufficient if you were able to accurately simulate cellular actions, protein folding, and physics in general, but we just can't do that yet, and it doesn't look like we'll be able to any time soon.

fl3tch 15y ago
Exactly. Distributed computing grids still have trouble folding single peptides with reasonable accuracy. We have the source code, but we don't have the compiler.
- rsl7 15y ago
  Worse, there are about to be seven billion brains on the planet. Not merely one.. and human brains aren't so great in isolation.
  - wanorris 15y ago
    Why would it need to be in isolation? Machine vision, voice recognition, speech synthesis, robotics -- there's no obvious reason why if we could build such a brain we couldn't find a way for it to interact with people.
    - rsl7 15y ago
      I agree, but the devil is in those details. Look at the variations among actual humans.
bermanoid 15y ago
Basically, it doesn't take into account all the interactions of the environment required to turn that "source code" into a person. This is an extremely common, and completely incorrect criticism as applied to AGI complexity estimates. Here's the way to think of it: the entirety of the information content required to move from non-intelligence to intelligence has to have been figured out some time between single-celled organisms and humans, because the substrate on which our intelligence is implemented literally didn't exist at that point. Which means that any part of the biological machinery that was in place when single-celled organisms ruled the planet does not count towards the complexity of the "intelligence algorithm" itself - it's irrelevant, accidental complexity, not information content that is required to get from "working computer" to "working intelligent computer". You'll find that almost all of the cellular actions, protein folding, and physics were already working just fine when the single-celled ickies were evolving, so it's all complexity that we can safely ignore, which means we can start the complexity count with DNA. Apart from (IMO) extremely minor epigenetic contributions, the pure-DNA information estimates should provide extremely hard upper bounds on the difficulty of the problem, estimates that we'll probably blow through quite easily once we know what we're doing - evolution rarely finds the optimal solutions to problems, I see no reason to assume that it stumbled across one in this case...

bh42222 15y ago

Allen writes that "the Law of Accelerating Returns (LOAR). . . is not a physical law." I would point out that most scientific laws are not physical laws, but result from the emergent properties of a large number of events at a finer level. A classical example is the laws of thermodynamics (LOT). If you look at the mathematics underlying the LOT, they model each particle as following a random walk. Oh that's a terrible point! Thermodynamics laws are nothing like predictions about the future. I would have thought linguistic slight of hand like this is beneath Kurzweil. Allen's statement that every structure and neural circuit is unique is simply impossible. That would mean that the design of the brain would require hundreds of trillions of bytes of information. Yet the design of the brain (like the rest of the body) is contained in the genome. The design of the human brain is not entirely contained in the genome! As soon as we mapped the human genome we were faced with a paradox. How come the complexity difference between us and mice for example, is NOT proportional to the difference in our genomes? Here's an article form 2002 "Just 2.5% of DNA turns mice into men": http://www.newscientist.com/article/dn2352-just-25-of-dna-turns-mice-into-men.html http://www.newscientist.com/article/dn2352-just-25-of-dna-tu... In other words, if you look at just how the genomes are different then humans and mice ought be a lot more similar than we are. We have since come to find out just what a huge role the feedback-interactions of DNA and its products, like proteins and all kinds of RNA, play in the development of life. This staggeringly complex feedback mechanism is why despite the mapping of the human genome, medical progress still remains excruciatingly slow. Much, much faster the before! But not nearly as fast as we had hopped when the human genome was first mapped. Note that epigenetic information (such as the peptides controlling gene expression) do not appreciably add to the amount of information in the genome. This is true in that they don't add much to the genome. But it is profoundly wrong in that they do add hugely to the actual resulting phenotype. Kurzweil continues in this same vane for a while. I don't know if he has just never bothered to look into the latest research or if his understandably strong desire to not die has resulted in a huge confirmation bias. When Kurzweil talks about the general trend of scientific progress I tend to agree with him. But neither Paul Allen nor anyone else disagrees with the notion that we will reach the singularity at some point in the future. The argument is about the timing. And timing the future, is like timing the stock market, something I don't care to try to do. But when Kurzweil attempts to convince the reader that the singularity is near, by using specific examples, that's when I start do disagree with him. Because once he starts being specific, it becomes easy for me to see where he is wrong, factually, objectively wrong.

api 15y ago

What about energy? It's true that if you look at most areas of technology they are advancing rapidly. Except energy. Energy has stagnated since the 1950s. I'm on the fence on this issue, but there are many very intelligent and knowledgeable people who are predicting a kind of anti-singularity: in the 21st century, fossil fuel depletion will send us way back, perhaps even de-industrialize most societies. Is our civilization simply a machine that is transferring the order (low entropy state) in fossil fuels into order within itself (technology and economic complexity), and when those fossil fuels run out will this ordering process cease? The lack of major breakthroughs in energy in the past 50 years is pretty dramatic. Nuclear looked like an energy panacea once, but it's turned out to be clunky and hard to scale. Solar panels and wind turbines are interesting, but the problem with those is that we basically can't store energy. Energy storage is either super-expensive per kilowatt-hour and not scalable (e.g. Li-Ion batteries) or very inefficient (e.g. water electrolysis to hydrogen). Without a breakthrough on the order of cheap ultra-capacitors or fusion, I'm afraid we'll be seeing peak everything pretty soon, including technological complexity. The thing is: all the technologies of the "singularity" are energy consumers. Where are the producers? What is going to power the singularity? Then there's another area that makes me horribly pessimistic: politics. Most of our societies are degenerating to banana republic levels of corruption. Even if the energy problem is technically solvable, it seems to me that our political systems may be set up to do the absolute worst possible thing in this area: ride the fossil fuel crash into the ground in an orgy of war and despotism.

lupatus 15y ago
Note: I made a similar comment a few days ago. It seems that you might be slightly misinformed about energy. Since the 1950s, there have been a number of advancements in energy production. Some examples are efficient shale oil extraction, tar sands oil extraction, horizontal drilling, deep water drilling, and high Arctic drilling. As The Futurist discusses in depth[1], annual world oil consumption has been hovering around 32 billion barrels since about 1982. That means oil consumption, at $100/barrel, is $3.2 trillion, or 5% of nominal world GDP. My take away from that is technology has made the oil supply a non-issue. And, it will continue not being a problem for the foreseeable future. It is not a hard resource limit. Also, check out the the research going into liquid fluoride thorium fusion. It seems to have potential as a way to do cheap, safe, "easy" nuclear fusion. Personally, I think that the problem is human willpower and chicken-little attitudes. To increase my technology optimism, I read nextbigfuture.com. Nearly daily, I see an article posted there that makes me go, "Holy crap! We can do that now??!" [1] http://www.singularity2050.com/2011/07/the-end-of-petrotyrany.html http://www.singularity2050.com/2011/07/the-end-of-petrotyran...

api 15y ago

Another response: Kurzweil also ignores economics. The advance of technology is driven in part by economic forces. Computing power may stagnate not because we have reached physical limits but because present-day computers are good enough for what 98% of the market wants. I see this trend developing. If anything, the trend in consumer computing is toward less powerful but lower-power and more portable computing devices. My current laptop -- a Macbook Air -- is actually slower than my previous laptop. But it is more portable and uses less energy. And it does everything I want. I don't need more power right now. The only areas driving the performance end are gaming, high performance computing, and high-capacity data centers. How long will those go until they too are basically satiated? We've seen this in other areas. The envelope for aviation maxed out in the 1970s with things like the U2. Space flight seems to just now be emerging from a long coma with things like SpaceX, but on closer examination SpaceX is just reviving 1960s ideas and doing them at a lower cost with modern control systems and materials technology. My other reply about energy deals with supply-side limits to growth. This response deals with demand limits to growth.

lupatus 15y ago
What the market wants is every map, book, song, movie, game, and poem ever made to be instantly accessible, searchable, and reviewable. They want their work to be autosyncing, autobackedup, and to follow them from device to device. They want to be able to securely talk with friends and family at any time, to publically talk with friends and family at any time, to discover new friends and family, and to be able to completely disconnect from friends and family at will. They want intelligent tools that keep them from making dumb decisions, tools to help them make even better good decisions, and tools that won't get in the way of them making dumb decisions. I think that the market for computing power has a looong way to go before it taps-out what the market demands.
- api 15y ago
  But how much do they want all this, and does this require major improvements in computing power? It looks to me that almost everything you listed could be done on the computers of five years ago. It's all nothing but software improvements. Existing hardware is good enough.

kenjackson 15y ago

What are the big advances in linear programming that happened since 1988? As api mentions in the comments here on HN there are areas of work where progress stopped. For example passenger jet speed was once thought to continue at a rapid rate such that LA to Europe flights would be a few hours. Skyscraper height was thought to continue, with advances in various technologies and engineering methods making it desirable. Both hit realities that signficantly slowed down their progress. I tend to side with Allen on this. While we're bright people, I don't know if I see us able to keep increasing computing power while keeping actual power consumption reasonably low.

api 15y ago
What passenger jet speed and skyscrapers really hit is economics: demand limits to growth. Most super-tall skyscrapers are economic disasters. There seems to be a maximum economically rational height to a skyscraper, and it's already been reached. You can build higher, but if you do you're wasting your money. A human-level or beyond AI would probably be like the Burj Khalifa: an economic disaster. Why build it when screwing and popping out babies is far cheaper and already works? If you want to exceed human intelligence, it would be a lot cheaper to augment human brains with external digital assistants (like what you're using now) or implants than to re-engineer an entirely new embodiment.
- eavc 15y ago
  >Why build it when screwing and popping out babies is far cheaper and already works? Why build a word processor when pencil, paper, and a scribe is far cheaper and already works? It's about scale.
- cadalac 15y ago
  The super tall skyscrapers are indeed an economical disaster in them selves, but they skyrocket the value of the surrounding land. The Saudi Prince who's building the Kingdom Tower owns a considerable amount of land around the building site so he'll make a lot of money off the land he sells. It kind of reminds me of how Google makes money with adverting on their free products.
15y ago

hugh3 15y ago

I grow tired of Kurzweil's vague arguments against people who disagree with his vague predictions. What I think Kurzweil doesn't understand is that in any argument about what's going to happen in the future, the onus of proof inevitably lies with the guy saying "This is what's going to happen", not the guy saying "Ehh, maybe not". I don't know what's going to happen in the future, and I don't pretend to know what's going to happen in the future, but whatever happens either (a) I'll find out eventually or (b) it'll happen after I'm dead anyway. But john_b's point about Kurzweil's lack of a null hypothesis is a good one. So my question for Kurzweil is this: what will the world look like if you're wrong? What possibilities are your predictions excluding? If I'm still alive in 2060, and I look around at the world around me, under precisely what conditions am I entitled to say "Well whaddya know, looks like Kurzweil was wrong about that Singularity thing after all"?

api 15y ago
IMHO, the future cannot be predicted. Period. I'm not talking impossible-as-in-hard. I am talking impossible-as-in-perpetual-motion. Nice well-behaved linear Newtonian systems can be modeled and predicted. There are systems that are chaotic but that can be modeled in the aggregate very well too, like thermodynamic systems and certain kinds of fluid flow. Life isn't like any of that. Life is complex, chaotic, computationally irreducible, and full of feedback loops on top of feedback loops. Even worse: predictions often create economic incentives to prove them wrong. Take a position on the stock market and you have created an incentive for your prediction to not come true. People have always wanted to deny the fundamental unpredictability of history, and have always clung to woo-woo prophecy superstitions toward this end. The ancients had Tarot cards and pig entrails. We have graphs and computer models.
- hugh3 15y ago
  Of course, predicting the future with 100% certainty is impossible. But that doesn't mean that making predictions is a mug's game. Predicting the future, with appropriate levels of uncertainty, is a very sensible thing to do with your time. I, for instance, predict that if I wander down to Cheeseboard in twenty minutes I'll find that they're selling delicious pizza. And I predict that if I eat that pizza, then it won't poison me. These are all useful predictions, which may be wrong, but are useful nonetheless. It's only when you start sticking inappropriate error bars on your predictions that it becomes a problem. Kurzweil predicts things which are unlikely or perhaps impossible as having probabilities near 100%.

cpeterso 15y ago

Here is a rough transcript of a Long Now Foundation talk by SF author Vernor Vinge (and coiner of the term "singularity") entitled "What If the Singularity Does NOT Happen?". He sees: * Scenario 1: A Return to MADness (nuclear war) * Scenario 2: The Golden Age (peace and prosperity) * Scenario 3: The Wheel of Time (catastrophic natural disaster) http://www-rohan.sdsu.edu/faculty/vinge/longnow/ http://www-rohan.sdsu.edu/faculty/vinge/longnow/

super_mario 15y ago

String AI is not a hardware problem. It's not a matter of lack of computational power. It is a software and modeling problem. If you had an AI algorithm and a model, you could still run it on any Turning machine. It would just take a lot longer (perhaps years or decades or more) to compute a single thought on current hardware instead of real time or faster than real time on some super fast future hardware. There are people (like Roger Penrose) who argued that intelligence and consciousness are not computational in nature (and hence no algorithm can be conscious). Penrose goes all the way down to quantum mechanical effects in the brain. I have not really followed developments on this and where Penrose's argument currently stands.

spot 15y ago
Nobody takes him seriously. He is just a Christian apologist who wraps it up with quantum who haa.