14 ms·

The Singularity is not coming (2012)

55 points by fchollet 12y ago

aet 12y ago
http://www.simulation-argument.com/simulation.html http://www.simulation-argument.com/simulation.html
briantakita 12y ago
> We didn't "get" physics 50 times better in 1990 compared to 1940 We don't have to. The fundamental rules are discovered slowly. The frontier has been expanded & there's plenty of room in the spaces between. We can now utilize these rules to create & build. The rules of physics are being assimilated by the rest of the culture & it's properties are being utilized more than ever. This trend is accelerating. There is plenty of room for exploration, "cheap" growth (scaling), & turning over new leaves in biotech, tech, biology, medicine, robotics, ai, software development patterns, etc. The nice thing about the globalization of the internet is there's an unprecedented amount, quality, & interconnections of minds to advance our global knowledge. We are also undergoing cultural change that fosters collaboration and decentralized intelligence. There's also a lower barrier of entry for one to contribute to global knowledge (i.e. you don't need a PhD nor do you have to be a white male from a privileged background).
eudox 12y ago
>After all the total quantity of intelligence and hard work available around is millionfold what you can provide –you're just a drop of water in the ocean. Rather use your imagination, the one thing that makes you a beautiful unique snowflake. Intelligence and hard work should be merely at the service of our imagination. Think outside of the box. Break out. Shake the axioms. Given this sort of nonsense, it's no surprise the author is (Probably, since they all are) some kind of vitalist. He probably doesn't think brain emulations are people!
- briantakita 12y ago
  Why is it nonsense?
  - api 12y ago
    There are very, very few people who are capable of arguing against an idea on its merit if that idea clashes strongly with their worldview. Usually the cognitive dissonance of a differing worldview eclipses understanding to the point that anything with a wildly different context is just "insane."
  - astrodust 12y ago
    "Intelligence" as we have come to understand it is not some magical thing, not some exotic element that's impossible to fabricate. In fact, it's probably something mundane and insultingly simple when we get to the core of it. Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be. We already have enough hardware to fake intelligence, to make a believable Turing Test candidate that could win, but we've yet to figure out how. That much will become clear in the coming decades, guaranteed. If you look at the progression in computer chess programs, where they were pathetic for the longest time until things came together and the performance of them rose exponentially, matching and then eclipsing grandmasters, it's inevitable that the same pattern will play out in the artificial intelligence space.
    - briantakita 12y ago
      > "Intelligence" as we have come to understand it is not some magical thing I agree that intelligence can be replicated. My preferred definition of intelligence is broad. Intelligence is a system that has i/o & a feedback cycle. Thus, I think all matter & concepts have intelligence. Of course, there are higher forms of intelligence, such as human intelligence (which is not limited to the brain as we have a complete nervous system & more generally, autonomous cells). I still think it's magical, however. Even if we understand all of the rules & procedures, the experience of it all coming together fills me with awe. It's like playing a game where you understand the rules, yet you can achieve zen & a higher consciousness when playing. Another example is Fractals. The driving equation is simple, yet infinite complexity & beauty is created. > Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be. Animals & possibly lower life forms have evolved emotions. It's an important part of what drives us to do whatever we do. I would not be surprised if AI will need emotions to reach higher intelligence. I have also been driven to honor all life & intelligence. I see the interconnections within the system of existence & aspire to respect all of it's parts. When I hold such a mindset, I'm at my best.
- jsun 12y ago
  Well he said if you kept giving the AI exponentially more computing resources it would get linear growth. I think it's fairly obvious that providing 1 computer exponentially more resources isn't much different from creating exponentially more brain simulations...
- dang 12y ago
  This is not a good Hacker News comment. It's snarky, resorts to name-calling, and presents no substantive view. Thus its signal/noise ratio is poor. Several people have observed that HN threads are sensitive to initial conditions. Threads seem to go more off the rails when a dismissive one-liner becomes the top comment early. I'm not sure how one could test this, but it does strike me as valid. All: Please don't post dismissive one-liners. They don't just lower the signal/noise ratio a little—they add a significant risk of lowering it a lot. It's better to post nothing. When you do comment, please do so "in the spirit of colleagues cooperating in good faith to figure out the truth about something". That's a pg phrase which captures the desired quality of HN so well that we intend to add it to the site guidelines.
  - VintageCool 12y ago
    > I would like to simply argue that scientific progress is in fact linear I stopped reading right there. The model that the author is looking for to express his ideas is "logistic growth", or "S curves".
mrrrgn 12y ago
This author rails about how scientific progress isn't exponential; but that's not even relevant. The idea of accelerating returns was only ever meant to apply to information technologies. Though they are related on various levels, fundamentally, science != technology. Only when scientific processes are converted into information technologies do they start becoming exponential (as was the case with genome sequencing).
gwern 12y ago
> That this deadline would arrive just in time to save the proponents of the Singularity from old age is just a weird coincidence that ought to be ignored. It's also not a true coincidence: http://intelligence.org/files/PredictingAI.pdf http://intelligence.org/files/PredictingAI.pdf It look like it was just an artifact of the 10 or so predictions Kelly compiled.
AndrewKemendo 12y ago
>But imagination is a fucking superpower. Indeed! Why the author decouples imagination from intelligence is odd to me, especially as he claims to be an AI researcher. Artificial Imagination and the like has been researched basically since the beginning of AI research. That is not to say that imagination is a solved problem within AI, far from it, but it is as covered in theory (which this article is about after all) as any other AI problem. >Rather use your imagination, the one thing that makes you a beautiful unique snowflake. My guess is that the author thinks there is something special or non-material about being human.
- fchollet 12y ago
  To clarify this point: at the time I meant "imagination" in a human context, as the ability to create new thought systems to operate in, as opposed to doing research within an existing thought framework (typically inherited from your thesis advisor, etc.). This is analogous to creating a new market with a completely new type of product vs. entering an existing market, for companies. No, I do not believe there is anything special about human intelligence. Artificial consciousness is among my research interests. As for artificial imagination, it is not exactly "covered in theory" by our current understanding of AI or the human mind --the only kind of AI we know are AIs that operate within rule systems determined by humans. Let's take a simple example, the most general AI we know of, genetic algorithms: if you launch a genetic algorithm to find a solution to some problem, it will merely search through a search space you will have to determine yourself. Artificial imagination would the capability to expand or modify that search space based on previous findings...
  - AndrewKemendo 12y ago
    >at the time I meant "imagination" in a human context, as the ability to create new thought systems to operate in, as opposed to doing research within an existing thought framework (typically inherited from your thesis advisor, etc.). I think that is a good enough definition for imagination universally - and I would still not differentiate. How we do it is not known, but it is not unknowable. At least I don't think so. So stating that it will never happen is a bridge too far and I think built on shaky rafters. >Artificial imagination would the capability to expand or modify that search space based on previous findings... Check out some of the evolved antennas that came out of AIAA a few years ago. Would that not meet this definition? There is also some work with computer vision that modify prior outputs for new inputs in searches, so that would meet the definition you give. That said, I won't argue that we have good examples of working imaginations in practice or research - though I bet there are some that I haven't seen - but at least there are papers on the theory. I even have a few of my own ideas that I just haven't put on paper.
tedks 12y ago
>Intelligence is just a skill, more precisely a meta-skill that defines your ability to get new skills. But imagination is a fucking superpower. Isn't this the core inconsistency? Intelligence explosion advocates obviously believe that imagination is not an irreducible superpower, but part of the same meta-skill that defines your ability to get new skills. If you assume that paradigm shifts can be farmed at the same rate of discoveries, it seems like you'd get explosive effects again. This also assumes that fields don't proliferate; i.e., that a single field couldn't produce multiple paradigm shifts that each open up a space of new low-hanging fruit. Still, good treatment of the question. Much more convincing than a lot of the explanations for.
izzydata 12y ago
But if science is becoming exponentially harder and we are making linear scientific progress, then wouldn't that imply that scientific endeavors are increasing exponentially?
- fchollet 12y ago
  Empirically, they are. As a civilization we are investing exponentially more in science over time, which is mentioned in the article.
  - softatlas 12y ago
    Here I say a better model for the Universe is not matter but information, an information architecture of the universe. Matter is buggy, less elegant, event-driven (whereas emergence (acausality) might help). We're investigating exponentially more if we remain vigilant in preserving the materialist's conceptual scheme.
ThomPete 12y ago
I am always amazed that the same people who are proponents of the scientific explanation (simple immaterial matter turning into self-aware human beings without any designer. At the same time talk about a very small part of evolution as something unique and magical as if it required a designer. Why is it so hard to believe that computers created and designed by human beings who themselves didn't even have a designer, can become self-aware? We also see this in the quantum physics field where some people insist on the "hidden variable" theory. Simply don't get it.
mitchmindtree 12y ago
> ...due to the "exponential" rate of progress of science. I don't remember any great futurist referring to the "exponential rate of progress in science", but rather the accelerating advances in regards to information processing technology specifically.
tim333 12y ago
Most of the article seems to be attacking a straw man argument saying science does not progress exponentially hence no singularity. The arguments for the singularity do not generally involve the advance of science. They are based largely on computing getting faster which is down to technology and economics and does not necessarily require any new scientific discoveries although there may well be some. The basic argument is that is computers keep getting more powerful in the way they have for the last century (see https://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg https://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg) then at some point they will have processing power greater than a human brain and assuming software development keeps up that they will be able to exhibit human level intelligence. The hardware getting up to human equivalent levels seems a near certainty and software development while less certain is coming along in a promising manner with Google's self driving cars and the like.
- austinz 12y ago
  1. Moore's Law is not an ironclad law, it's an observation about improvements in integrated circuit transistor density. Given that there are not that many node shrinks remaining before we hit the fundamental limits of our current technology, scientific discoveries will be necessary to continue increasing transistor density into the far future. It is absolutely not a given that Moore's Law has to continue until the Singularity can occur, although it isn't impossible that it'll be around for several more decades either. 2. The most powerful hardware in the world is useless without software. Improvements in AI theory don't follow Moore's law, and there is no reason they should. Maybe your super-fast computer can run machine learning algorithms on a huge dataset, but it's not clear machine learning alone is sufficient to produce something on the scale of the Singularity. Anyways, strong AI is a very difficult problem, but in no case I know of has the limiting factor ever been the power of the hardware (except maybe for neuronal simulations trying to emulate the activity of a human brain).
  - tim333 12y ago
    1 - you don't need Moore's law, just the observation that computers have got steadily faster for decades and will probably continue. 2 >I know of has the limiting factor ever been the power of the hardware One example would be computer chess: "In 1990, futurist Ray Kurzweil predicted that a computer would win a game of chess against a human by 1998. His prediction came true on May 11, 1997, when IBM’s Deep Blue defeated the world chess champion Garry Kasparov." that prediction was done pretty much entirely on figuring the computational power needed and projecting when we'd get there. Not really rocket science, just some back of the envelope calculations. His 2029 prediction for the turing test is the same stuff.
kawa 12y ago
> scientific progress has really been linear. We didn't "get" physics 50 times better in 1990 compared to 1940 It's very hard to quantify scientific progress. The main reason is that we first pick the "low hanging fruits" and advancements get progressively harder. Just think of particle physics: While the first steps could be done with basic lab bench equipment, today we need to build gigantic accelerators to push the boundaries even a little bit. The same is true in other areas: The first steps are comparatively easy, but the more you know the harder it gets to solve the remaining problems, because you already got the easy ones. So even if perceived progress is linear, considering the increasing difficulty we may have exponential scientific progress.
- andybalholm 12y ago
  That is the OP's main point, except turned around. He was measuring progress by results, not by effort expended, and saying that it takes exponential effort to reach linear results.
  - kawa 12y ago
    But considering that the total amount of scientific knowledge is limited, linear progress will find "everything" in a manageable amount of time. Of course we don't know how big the percentage of our total knowledge is, but it's quite possible that we are already getting close to know "everything". The singularity doesn't promise certain invention (for example it's possible that there is no and will never be a way to go faster than the speed of light) it only promises exponential growth which will bring the "end of science" into the foreseeable future. So if we found maybe 80% of all physics in the last 200 years, with linear progress we would reach the end of physics in 40 years.
droopyEyelids 12y ago
I'm of the opinion that the singularity has been here for a while. There have been self-modifying, self-improving biorobots terraforming the Earth to augment themselves physically and mentally since before my grandparents were born.
DennisP 12y ago
The model assumes that researchers can predict the impact and effort of their discoveries. If that's the case, then of course progress slows down as earlier researchers pick the low-hanging tasty fruit. But if we really knew what the research would show, we wouldn't have to do the research. A smaller point is that it's not really true that progress in algorithms is slowing down. According to a study a couple years ago, in the time it took hardware to improve by a factor of 1000, algorithms improved by a factor of 43,000, for a total speedup of 43 million. http://bits.blogs.nytimes.com/2011/03/07/software-progress-beats-moores-law/?_php=true&_type=blogs&_r=0 http://bits.blogs.nytimes.com/2011/03/07/software-progress-b...
gerbal 12y ago
I've long thought that among the chief promises of a self-improving AI is not that it will exponentially make new discoveries in Science. But that it will be able to synthesize and combine existing knowledge in a manner too complex for most humans. The problem with current knowledge is that humans just aren't capable of handling more than a tiny fraction of it. A functional AI (even of the weaker variety) is capable of handling orders of magnitude more information and arriving at novel combinations of existing knowledge. Part of the reason the impact-per-researcher has fallen so much is that there is simply so much more to know and consider than there was a century ago. And Computers are really, really good at dealing with volumes of information that would overwhelm a person. And while a self-improving AI may someday slow down it's rate of self improvement, in the short term its rate of self-improvement would appear exponential as it groks scales of knowledge humans can't cope with.
- fixermark 12y ago
  This is a pretty valuable insight. Even without some kind of innovation to intelligence itself, simply taking the human cognitive process and mapping it onto a strata with several factors more bandwidth (an expectation that isn't unrealistic, given the current implementation is low-speed potential flows and chemical squirting) could significantly push science forward. This one is an easy bet, since we're already doing this; modern researchers (even the professionals ;) ) use Google and more specialized search tools, which are little more than machines that can read, digest, and correlate symbols at speeds much, _much_ faster than a human (though in a far more narrowly-scoped domain than a human).
ThrustVectoring 12y ago
It's really a question of how quickly it gets difficult to improve an arbitrary AI algorithm, in terms of the problem-solving capabilities unlocked by having an AI algorithm of that strength. Once you have given beliefs about these quantities, you've got a belief about the result of AI research. I think that the AI problem gets hard really quickly. It's less clear how much problem solving ability having an AI algorithm gives you. There's three ways I'd use an AI to do AI research better - AI(improve_AI_algorithm), AI(get_more_computing_power), and AI(improve_AI_research_process). The first is pretty much a run-once sort of thing for a fixed gain. The second is a lot less clear to me - I'm not really familiar with the constraints of chip design, and convincing people to put more resources into AI research hits diminishing returns. The third is much more promising at the current level of AI sophistication, for doing things like hire/fire decisions, scheduling and resource allocation, etc. Anyhow, my model is roughly a sideways S, with low-hanging fruit getting more reachable at first until it gets exhausted in fairly short order. There's a rather broad distribution of where it ends up afterwards and how quickly the easier improvements get found. The low end wouldn't get classified as a singularity, but the high end would.
smky80 12y ago
I came to the conclusion a while back that humans are just fundamentally religious. Even most people who don't believe in "gods" seem to end up largely clinging to some over-simplified, basically religious model of the world. As examples: Modern liberalism = New Testament Christian ethics with the government replacing god. "Universe is a simulation" = Gnostic/platonic belief system (our universe as a sub-creation created by a flawed sub-creator), with a technological guise. Belief in the Singularity probably appeals to some people in the same way that an imminent Rapture or Second Coming appeals to others. Especially with the uploading consciousness to computers and living eternally ...
- fixermark 12y ago
  "There's nothing new under the sun" ... except for all the new stuff, of course. The fact that Da Vinci dreamed of flying machines doesn't diminish the significance of the actual invention of the airplane or modern commercial air travel.
markpundmann 12y ago
I have read a lot of thoughts on the supposed "singularity", and something I still have yet to see is for someone to realize that every person who believes in the singularity disregards economics. They all seem to think that this intelligent machine will improve it's algorithm or find ways to make smaller, more efficient, and better designed chips. On the software side, there is only so much one can do to maximize the intelligent aspect of the algorithm. At one point or another, the program will be maximally efficient at solving whatever problem the algorithm is designed to solve. On the hardware side, the singularitists sometimes argue that the program will design every better, bigger, and more efficient chip designs as the news chips would be used to build even better chips. However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve. I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for. Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy.
- edanm 12y ago
  "I have read a lot of thoughts on the supposed "singularity", and something I still have yet to see is for someone to realize that every person who believes in the singularity disregards economics." I don't think that's accurate. LessWrong, one of the largest online communities talking about (friendly) AI and the Intelligence Explosion, was founded by Eliezer Yudkowky. He was a co-blogger with Economist Robin Hanson, also a contributor to LessWrong. That's not to say that Robin Hanson agrees with all the views, but they are aware of economics. In fact, Eliezer Yudkowsky wrote a paper specifically on the economics of the Intelligence Explosion (https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf). He also regularly writes about economic matters. As to your specific points: "However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve." "I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for." For a very advanced intelligence, creating its own resources isn't a problem. E.g. humans. We make our own resources. The AI doesn't have to rely on humans necessarily. Maybe only in the beginning. "Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy." That sounds reasonable. But the limit that a superior intelligence reaches can be many orders of magnitude above where humans are on the spectrum. E.g., things that are overly complex for monkeys have been achieved by humans, despite the fact that they are much more complex.
sergiosgc 12y ago
Interesting text, but flawed in many ways. A couple of the more important flaws: - Exponentials often look linear in short x axis observations. Science at t-50 and t-100 may be too short an observation period. Looking at the past 3000 years the exponential characteristic is quite obvious. - Measuring scientific progress as defined in the paper is obviously simplistic. You don't have a limited set of equal valued discoveries to be found randomly. You have a limited set of increasingly valuable[a] discoveries, with dependencies, to be found. The value rate of discoveries is ignored in the text, but is relevant. Say, observing the periodicity of star movement was important, but the gravitic theory built on that to produce even more valuable information. One allows you to count time, the other is fundamental in travelling to the moon. One can be discovered by an uneducated farmer, the other requires calculus knowledge. [a] The definition of value is itself difficult. Impact value, as in practical application value, is a metric. Foundational value, as in which doors are now open for research is another. The utility function is probably a mix of both. I'd wager more recent discoveries are better than older ones in both counts (antibiotics vs immune system programming for example). Do not get fooled by the relative impact (you get a division by zero error in the first scientific discovery ever made)
Houshalter 12y ago
Exponential self-improvement may or may not happen. I'm of the opinion that it will. The first AI is likely to be vastly suboptimal and just thrown together. The very first thing that actually works, but not the best possible thing (same is true of humans.) Therefore there are likely to be lots of optimizations and improvements that can be made - low hanging fruit. So lots of progress will happen very quickly until it starts to approach the actual limits. But those limits don't have to be anything near humans. Transistors are many thousands of times faster than neurons. It will have massively superior serial processing power, which is good at doing optimization tasks (i.e. the kind of tasks we want it to do or that will be useful in improving itself.) They are general purpose and can be reprogrammed for specific tasks, unlike our brains. And we can create infinitely many instances of them to work on every conceivable problem in parallel, and share resources (human brains are quite limited at that.) There is Plenty of Room Above Us: http://intelligenceexplosion.com/2011/plenty-of-room-above-us/ http://intelligenceexplosion.com/2011/plenty-of-room-above-u...
edanm 12y ago
Not sure I understand the gist of the argument. First, let's separate "Intelligence Explosion" from "Singularity" as to many people those terms mean different things. A lot of the people talking about an "Intelligence Explosion" do not agree that the progress of science has been getting exponentially faster. What they are talking about, is that once we create an AI that can improve its own intelligence in a way that humans can't, then the improved intelligence will know how to further improve its own intelligence, and so on. Even if there is a limit to the number of discoveries possible in any field, if you're affecting the base rate at which you're making new discoveries, you can (potentially) continue making progress. More importantly, all of this is besides the point - it could easily be that all the "low hanging fruit" in improving intelligence can make an intelligence that is vastly more intelligent than a human, at which point the so-called "singularity" will have been reached. Or in the words of the "Intelligence Explosion" camp, we will have reached a world where we are no longer the dominant intelligence, possibly by a very wide gap. I see plenty of reason to think that human intelligence is not a good benchmark for "the maximum intelligence you can get by picking all the low-hanging fruit", since there are many, many easy improvements we can imagine that will almost certainly improve human intelligence (e.g., perfect memory, ability to partially reprogram yourself to do stuff, like for example not to have "bad" desires like eating unhealthily, etc.).
fthssht 12y ago
The biggest flaw was the articles strawman that science laws as in physics knowledge as opposed to information technologies like millions of instructions per second or memory capacity are exponentially increasing. Funny hiw he arrogantly "objects your honor" on science exponentially ibcreasing while missing this.

55 points by fchollet 12y ago

aet 12y ago

http://www.simulation-argument.com/simulation.html http://www.simulation-argument.com/simulation.html

> We didn't "get" physics 50 times better in 1990 compared to 1940 We don't have to. The fundamental rules are discovered slowly. The frontier has been expanded & there's plenty of room in the spaces between. We can now utilize these rules to create & build. The rules of physics are being assimilated by the rest of the culture & it's properties are being utilized more than ever. This trend is accelerating. There is plenty of room for exploration, "cheap" growth (scaling), & turning over new leaves in biotech, tech, biology, medicine, robotics, ai, software development patterns, etc. The nice thing about the globalization of the internet is there's an unprecedented amount, quality, & interconnections of minds to advance our global knowledge. We are also undergoing cultural change that fosters collaboration and decentralized intelligence. There's also a lower barrier of entry for one to contribute to global knowledge (i.e. you don't need a PhD nor do you have to be a white male from a privileged background).

eudox 12y ago

>After all the total quantity of intelligence and hard work available around is millionfold what you can provide –you're just a drop of water in the ocean. Rather use your imagination, the one thing that makes you a beautiful unique snowflake. Intelligence and hard work should be merely at the service of our imagination. Think outside of the box. Break out. Shake the axioms. Given this sort of nonsense, it's no surprise the author is (Probably, since they all are) some kind of vitalist. He probably doesn't think brain emulations are people!

briantakita 12y ago
Why is it nonsense?
- api 12y ago
  There are very, very few people who are capable of arguing against an idea on its merit if that idea clashes strongly with their worldview. Usually the cognitive dissonance of a differing worldview eclipses understanding to the point that anything with a wildly different context is just "insane."
- astrodust 12y ago
  "Intelligence" as we have come to understand it is not some magical thing, not some exotic element that's impossible to fabricate. In fact, it's probably something mundane and insultingly simple when we get to the core of it. Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be. We already have enough hardware to fake intelligence, to make a believable Turing Test candidate that could win, but we've yet to figure out how. That much will become clear in the coming decades, guaranteed. If you look at the progression in computer chess programs, where they were pathetic for the longest time until things came together and the performance of them rose exponentially, matching and then eclipsing grandmasters, it's inevitable that the same pattern will play out in the artificial intelligence space.
  - briantakita 12y ago
    > "Intelligence" as we have come to understand it is not some magical thing I agree that intelligence can be replicated. My preferred definition of intelligence is broad. Intelligence is a system that has i/o & a feedback cycle. Thus, I think all matter & concepts have intelligence. Of course, there are higher forms of intelligence, such as human intelligence (which is not limited to the brain as we have a complete nervous system & more generally, autonomous cells). I still think it's magical, however. Even if we understand all of the rules & procedures, the experience of it all coming together fills me with awe. It's like playing a game where you understand the rules, yet you can achieve zen & a higher consciousness when playing. Another example is Fractals. The driving equation is simple, yet infinite complexity & beauty is created. > Reproducing intelligence is mostly a matter of finding the correct formula, the right structures, the right technology. After that it will be boring, ordinary, even disposable. This doesn't sit well with some people who reject that on an emotional level even if they can't figure out any concrete reason why. It just can't be. Animals & possibly lower life forms have evolved emotions. It's an important part of what drives us to do whatever we do. I would not be surprised if AI will need emotions to reach higher intelligence. I have also been driven to honor all life & intelligence. I see the interconnections within the system of existence & aspire to respect all of it's parts. When I hold such a mindset, I'm at my best.
jsun 12y ago
Well he said if you kept giving the AI exponentially more computing resources it would get linear growth. I think it's fairly obvious that providing 1 computer exponentially more resources isn't much different from creating exponentially more brain simulations...
dang 12y ago
This is not a good Hacker News comment. It's snarky, resorts to name-calling, and presents no substantive view. Thus its signal/noise ratio is poor. Several people have observed that HN threads are sensitive to initial conditions. Threads seem to go more off the rails when a dismissive one-liner becomes the top comment early. I'm not sure how one could test this, but it does strike me as valid. All: Please don't post dismissive one-liners. They don't just lower the signal/noise ratio a little—they add a significant risk of lowering it a lot. It's better to post nothing. When you do comment, please do so "in the spirit of colleagues cooperating in good faith to figure out the truth about something". That's a pg phrase which captures the desired quality of HN so well that we intend to add it to the site guidelines.
- VintageCool 12y ago
  > I would like to simply argue that scientific progress is in fact linear I stopped reading right there. The model that the author is looking for to express his ideas is "logistic growth", or "S curves".

mrrrgn 12y ago

This author rails about how scientific progress isn't exponential; but that's not even relevant. The idea of accelerating returns was only ever meant to apply to information technologies. Though they are related on various levels, fundamentally, science != technology. Only when scientific processes are converted into information technologies do they start becoming exponential (as was the case with genome sequencing).

gwern 12y ago

> That this deadline would arrive just in time to save the proponents of the Singularity from old age is just a weird coincidence that ought to be ignored. It's also not a true coincidence: http://intelligence.org/files/PredictingAI.pdf http://intelligence.org/files/PredictingAI.pdf It look like it was just an artifact of the 10 or so predictions Kelly compiled.

AndrewKemendo 12y ago

>But imagination is a fucking superpower. Indeed! Why the author decouples imagination from intelligence is odd to me, especially as he claims to be an AI researcher. Artificial Imagination and the like has been researched basically since the beginning of AI research. That is not to say that imagination is a solved problem within AI, far from it, but it is as covered in theory (which this article is about after all) as any other AI problem. >Rather use your imagination, the one thing that makes you a beautiful unique snowflake. My guess is that the author thinks there is something special or non-material about being human.

fchollet 12y ago
To clarify this point: at the time I meant "imagination" in a human context, as the ability to create new thought systems to operate in, as opposed to doing research within an existing thought framework (typically inherited from your thesis advisor, etc.). This is analogous to creating a new market with a completely new type of product vs. entering an existing market, for companies. No, I do not believe there is anything special about human intelligence. Artificial consciousness is among my research interests. As for artificial imagination, it is not exactly "covered in theory" by our current understanding of AI or the human mind --the only kind of AI we know are AIs that operate within rule systems determined by humans. Let's take a simple example, the most general AI we know of, genetic algorithms: if you launch a genetic algorithm to find a solution to some problem, it will merely search through a search space you will have to determine yourself. Artificial imagination would the capability to expand or modify that search space based on previous findings...
- AndrewKemendo 12y ago
  >at the time I meant "imagination" in a human context, as the ability to create new thought systems to operate in, as opposed to doing research within an existing thought framework (typically inherited from your thesis advisor, etc.). I think that is a good enough definition for imagination universally - and I would still not differentiate. How we do it is not known, but it is not unknowable. At least I don't think so. So stating that it will never happen is a bridge too far and I think built on shaky rafters. >Artificial imagination would the capability to expand or modify that search space based on previous findings... Check out some of the evolved antennas that came out of AIAA a few years ago. Would that not meet this definition? There is also some work with computer vision that modify prior outputs for new inputs in searches, so that would meet the definition you give. That said, I won't argue that we have good examples of working imaginations in practice or research - though I bet there are some that I haven't seen - but at least there are papers on the theory. I even have a few of my own ideas that I just haven't put on paper.

tedks 12y ago

>Intelligence is just a skill, more precisely a meta-skill that defines your ability to get new skills. But imagination is a fucking superpower. Isn't this the core inconsistency? Intelligence explosion advocates obviously believe that imagination is not an irreducible superpower, but part of the same meta-skill that defines your ability to get new skills. If you assume that paradigm shifts can be farmed at the same rate of discoveries, it seems like you'd get explosive effects again. This also assumes that fields don't proliferate; i.e., that a single field couldn't produce multiple paradigm shifts that each open up a space of new low-hanging fruit. Still, good treatment of the question. Much more convincing than a lot of the explanations for.

izzydata 12y ago

But if science is becoming exponentially harder and we are making linear scientific progress, then wouldn't that imply that scientific endeavors are increasing exponentially?

fchollet 12y ago
Empirically, they are. As a civilization we are investing exponentially more in science over time, which is mentioned in the article.
- softatlas 12y ago
  Here I say a better model for the Universe is not matter but information, an information architecture of the universe. Matter is buggy, less elegant, event-driven (whereas emergence (acausality) might help). We're investigating exponentially more if we remain vigilant in preserving the materialist's conceptual scheme.

ThomPete 12y ago

I am always amazed that the same people who are proponents of the scientific explanation (simple immaterial matter turning into self-aware human beings without any designer. At the same time talk about a very small part of evolution as something unique and magical as if it required a designer. Why is it so hard to believe that computers created and designed by human beings who themselves didn't even have a designer, can become self-aware? We also see this in the quantum physics field where some people insist on the "hidden variable" theory. Simply don't get it.

mitchmindtree 12y ago

> ...due to the "exponential" rate of progress of science. I don't remember any great futurist referring to the "exponential rate of progress in science", but rather the accelerating advances in regards to information processing technology specifically.

tim333 12y ago

Most of the article seems to be attacking a straw man argument saying science does not progress exponentially hence no singularity. The arguments for the singularity do not generally involve the advance of science. They are based largely on computing getting faster which is down to technology and economics and does not necessarily require any new scientific discoveries although there may well be some. The basic argument is that is computers keep getting more powerful in the way they have for the last century (see https://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg https://en.wikipedia.org/wiki/File:PPTMooresLawai.jpg) then at some point they will have processing power greater than a human brain and assuming software development keeps up that they will be able to exhibit human level intelligence. The hardware getting up to human equivalent levels seems a near certainty and software development while less certain is coming along in a promising manner with Google's self driving cars and the like.

austinz 12y ago
1. Moore's Law is not an ironclad law, it's an observation about improvements in integrated circuit transistor density. Given that there are not that many node shrinks remaining before we hit the fundamental limits of our current technology, scientific discoveries will be necessary to continue increasing transistor density into the far future. It is absolutely not a given that Moore's Law has to continue until the Singularity can occur, although it isn't impossible that it'll be around for several more decades either. 2. The most powerful hardware in the world is useless without software. Improvements in AI theory don't follow Moore's law, and there is no reason they should. Maybe your super-fast computer can run machine learning algorithms on a huge dataset, but it's not clear machine learning alone is sufficient to produce something on the scale of the Singularity. Anyways, strong AI is a very difficult problem, but in no case I know of has the limiting factor ever been the power of the hardware (except maybe for neuronal simulations trying to emulate the activity of a human brain).
- tim333 12y ago
  1 - you don't need Moore's law, just the observation that computers have got steadily faster for decades and will probably continue. 2 >I know of has the limiting factor ever been the power of the hardware One example would be computer chess: "In 1990, futurist Ray Kurzweil predicted that a computer would win a game of chess against a human by 1998. His prediction came true on May 11, 1997, when IBM’s Deep Blue defeated the world chess champion Garry Kasparov." that prediction was done pretty much entirely on figuring the computational power needed and projecting when we'd get there. Not really rocket science, just some back of the envelope calculations. His 2029 prediction for the turing test is the same stuff.

kawa 12y ago

> scientific progress has really been linear. We didn't "get" physics 50 times better in 1990 compared to 1940 It's very hard to quantify scientific progress. The main reason is that we first pick the "low hanging fruits" and advancements get progressively harder. Just think of particle physics: While the first steps could be done with basic lab bench equipment, today we need to build gigantic accelerators to push the boundaries even a little bit. The same is true in other areas: The first steps are comparatively easy, but the more you know the harder it gets to solve the remaining problems, because you already got the easy ones. So even if perceived progress is linear, considering the increasing difficulty we may have exponential scientific progress.

andybalholm 12y ago
That is the OP's main point, except turned around. He was measuring progress by results, not by effort expended, and saying that it takes exponential effort to reach linear results.
- kawa 12y ago
  But considering that the total amount of scientific knowledge is limited, linear progress will find "everything" in a manageable amount of time. Of course we don't know how big the percentage of our total knowledge is, but it's quite possible that we are already getting close to know "everything". The singularity doesn't promise certain invention (for example it's possible that there is no and will never be a way to go faster than the speed of light) it only promises exponential growth which will bring the "end of science" into the foreseeable future. So if we found maybe 80% of all physics in the last 200 years, with linear progress we would reach the end of physics in 40 years.

droopyEyelids 12y ago

I'm of the opinion that the singularity has been here for a while. There have been self-modifying, self-improving biorobots terraforming the Earth to augment themselves physically and mentally since before my grandparents were born.

DennisP 12y ago

The model assumes that researchers can predict the impact and effort of their discoveries. If that's the case, then of course progress slows down as earlier researchers pick the low-hanging tasty fruit. But if we really knew what the research would show, we wouldn't have to do the research. A smaller point is that it's not really true that progress in algorithms is slowing down. According to a study a couple years ago, in the time it took hardware to improve by a factor of 1000, algorithms improved by a factor of 43,000, for a total speedup of 43 million. http://bits.blogs.nytimes.com/2011/03/07/software-progress-beats-moores-law/?_php=true&_type=blogs&_r=0 http://bits.blogs.nytimes.com/2011/03/07/software-progress-b...

gerbal 12y ago

I've long thought that among the chief promises of a self-improving AI is not that it will exponentially make new discoveries in Science. But that it will be able to synthesize and combine existing knowledge in a manner too complex for most humans. The problem with current knowledge is that humans just aren't capable of handling more than a tiny fraction of it. A functional AI (even of the weaker variety) is capable of handling orders of magnitude more information and arriving at novel combinations of existing knowledge. Part of the reason the impact-per-researcher has fallen so much is that there is simply so much more to know and consider than there was a century ago. And Computers are really, really good at dealing with volumes of information that would overwhelm a person. And while a self-improving AI may someday slow down it's rate of self improvement, in the short term its rate of self-improvement would appear exponential as it groks scales of knowledge humans can't cope with.

fixermark 12y ago
This is a pretty valuable insight. Even without some kind of innovation to intelligence itself, simply taking the human cognitive process and mapping it onto a strata with several factors more bandwidth (an expectation that isn't unrealistic, given the current implementation is low-speed potential flows and chemical squirting) could significantly push science forward. This one is an easy bet, since we're already doing this; modern researchers (even the professionals ;) ) use Google and more specialized search tools, which are little more than machines that can read, digest, and correlate symbols at speeds much, _much_ faster than a human (though in a far more narrowly-scoped domain than a human).

ThrustVectoring 12y ago

It's really a question of how quickly it gets difficult to improve an arbitrary AI algorithm, in terms of the problem-solving capabilities unlocked by having an AI algorithm of that strength. Once you have given beliefs about these quantities, you've got a belief about the result of AI research. I think that the AI problem gets hard really quickly. It's less clear how much problem solving ability having an AI algorithm gives you. There's three ways I'd use an AI to do AI research better - AI(improve_AI_algorithm), AI(get_more_computing_power), and AI(improve_AI_research_process). The first is pretty much a run-once sort of thing for a fixed gain. The second is a lot less clear to me - I'm not really familiar with the constraints of chip design, and convincing people to put more resources into AI research hits diminishing returns. The third is much more promising at the current level of AI sophistication, for doing things like hire/fire decisions, scheduling and resource allocation, etc. Anyhow, my model is roughly a sideways S, with low-hanging fruit getting more reachable at first until it gets exhausted in fairly short order. There's a rather broad distribution of where it ends up afterwards and how quickly the easier improvements get found. The low end wouldn't get classified as a singularity, but the high end would.

smky80 12y ago

I came to the conclusion a while back that humans are just fundamentally religious. Even most people who don't believe in "gods" seem to end up largely clinging to some over-simplified, basically religious model of the world. As examples: Modern liberalism = New Testament Christian ethics with the government replacing god. "Universe is a simulation" = Gnostic/platonic belief system (our universe as a sub-creation created by a flawed sub-creator), with a technological guise. Belief in the Singularity probably appeals to some people in the same way that an imminent Rapture or Second Coming appeals to others. Especially with the uploading consciousness to computers and living eternally ...

fixermark 12y ago
"There's nothing new under the sun" ... except for all the new stuff, of course. The fact that Da Vinci dreamed of flying machines doesn't diminish the significance of the actual invention of the airplane or modern commercial air travel.

markpundmann 12y ago

I have read a lot of thoughts on the supposed "singularity", and something I still have yet to see is for someone to realize that every person who believes in the singularity disregards economics. They all seem to think that this intelligent machine will improve it's algorithm or find ways to make smaller, more efficient, and better designed chips. On the software side, there is only so much one can do to maximize the intelligent aspect of the algorithm. At one point or another, the program will be maximally efficient at solving whatever problem the algorithm is designed to solve. On the hardware side, the singularitists sometimes argue that the program will design every better, bigger, and more efficient chip designs as the news chips would be used to build even better chips. However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve. I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for. Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy.

edanm 12y ago
"I have read a lot of thoughts on the supposed "singularity", and something I still have yet to see is for someone to realize that every person who believes in the singularity disregards economics." I don't think that's accurate. LessWrong, one of the largest online communities talking about (friendly) AI and the Intelligence Explosion, was founded by Eliezer Yudkowky. He was a co-blogger with Economist Robin Hanson, also a contributor to LessWrong. That's not to say that Robin Hanson agrees with all the views, but they are aware of economics. In fact, Eliezer Yudkowsky wrote a paper specifically on the economics of the Intelligence Explosion (https://intelligence.org/files/IEM.pdf https://intelligence.org/files/IEM.pdf). He also regularly writes about economic matters. As to your specific points: "However, who is going to pay for the manufacturing, energy, and maintenance costs of this self improving machine? Eventually, this machine will have to quit trying to design more intelligent machines and start solving HUMAN PROBLEMS. Why? Cause humans will actually pay for the solutions to problems that this intelligent machine can solve." "I'm not saying that this type of self improving machine would never be built. I just think that we will end up splitting the line between devoting resources to improving the machine to encourage future growth, and making money by solving problems that people are willing to pay for." For a very advanced intelligence, creating its own resources isn't a problem. E.g. humans. We make our own resources. The AI doesn't have to rely on humans necessarily. Maybe only in the beginning. "Also note that as this machine gets more and more powerful, the problem of making itself better will get more and more complex, and as I think we can all agree on, complexity is our greatest enemy." That sounds reasonable. But the limit that a superior intelligence reaches can be many orders of magnitude above where humans are on the spectrum. E.g., things that are overly complex for monkeys have been achieved by humans, despite the fact that they are much more complex.

sergiosgc 12y ago

Interesting text, but flawed in many ways. A couple of the more important flaws: - Exponentials often look linear in short x axis observations. Science at t-50 and t-100 may be too short an observation period. Looking at the past 3000 years the exponential characteristic is quite obvious. - Measuring scientific progress as defined in the paper is obviously simplistic. You don't have a limited set of equal valued discoveries to be found randomly. You have a limited set of increasingly valuable[a] discoveries, with dependencies, to be found. The value rate of discoveries is ignored in the text, but is relevant. Say, observing the periodicity of star movement was important, but the gravitic theory built on that to produce even more valuable information. One allows you to count time, the other is fundamental in travelling to the moon. One can be discovered by an uneducated farmer, the other requires calculus knowledge. [a] The definition of value is itself difficult. Impact value, as in practical application value, is a metric. Foundational value, as in which doors are now open for research is another. The utility function is probably a mix of both. I'd wager more recent discoveries are better than older ones in both counts (antibiotics vs immune system programming for example). Do not get fooled by the relative impact (you get a division by zero error in the first scientific discovery ever made)

Houshalter 12y ago

Exponential self-improvement may or may not happen. I'm of the opinion that it will. The first AI is likely to be vastly suboptimal and just thrown together. The very first thing that actually works, but not the best possible thing (same is true of humans.) Therefore there are likely to be lots of optimizations and improvements that can be made - low hanging fruit. So lots of progress will happen very quickly until it starts to approach the actual limits. But those limits don't have to be anything near humans. Transistors are many thousands of times faster than neurons. It will have massively superior serial processing power, which is good at doing optimization tasks (i.e. the kind of tasks we want it to do or that will be useful in improving itself.) They are general purpose and can be reprogrammed for specific tasks, unlike our brains. And we can create infinitely many instances of them to work on every conceivable problem in parallel, and share resources (human brains are quite limited at that.) There is Plenty of Room Above Us: http://intelligenceexplosion.com/2011/plenty-of-room-above-us/ http://intelligenceexplosion.com/2011/plenty-of-room-above-u...

edanm 12y ago

Not sure I understand the gist of the argument. First, let's separate "Intelligence Explosion" from "Singularity" as to many people those terms mean different things. A lot of the people talking about an "Intelligence Explosion" do not agree that the progress of science has been getting exponentially faster. What they are talking about, is that once we create an AI that can improve its own intelligence in a way that humans can't, then the improved intelligence will know how to further improve its own intelligence, and so on. Even if there is a limit to the number of discoveries possible in any field, if you're affecting the base rate at which you're making new discoveries, you can (potentially) continue making progress. More importantly, all of this is besides the point - it could easily be that all the "low hanging fruit" in improving intelligence can make an intelligence that is vastly more intelligent than a human, at which point the so-called "singularity" will have been reached. Or in the words of the "Intelligence Explosion" camp, we will have reached a world where we are no longer the dominant intelligence, possibly by a very wide gap. I see plenty of reason to think that human intelligence is not a good benchmark for "the maximum intelligence you can get by picking all the low-hanging fruit", since there are many, many easy improvements we can imagine that will almost certainly improve human intelligence (e.g., perfect memory, ability to partially reprogram yourself to do stuff, like for example not to have "bad" desires like eating unhealthily, etc.).

fthssht 12y ago

The biggest flaw was the articles strawman that science laws as in physics knowledge as opposed to information technologies like millions of instructions per second or memory capacity are exponentially increasing. Funny hiw he arrogantly "objects your honor" on science exponentially ibcreasing while missing this.