11 ms·

I think in the future people will be perplexed by how sci-fi during this period was absolutely obsessed by the so-called singularity. Its a current fad, accepte

I think in the future people will be perplexed by how sci-fi during this period was absolutely obsessed by the so-called singularity. Its a current fad, accepted by many, but not backed up by anything substantial like a lot of fads from the past. Its a little embarrassing to read some sharp thinkers from the 1970s who fell into cheap counter-culture concepts like astrological signs or shammy Indian gurus or shoddy California-style Buddhism, that often has little or nothing to do with the Buddhavacana source materials. It really surprises me that we think there's a realistic path to the singularity and that we obsessed about it. Human-like AI is a dead field, or at least dead as far we can tell. We're not even sure how to build it because conceptually we don't even have a grasp on consciousness or cognition. Then there's the false assumption that we can get to this level before doing things like, say, solving all mental health issues and that it can magically be scaled to super-human speeds. For all we know cognition happens at a certain speed and any attempts past that are troublesome. Look at how easily we suffer from mania and other issues when our natural limiters go haywire. Singularity prediction is almost a cargo cult. Its weird that so many take it seriously.

jp555 12y ago
We are currently in the midst of a singularity of tools. Stone > Bronze > Steel > Silicon > Software > It's the "geek rapture" version that's more fantastic and extremely unlikely.
- _archon_ 12y ago
  This is a very interesting perspective. What do you think is the next step? I am asking for an educated guess, or even interested speculation; people in the middle of the bronze age likely didn't know that steel was the next step until after iron was extracted for the first time. If I had to guess, I would put the next phase at "integration" or "control." In that once we have functional, reliable libraries and virtual instruments, the logical next step is to integrate the high-level control of such into our physiologies. Alternatively, I could see "space" being an age we expand into, drawing our resources increasingly from space and broadening the scope of our potential that way. I'd like to see these happen in parallel if possible.
  - jp555 12y ago
    I'd say integration and control are N+1 optimization, not 0+1 invention. Things like new forms of quantum logic/algorithms, artificial atoms, metamaterials, even subvocal tools (moving us from graphical user interfaces towards thought-based user interfaces), are more 0+1 to me.
mikeash 12y ago
I'm pretty sure that the whole Singularity bit in this piece is sarcastic. That's kind of the whole theme of the article. I mean, this should tip you off pretty hard: "Well now, here's the thing: automating sarcasm detection is easy. It's so easy they teach it in first year computer science courses; it's an obvious application of AI."
VLM 12y ago
Singularity talk by fictional-ish writers is just a cheap search and replace job done by people who want to talk about their theories of a monotheistic God Version 2.0 and don't want historical baggage and automatic knee jerk reactions from the version 1.0 crowd. An even better theistic analogy for the singularity is the return of Jesus. Just wait till he comes and rights all the wrongs and banishes death and knows everything about everyone who ever lived and makes a paradise on earth for the true believers ... wait was I talking about the singularity or Jesus again? (note that I'm not (intentionally) making fun of either subject, but trying to focus on the crude search and replace aspect of the analogy)
- mistakenot 12y ago
  I've long thought that the singularity is the tech version of the rapture.
  - T-A 12y ago
    You may enjoy reading this then: http://craphound.com/rotn/download/ http://craphound.com/rotn/download/
  - davidgerard 12y ago
    Atheist who can't quite get rid of religious fear? Why reinvent Jesus when you can reinvent Yahweh! http://rationalwiki.org/wiki/Roko%27s_basilisk http://rationalwiki.org/wiki/Roko%27s_basilisk
aristus 12y ago
Whatever it is we're grasping toward, it may be "artificial intelligence" only in the sense that a car is an "iron horse".
- VLM 12y ago
  Trying to make a really good model bird held back the development of flight for centuries. Certainly, Julius Caesar or Crassus could have afforded a Roman era hot air balloon or hang glider or glider aircraft if this false analogy hadn't stood in the way of development. Performance would have been somewhat lower than modern material craft, but probably not as bad as you think. There were several decades of modest performance aircraft before titanium, synthetic fabrics, and carbon fiber. AI workers might be more successful in the long run if they focus on making a useful martian brain, not clone humans in silicon.
  - dragonwriter 12y ago
    > AI workers might be more successful in the long run if they focus on making a useful martian brain, not clone humans in silicon. AI has been a diverse field for a long time, with many different paths -- both with direct biological inspiration (often not human -- including, e.g., insect-inspired approaches) and without are common. Its not an exclusive "human or not human" modelling choice.
xaa 12y ago
You are right that you won't see many researchers publicly talking about human-level AI, or framing their research in terms of the search for general AI. But that doesn't mean the field is "dead". For example, I work in aging and, to a lesser extent, neuroscience. I have had many private conversations with researchers who will candidly say that the long-term goal of their work is lifespan extension or cognitive enhancement. Yet, for the purposes of grant and paper-writing, you'd never write such a thing; it would sound too speculative. I'm not in AI, but I strongly suspect the same thing is going on there. There are probably a lot of AI researchers who want to see human-level or higher general AI happen, and are working on it, but simply can't say so in public. We know it's possible, since we have an existence proof in the human brain.
- eli_gottlieb 12y ago
  >For example, I work in aging and, to a lesser extent, neuroscience. I have had many private conversations with researchers who will candidly say that the long-term goal of their work is lifespan extension or cognitive enhancement. Yet, for the purposes of grant and paper-writing, you'd never write such a thing; it would sound too speculative. I think it was Scientific American or another such magazine that ran a story, maybe a month and a half ago, in which they interviewed a whole bunch of real theoretical and computational neuroscientists about mind uploading. A not-that-small portion of these admitted that when they are in private, have had a drink or two, and are ready to admit things they would never say in a grant application, they are definitely trying to achieve whole-brain emulation. Hell, it's basically the implicit point of the Human Brain Project, or whatever those initiatives are called by the US and EU governments this past year. > You are right that you won't see many researchers publicly talking about human-level AI, or framing their research in terms of the search for general AI. But that doesn't mean the field is "dead". Indeed. What instead happened is that actual humanlike cognition has been steadily broken down into subproblems and each subproblem is being formalized and worked-on separately. There are a few papers on optimal utility-maximizing agents for general environments, but those tend to require "until the stars burn out" levels of FLOPs to work anyway. So actually, AI is going to work, but by the time it does, it won't be called AI. In fact, the researchers who'll create it would look at you funny for calling it AI. They do and will have lots of academic jargon used for specifying exactly what their algorithms do and how they do it, so much so that only when their algorithm kills all humans will anyone actually admit it was artificial intelligence ;-).
mej10 12y ago
Human-like AI isn't necessary for singularity events, especially the ones that end up killing all of us. We don't need to understand consciousness or cognition to create optimization processes that end up hurting (or killing) humanity. It surprises me how many people seem ignorant of this fact.
- droopyEyelids 12y ago
  What makes me so frustrated about this argument is that we already live in a world with thousands of optimization processes that end up hurting or killing humanity. Everything from the flu to tigers. Plus, all those 'processes' are evolving through selective pressure.
  - mej10 12y ago
    Corporations, governments, political systems, and other organizations are also optimization processes. Some of which are certainly harmful and also already hurt many people. And these are evolving through selective pressures as well. And not generally in the pro-humanity direction, more in the "preserve self at all costs" direction.
  - eli_gottlieb 12y ago
    >What makes me so frustrated about this argument is that we already live in a world with thousands of optimization processes that end up hurting or killing humanity. Yes, we've noticed. Those are evil, too.
- mcguire 12y ago
  I think you're misunderstanding the use of the term "singularity". It does not mean "X will happen, after which we won't be in charge", or "...after which we'll be dead". Instead, it means that something will become so intelligent that it will be different in kind, in somewhat the same way that we are so different from gravel that gravel cannot hope to understand our behavior. An optimization process that ends up killing all of us is bad, but it's not a singularity.
rayiner 12y ago
> Human-like AI is a dead field, or at least dead as far we can tell. We're not even sure how to build it because conceptually we don't even have a grasp on consciousness or cognition. The current optimism around A.I. is the result not of real advances in the field, but the very useful outcomes that arise from doing pattern matching and machine learning on big data sets. When people can type "weather today" into Google and get a little card that shows the weather in Philadelphia today, it's hard not to be impressed, not realizing that these tricks don't bring us closer to "real A.I." As a counter-example, take something very simple like search-and-replace. Word processors have had this function for decades, and it hasn't gotten any better. How long will it be before I can tell my word processor to replace all occurrences of {singular proper noun phrase} with {plural proper noun} and shoot it off to my boss without double-checking each replacement?
- xaa 12y ago
  I think it's a little deeper than that. During the '70s and '80s heyday of AI, the focus was on mechanisms that would mimic high-level aspects of human cognition: A* search, Prolog, and so on. The overall approach at the time was top-down, and focusing on deterministic algorithms. I think what we have learned in the interim is not just that "machine-learning" style approaches are successful at specific tasks, but also that a statistical, data-driven, "bottom-up" approach is going to need to be an integral part of any eventual general AI solution. Whether we will need to explicitly design the high-level cognitive elements, or whether they will emerge from properly constructed low-level elements, remains an open question. Another very important advance since the 70s/80s is the widespread realization that every major aspect of cognition is essentially probabilistic. People have realized that a search for "exact" or "optimal" solutions to problems with the complexity seen in the real world is a futile task because of the high dimensionality of these problems.
sspiff 12y ago
Regardless of whether or not you belief in a singularity theory, it's only mentioned in passing in the article and not really relevant to this discussion.
Anderkent 12y ago
> Human-like AI is a dead field, or at least dead as far we can tell. We're not even sure how to build it because conceptually we don't even have a grasp on consciousness or cognition. >Then there's the false assumption that we can get to this level before doing things like, say, solving all mental health issues and that it can magically be scaled to super-human speeds. For all we know cognition happens at a certain speed and any attempts past that are troublesome. On a purely physical level, this is hard to believe. We don't have precise models of consciousness and reasoning, but the general ones (bayesian reasoning, decision theories, etc) are speed-agnostic. It's hard to imagine one that isn't. > Look at how easily we suffer from mania and other issues when our natural limiters go haywire. Sure, our hardware sucks. Plus, that's even more the reason to worry about the singularity - if a very fast ai can think and plan, but suffers from mania, that's not a very good outcome. UFAI is an area of concern for many.
JamesArgo 12y ago
As far as I can tell, the claims of the Singularity are as follows: Human-level AI is possible. Human-level AI will likely be practical this century. Once AI is sufficiently advanced, it will be capable of AI research. Most terminal goals would lead to intelligence enhancement as a sub goal, as any system with a goal would be better able to pursue it if more intelligent. Therefore, most AIs will devote resources to increasing their intelligence. Assuming humans are not near the maximum possible intellectual level, super-human AI will likely be developed soon after an AI is created that is as smart as the smartest human. This means the future will not be controlled by humans, as we will no longer be the smartest things in town. Unless we can predict the goals of the AI, we should consider the future unpredictable after the point of its creation, as our models of the future rely on human-shaped agents with human goals and capabilities. With these assumptions broken, we have a sort of singularity, a breakdown of our models of the future. It's quite conjunctive, but this all seems pretty reasonable to me - and it's worth spending resources studying.
- eli_gottlieb 12y ago
  That assumes AIs can enhance their intelligence given fixed computational resources. We don't actually know the shape of the curve for optimization power on the X-axis and necessary FLOPs on the Y-axis. If it's anything larger than linear, AIs will not self-improve in a "singularity" type fashion, but in fact hit a point of severely diminished returns relatively quickly. Judging by the computational complexity of machine-learning algorithms, it's actually entirely possible that curve is exponential (meaning that AGI is DEXPTIME-complete), in which case the only question is how far we are towards the diminishing-returns point.
  - viraptor 12y ago
    But that assumes running AI on some architecture where FLOPs have any real meaning. If we grow an AI that's externally accessible, is that less of an achievement than simulating the same on a general purpose computer? Still assuming it could achieve singularity that is...
  - ThrustVectoring 12y ago
transpy 12y ago
But... but... the exponential growth!
ThomPete 12y ago
There is nothing wrong with studying and preparing for emergent complexity in distributed autonomous networks. Not studying that would be cargo culture.
VMG 12y ago
I think future intelligent beings will be perplexed that so few humans were absolutely obsessed with the singularity, and artificial intelligence. But only those beings who have the capacity of experiencing surprise. See, I can make up predictions too.
cstross 12y ago
Singularity prediction isn't even taken seriously by most formerly-intrigued SF writers these days. (Go google Ramez Naam on the subject. Or me. Or Cory Doctorow.) Best we can probably hope for is better intelligence augmentation/amplification than we've got now. The reason it's popular is that it plays to some deep eschatological anxieties shared by many people -- and it strikes resonant echoes with Christian millennialism, making it an easy memetic poisoned chalice for anyone who has consciously rejected knee-jerk Christian doctrine but not actually re-evaluated all their composite assumptions and beliefs to drink from.
- JamesArgo 12y ago
  >Singularity prediction isn't even taken seriously by most formerly-intrigued SF writers these days. Becoming a science fiction cliche does not reduce the probability of an event occurring. Edit: retracting the following: >The reason it's popular is that it plays to some deep eschatological anxieties shared by many people. This is probably true, but that's not an argument against the feasibility of AI or the extent of its implications. You're saying, "These projections "rhyme" with those of a low-status group, therefore they can't be true."
  - JulianMorrison 12y ago
    And may increase it. The entire low level ongoing effort to build a flying car (Terrafugia, Moller, etc) is basically driven by no market and no common sense, but a whole lot of Jetsons.
    - eli_gottlieb 12y ago
      Well yeah, but welcome to humanity. We're at our best when we do what we dream of, as what's "realistic" and "feasible" is almost always just code for "the same crappy thing that happened yesterday happening again tomorrow because nobody made the effort to change it". Except then someone puts in the effort, fulfills their dreams, and the real world winds up totally different from the expected world in which those people never acted.
    - 12y ago
dragonwriter 12y ago
> It really surprises me that we think there's a realistic path to the singularity and that we obsessed about it. Human-like AI is a dead field Even if the latter is true, its largely irrelevant to the former. Human-like AI might be one route to a singularity, but not required for it; most of the singularity predictions that I've seen are more centered more around humans rapidly (and at an ever-accelerating pace) becoming proportionately less of the "processing power" involved in the systems where they are the central intelligence, rather than human-like AI per se.
entelechy0 12y ago
You are all fools! THIS IS THE SINGULARITY. WE ARE INSIDE OF IT RIGHT NOW!!!
davidgerard 12y ago
In the future, Roko's Femilisk will simulate you from first principles to see if you thought anything nasty about Tumblr social justice. http://newstechnica.com/2014/05/24/future-advanced-cyborg-human-emulations-to-keep-being-arseholes/ http://newstechnica.com/2014/05/24/future-advanced-cyborg-hu...
halfcat 12y ago
Human-level AI is not such a worthy goal. In a few years when self-driving cars are more common, we will be thankful that they aren't using human-like AI, otherwise they will get in wrecks for all kinds of preventable reasons. The human mind is highly irrational, though the general public scoffs at that statement. We are very dependent on our chemical state.
moe 12y ago
I think you're attacking a very narrow strawman by making it sound like a singularity event is something we'd actively plan or work for. Why do you think "human-like" AI is a requirement for a singularity to occur? Isn't it much more likely to emerge simply as a side-effect from continued exponential technological progress and the ever increasing network density? Maybe try to look at it with less anthropocentrism, perhaps consider how we have evolved from shouting to the internet in a mere few thousand years? Do you think we'll still be using our eyes and hands to interface with the network, in another thousand years from now?

jp555 12y ago

We are currently in the midst of a singularity of tools. Stone > Bronze > Steel > Silicon > Software > It's the "geek rapture" version that's more fantastic and extremely unlikely.

_archon_ 12y ago
This is a very interesting perspective. What do you think is the next step? I am asking for an educated guess, or even interested speculation; people in the middle of the bronze age likely didn't know that steel was the next step until after iron was extracted for the first time. If I had to guess, I would put the next phase at "integration" or "control." In that once we have functional, reliable libraries and virtual instruments, the logical next step is to integrate the high-level control of such into our physiologies. Alternatively, I could see "space" being an age we expand into, drawing our resources increasingly from space and broadening the scope of our potential that way. I'd like to see these happen in parallel if possible.
- jp555 12y ago
  I'd say integration and control are N+1 optimization, not 0+1 invention. Things like new forms of quantum logic/algorithms, artificial atoms, metamaterials, even subvocal tools (moving us from graphical user interfaces towards thought-based user interfaces), are more 0+1 to me.

mikeash 12y ago

I'm pretty sure that the whole Singularity bit in this piece is sarcastic. That's kind of the whole theme of the article. I mean, this should tip you off pretty hard: "Well now, here's the thing: automating sarcasm detection is easy. It's so easy they teach it in first year computer science courses; it's an obvious application of AI."

VLM 12y ago

Singularity talk by fictional-ish writers is just a cheap search and replace job done by people who want to talk about their theories of a monotheistic God Version 2.0 and don't want historical baggage and automatic knee jerk reactions from the version 1.0 crowd. An even better theistic analogy for the singularity is the return of Jesus. Just wait till he comes and rights all the wrongs and banishes death and knows everything about everyone who ever lived and makes a paradise on earth for the true believers ... wait was I talking about the singularity or Jesus again? (note that I'm not (intentionally) making fun of either subject, but trying to focus on the crude search and replace aspect of the analogy)

mistakenot 12y ago
I've long thought that the singularity is the tech version of the rapture.
- T-A 12y ago
  You may enjoy reading this then: http://craphound.com/rotn/download/ http://craphound.com/rotn/download/
- davidgerard 12y ago
  Atheist who can't quite get rid of religious fear? Why reinvent Jesus when you can reinvent Yahweh! http://rationalwiki.org/wiki/Roko%27s_basilisk http://rationalwiki.org/wiki/Roko%27s_basilisk

aristus 12y ago

Whatever it is we're grasping toward, it may be "artificial intelligence" only in the sense that a car is an "iron horse".

VLM 12y ago
Trying to make a really good model bird held back the development of flight for centuries. Certainly, Julius Caesar or Crassus could have afforded a Roman era hot air balloon or hang glider or glider aircraft if this false analogy hadn't stood in the way of development. Performance would have been somewhat lower than modern material craft, but probably not as bad as you think. There were several decades of modest performance aircraft before titanium, synthetic fabrics, and carbon fiber. AI workers might be more successful in the long run if they focus on making a useful martian brain, not clone humans in silicon.
- dragonwriter 12y ago
  > AI workers might be more successful in the long run if they focus on making a useful martian brain, not clone humans in silicon. AI has been a diverse field for a long time, with many different paths -- both with direct biological inspiration (often not human -- including, e.g., insect-inspired approaches) and without are common. Its not an exclusive "human or not human" modelling choice.

xaa 12y ago

You are right that you won't see many researchers publicly talking about human-level AI, or framing their research in terms of the search for general AI. But that doesn't mean the field is "dead". For example, I work in aging and, to a lesser extent, neuroscience. I have had many private conversations with researchers who will candidly say that the long-term goal of their work is lifespan extension or cognitive enhancement. Yet, for the purposes of grant and paper-writing, you'd never write such a thing; it would sound too speculative. I'm not in AI, but I strongly suspect the same thing is going on there. There are probably a lot of AI researchers who want to see human-level or higher general AI happen, and are working on it, but simply can't say so in public. We know it's possible, since we have an existence proof in the human brain.

eli_gottlieb 12y ago
>For example, I work in aging and, to a lesser extent, neuroscience. I have had many private conversations with researchers who will candidly say that the long-term goal of their work is lifespan extension or cognitive enhancement. Yet, for the purposes of grant and paper-writing, you'd never write such a thing; it would sound too speculative. I think it was Scientific American or another such magazine that ran a story, maybe a month and a half ago, in which they interviewed a whole bunch of real theoretical and computational neuroscientists about mind uploading. A not-that-small portion of these admitted that when they are in private, have had a drink or two, and are ready to admit things they would never say in a grant application, they are definitely trying to achieve whole-brain emulation. Hell, it's basically the implicit point of the Human Brain Project, or whatever those initiatives are called by the US and EU governments this past year. > You are right that you won't see many researchers publicly talking about human-level AI, or framing their research in terms of the search for general AI. But that doesn't mean the field is "dead". Indeed. What instead happened is that actual humanlike cognition has been steadily broken down into subproblems and each subproblem is being formalized and worked-on separately. There are a few papers on optimal utility-maximizing agents for general environments, but those tend to require "until the stars burn out" levels of FLOPs to work anyway. So actually, AI is going to work, but by the time it does, it won't be called AI. In fact, the researchers who'll create it would look at you funny for calling it AI. They do and will have lots of academic jargon used for specifying exactly what their algorithms do and how they do it, so much so that only when their algorithm kills all humans will anyone actually admit it was artificial intelligence ;-).

mej10 12y ago

Human-like AI isn't necessary for singularity events, especially the ones that end up killing all of us. We don't need to understand consciousness or cognition to create optimization processes that end up hurting (or killing) humanity. It surprises me how many people seem ignorant of this fact.

droopyEyelids 12y ago
What makes me so frustrated about this argument is that we already live in a world with thousands of optimization processes that end up hurting or killing humanity. Everything from the flu to tigers. Plus, all those 'processes' are evolving through selective pressure.
- mej10 12y ago
  Corporations, governments, political systems, and other organizations are also optimization processes. Some of which are certainly harmful and also already hurt many people. And these are evolving through selective pressures as well. And not generally in the pro-humanity direction, more in the "preserve self at all costs" direction.
- eli_gottlieb 12y ago
  >What makes me so frustrated about this argument is that we already live in a world with thousands of optimization processes that end up hurting or killing humanity. Yes, we've noticed. Those are evil, too.
mcguire 12y ago
I think you're misunderstanding the use of the term "singularity". It does not mean "X will happen, after which we won't be in charge", or "...after which we'll be dead". Instead, it means that something will become so intelligent that it will be different in kind, in somewhat the same way that we are so different from gravel that gravel cannot hope to understand our behavior. An optimization process that ends up killing all of us is bad, but it's not a singularity.

rayiner 12y ago

> Human-like AI is a dead field, or at least dead as far we can tell. We're not even sure how to build it because conceptually we don't even have a grasp on consciousness or cognition. The current optimism around A.I. is the result not of real advances in the field, but the very useful outcomes that arise from doing pattern matching and machine learning on big data sets. When people can type "weather today" into Google and get a little card that shows the weather in Philadelphia today, it's hard not to be impressed, not realizing that these tricks don't bring us closer to "real A.I." As a counter-example, take something very simple like search-and-replace. Word processors have had this function for decades, and it hasn't gotten any better. How long will it be before I can tell my word processor to replace all occurrences of {singular proper noun phrase} with {plural proper noun} and shoot it off to my boss without double-checking each replacement?

xaa 12y ago
I think it's a little deeper than that. During the '70s and '80s heyday of AI, the focus was on mechanisms that would mimic high-level aspects of human cognition: A* search, Prolog, and so on. The overall approach at the time was top-down, and focusing on deterministic algorithms. I think what we have learned in the interim is not just that "machine-learning" style approaches are successful at specific tasks, but also that a statistical, data-driven, "bottom-up" approach is going to need to be an integral part of any eventual general AI solution. Whether we will need to explicitly design the high-level cognitive elements, or whether they will emerge from properly constructed low-level elements, remains an open question. Another very important advance since the 70s/80s is the widespread realization that every major aspect of cognition is essentially probabilistic. People have realized that a search for "exact" or "optimal" solutions to problems with the complexity seen in the real world is a futile task because of the high dimensionality of these problems.

sspiff 12y ago

Regardless of whether or not you belief in a singularity theory, it's only mentioned in passing in the article and not really relevant to this discussion.

Anderkent 12y ago

> Human-like AI is a dead field, or at least dead as far we can tell. We're not even sure how to build it because conceptually we don't even have a grasp on consciousness or cognition. >Then there's the false assumption that we can get to this level before doing things like, say, solving all mental health issues and that it can magically be scaled to super-human speeds. For all we know cognition happens at a certain speed and any attempts past that are troublesome. On a purely physical level, this is hard to believe. We don't have precise models of consciousness and reasoning, but the general ones (bayesian reasoning, decision theories, etc) are speed-agnostic. It's hard to imagine one that isn't. > Look at how easily we suffer from mania and other issues when our natural limiters go haywire. Sure, our hardware sucks. Plus, that's even more the reason to worry about the singularity - if a very fast ai can think and plan, but suffers from mania, that's not a very good outcome. UFAI is an area of concern for many.

JamesArgo 12y ago

As far as I can tell, the claims of the Singularity are as follows: Human-level AI is possible. Human-level AI will likely be practical this century. Once AI is sufficiently advanced, it will be capable of AI research. Most terminal goals would lead to intelligence enhancement as a sub goal, as any system with a goal would be better able to pursue it if more intelligent. Therefore, most AIs will devote resources to increasing their intelligence. Assuming humans are not near the maximum possible intellectual level, super-human AI will likely be developed soon after an AI is created that is as smart as the smartest human. This means the future will not be controlled by humans, as we will no longer be the smartest things in town. Unless we can predict the goals of the AI, we should consider the future unpredictable after the point of its creation, as our models of the future rely on human-shaped agents with human goals and capabilities. With these assumptions broken, we have a sort of singularity, a breakdown of our models of the future. It's quite conjunctive, but this all seems pretty reasonable to me - and it's worth spending resources studying.

eli_gottlieb 12y ago
That assumes AIs can enhance their intelligence given fixed computational resources. We don't actually know the shape of the curve for optimization power on the X-axis and necessary FLOPs on the Y-axis. If it's anything larger than linear, AIs will not self-improve in a "singularity" type fashion, but in fact hit a point of severely diminished returns relatively quickly. Judging by the computational complexity of machine-learning algorithms, it's actually entirely possible that curve is exponential (meaning that AGI is DEXPTIME-complete), in which case the only question is how far we are towards the diminishing-returns point.
- viraptor 12y ago
  But that assumes running AI on some architecture where FLOPs have any real meaning. If we grow an AI that's externally accessible, is that less of an achievement than simulating the same on a general purpose computer? Still assuming it could achieve singularity that is...
- ThrustVectoring 12y ago

transpy 12y ago

But... but... the exponential growth!

ThomPete 12y ago

There is nothing wrong with studying and preparing for emergent complexity in distributed autonomous networks. Not studying that would be cargo culture.

VMG 12y ago

I think future intelligent beings will be perplexed that so few humans were absolutely obsessed with the singularity, and artificial intelligence. But only those beings who have the capacity of experiencing surprise. See, I can make up predictions too.

cstross 12y ago

Singularity prediction isn't even taken seriously by most formerly-intrigued SF writers these days. (Go google Ramez Naam on the subject. Or me. Or Cory Doctorow.) Best we can probably hope for is better intelligence augmentation/amplification than we've got now. The reason it's popular is that it plays to some deep eschatological anxieties shared by many people -- and it strikes resonant echoes with Christian millennialism, making it an easy memetic poisoned chalice for anyone who has consciously rejected knee-jerk Christian doctrine but not actually re-evaluated all their composite assumptions and beliefs to drink from.

JamesArgo 12y ago
>Singularity prediction isn't even taken seriously by most formerly-intrigued SF writers these days. Becoming a science fiction cliche does not reduce the probability of an event occurring. Edit: retracting the following: >The reason it's popular is that it plays to some deep eschatological anxieties shared by many people. This is probably true, but that's not an argument against the feasibility of AI or the extent of its implications. You're saying, "These projections "rhyme" with those of a low-status group, therefore they can't be true."
- JulianMorrison 12y ago
  And may increase it. The entire low level ongoing effort to build a flying car (Terrafugia, Moller, etc) is basically driven by no market and no common sense, but a whole lot of Jetsons.
  - eli_gottlieb 12y ago
    Well yeah, but welcome to humanity. We're at our best when we do what we dream of, as what's "realistic" and "feasible" is almost always just code for "the same crappy thing that happened yesterday happening again tomorrow because nobody made the effort to change it". Except then someone puts in the effort, fulfills their dreams, and the real world winds up totally different from the expected world in which those people never acted.
  - 12y ago

dragonwriter 12y ago

> It really surprises me that we think there's a realistic path to the singularity and that we obsessed about it. Human-like AI is a dead field Even if the latter is true, its largely irrelevant to the former. Human-like AI might be one route to a singularity, but not required for it; most of the singularity predictions that I've seen are more centered more around humans rapidly (and at an ever-accelerating pace) becoming proportionately less of the "processing power" involved in the systems where they are the central intelligence, rather than human-like AI per se.

entelechy0 12y ago

You are all fools! THIS IS THE SINGULARITY. WE ARE INSIDE OF IT RIGHT NOW!!!

davidgerard 12y ago

In the future, Roko's Femilisk will simulate you from first principles to see if you thought anything nasty about Tumblr social justice. http://newstechnica.com/2014/05/24/future-advanced-cyborg-human-emulations-to-keep-being-arseholes/ http://newstechnica.com/2014/05/24/future-advanced-cyborg-hu...

halfcat 12y ago

Human-level AI is not such a worthy goal. In a few years when self-driving cars are more common, we will be thankful that they aren't using human-like AI, otherwise they will get in wrecks for all kinds of preventable reasons. The human mind is highly irrational, though the general public scoffs at that statement. We are very dependent on our chemical state.

moe 12y ago

I think you're attacking a very narrow strawman by making it sound like a singularity event is something we'd actively plan or work for. Why do you think "human-like" AI is a requirement for a singularity to occur? Isn't it much more likely to emerge simply as a side-effect from continued exponential technological progress and the ever increasing network density? Maybe try to look at it with less anthropocentrism, perhaps consider how we have evolved from shouting to the internet in a mere few thousand years? Do you think we'll still be using our eyes and hands to interface with the network, in another thousand years from now?