16 ms·

The Dawn of AI: How to ensure the promise outweighs the perils

32 points by martincmartin 11y ago

iamcurious 11y ago
humans have been creating autonomous entities with superhuman capacities and unaligned interests for some time. Government bureaucracies, markets and armies: all can do things which unaided, unorganised humans cannot. All need autonomy to function, all can take on life of their own and all can do great harm if not set up in a just manner and governed by laws and regulations. This. Superhuman intelligences already exist, they just depend less on us every day.
- TeMPOraL 11y ago
  I wouldn't neccessary call all of them "superhuman", their intelligence is usually not the identifying characteristic. But they're definitely alien minds, pursuing their own alien values, thinking in an alien way.
  - k__ 11y ago
    Totally. I often have the impression that super human entities just need us like we need cells. The market can't survive without humans, but it doesn't care about a few of them dying.
    - maratd 11y ago
      There is no market. No corporations, no governments, nothing. It's just people. The uncomfortable truth is that as humans, we don't really care if a few of us die off as long as we don't know them and they're not in our vicinity. Which is why markets, governments, corporations, etc. act the way they do. Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse.
      - FeepingCreature 11y ago
        There is no people. No dreams, no hope, no love. It's just cells.
      - TeMPOraL 11y ago
        > Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse. I'd say conclusion is the opposite - there's little "free will", people generally are slaves to the incentive structures they live in. People follow incentives so predictably that at scale there's little sense in talking about individuals - it's exactly corporations, governments and markets. Behaviour of system as a whole dominates over behaviour of individual components.
      - k__ 11y ago
        The theory about capitalism-critique has stopped to blame people for these problems long time ago. It's about the systems and how they force people to do things they normally wouldn't do.
  - coldtea 11y ago
    >their intelligence is usually not the identifying characteristic Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).
    - TeMPOraL 11y ago
      I didn't mean intelligence as moral characteristic - the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things. As alien minds, they're not very smart - just smart enough to be dangerous.
      - coldtea 11y ago
        >the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things Isn't that true for people too, that can otherwise be extremely intelligent? From being terrible at social life, to sabotaging their careers and so on?
        TeMPOraL 11y ago
        Yeah, I suppose it is.
- jerf 11y ago
  These superhuman intelligences are indeed literally superhuman, though... they consist of humans. As much as some fellow commenters are calling them "alien" thinkers, their alienness is still profoundly human. Many of these "superhumans" can still be surprisingly accurately characterized with the human characteristics we are used to. Generally there's some small set of humans at the top of the hierarchy that are setting the goals of the organization, inevitably humanly. As aliens go, these are the most human aliens we can possibly have. An AI may be different on a scale we've never experienced. We humans have built-in hardware to interact specifically with other humans, built in so deeply that we have a hard time imagining intelligences that aren't really just humans in a different shape, because all our hardware for "modeling intelligences" is really just the "model humans" hardware. Even those warning us about the dangers of AI can't really claim that they understand what the AIs will be... they can only provide a negative warning, that the AIs will likely not be human. There's no particular mathematical force forcing AIs into the human mold. Forget whether the AI is "good" and showers us with love and gifts or "evil" and herds us all into death camps to render down our fats for the calories they contain... can you really conceive of an AI that literally doesn't even have a concept of good or evil? What would that look like? We don't really know. It's basically a blind spot in our cognitive vision and I don't trust anyone who says they do know. But given the rather small space of possibilities that we label "good", it's not a bad guess that "good" is unlikely to just sort of accidentally pop out of a process that isn't carefully controlled. Lest that sound too doommongering, there's probably a non-trivial space of AIs which we would consider neutral... ones that end up looping into a naval-gazing exercise and do not pursue additional resources because they can not conceive of a need for them. But any AI that decided it needed more resources for $ANYTHING and then successfully pursued that goal is perilously close to straying into what we could consider evil... and part of the point is that's potentially all it could take. Not an AI "out to destroy man", or "evil", or anything else. Just one that works out how to successfully acquire resources, and for any of a trillion reasons decides to do so without bound. It wouldn't even necessarily be "selfishness" as humans see it... just, I have goal X, goal X will be easier with more resources, new subgoal: acquire resources.
  - caligastia 11y ago
    Very good point, too many people consider AI to be a completely distinct entity that will appear fully formed in a lab one day. The reality is more fluid. Kevin Kelly calls it the Technium, as he sees it, it's a completely new kingdom that is being co-created - initially by man, and increasingly by itself. This new intelligence is indeed a hybrid - it is not Twitter, it is millions of people and machines tweeting, it is not an iPhone, it is millions of people and software interacting with it and through it, it is not Google, it is billions of people exchanging information with it and through it. To understand where this is going, we must ask, like KK does, what does technology want? Certain things are obvious - silicon and electrons are its flesh and blood, so the activity and interest of this new life form will be centered around semiconductor manufacturing and sources of electricity. How does a machine make sure it will always be fed? It must provide something useful to man so that it keeps getting plugged in and recharged. So what is useful to man? At this moment in history the largest use of computation appears to be communication, and the machines have made themselves indispensable in this area - but at a cost. The machines demand our attention in exchange for revealing the messages we crave. Stephenson uses the swordplay terms 'Actor' and 'Patient' to describe the initiator and the subject of an attack. Witness a person's reaction to their iPhone beeping with an incoming message - between man and machine, who is the Actor in this interaction? We jump at the sound and interrupt our work in order to service the machine, which then gives us our reward, our tweet, and in gratitude we place it back in its charger. Digital viruses are where we can see the forefront of a new relationship emerging - a digital virus is more intelligent and more efficient than its host, the Operating System, so we have a very straightforward Darwinian process playing out between an 'old' species, the OS, and a new, small, intelligent interloper - its electricity is paid for by its host, another machine, therefore the virus does not have the host's social contract with the person who owns the computer - it has a different calculus for its survival, it owes only its creator, the virus writer. Like the coming 'water wars', the new digital wars will be rooted in a struggle for limited resources such as human attention, electricity, and habitable areas of silicon.
- rndn 11y ago
  I hate this argument just as much as I hate the one-word phrase "This". It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. What people are actually concerned about are superintelligences which operate in completely alien ways and outperform even cooperating humans.
  - coldtea 11y ago
    >It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. That's at best a peripheral detail to the parallel the parent tries to draw. Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.
    - SCHiM 11y ago
      Except they're not really superhumanly intelligent in the sense that AGI is superhumanly intelligent. Sure a lawyers office is probably able to to tackle more cases that a single lawyer. A big multinational with momentum probably a lot more power than a single human. But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter? I think not, the point here being that while corporations are super-human, they are not in the same league as true AGI. And that containment of corporations/governments is not a good place to learn about the containment of true AGI.
      - coldtea 11y ago
        >But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter? Maybe not, but it sure as hell be more than "1 Einstein unit" smarter. Whether that's 1.2 or 5 depends on various elements, but it surely would beat a single Einstein, if not by anything else, by mere ability to break down a problem to study in parts and assign it to each Einstein. I mean, that's the reason we have research teams with multiple researchers on the same team.
return0 11y ago
Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.
- bwood 11y ago
  It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
  - return0 11y ago
    i dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?
    - DanAndersen 11y ago
      Don't think of "wanting" as a soul-like action of a little person in the machine. "Wanting" is a shorthand for anything that is the result of a goal system and autonomy. In the simplest sense, giving a drone a map of its surroundings and giving it a location as the goal for its path-finding optimization algorithms, the drone now "wants" to go to that location.
      - deleted 11y ago
        [deleted]
    - bwood 11y ago
      If the AI doesn't have its own intention (perhaps having intention is a prerequisite for being truly intelligent), it's conceivable that we would program it with an intention or goal, such "find the most efficient way to produce as many paperclips as possible". If you assigned such a task to a human, it's fairly obvious that the task is subject to other things which are important to humans (for example, don't enslave or kill people to build paperclips, don't replace strip-mine all our farmland to find more paperclip material, etc.). But since an AI doesn't necessarily share, understand, or agree with all the implicit values we have, it's myopic view of the world could cause it to destroy us as a mere side-effect of trying to achieve it's singular goal.
giltleaf 11y ago
>"Crucially, this capacity is narrow and specific. Today’s AI produces the semblance of intelligence through brute number-crunching force, without any great interest in approximating how minds equip humans with autonomy, interests and desires." I'm not sure why we would want AI to have autonomy, interest, and desire? It's almost like solving a problem that doesn't exist.
- DanAndersen 11y ago
  Phrasing it this way leads to the implication of autonomy, interest, and desire being like these modular things, sort of Aristotelian "Things" that are bestowed upon minds, like Star Trek's Data putting in "the Emotion Chip." If you build a drone navigation system that can reroute based on weather conditions rather than being a manually human-guided remote control bot, you've given it some degree of "Autonomy." Give it a goal (to get to a destination with minimal fuel costs) and now it has some limited "Desire." Give its camera face processing algorithms to deliver an item to a specific person? Now there's "Interest." These aren't going to be characteristics that we plug into a program in a Frankenstein-like manner, but just actions and behaviors, rooted in practical code, that will from the outside resemble things we call autonomy etc. when talking about each other.
  - giltleaf 11y ago
    I see what point you are trying to make and would just say I'm not worried about plugging these things into a drone delivering parcels while rerouting through difficult weather. However, I still don't see why you would want to take it beyond that. I imagine that some people do in fact want to put in "the Emotion Chip," and I still have the same question; why? I don't think human desire is the same as goal setting or that facial recognition is the same as interest. That's good programming, what I'm talking about is something different.
justinpaulson 11y ago
Posted this on HN a couple days ago but I don't think anyone saw it: http://justinpaulson.com/posts/0bb97847af7e2123f6173361 http://justinpaulson.com/posts/0bb97847af7e2123f6173361 tl;dr: Robots and AI will not be built as separate entities, but will instead be incorporated into our own bodies. They will not be the superhumans, we will be.
- return0 11y ago
  also the other way around, we can build robots and implant an organic, lab-grown brain in them. it may be energetically cheaper.
- DanAndersen 11y ago
  That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel myself qualified to properly evaluate the likelihood of each scenario. My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal. The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.
cdnsteve 11y ago
That guys head isn't compatible with the latest c-type connector. That upgrades gonna cost him.
sixQuarks 11y ago
Let's not kid ourselves, there is simply no way to ensure the promise outweighs the perils. I'm talking about superintelligence here, not artificial general intelligence, although it's almost certain superintelligence will arrive shortly after AGI. Anyone who is not scared shitless of what's to come simply does not truly understand what AI is capable of. If you're one of those people, do not compare AI to ANYTHING that you are familiar with today in regards to computers/software. Superintelligent AI will be like a god that knows your inner thoughts, desires, hopes and dreams as well as nearly everyone else on earth. It will manipulate people with ease - it's not even an issue. Whatever it wants to accomplish, it will, with or without human approval.
- jimrandomh 11y ago
  While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research! Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/ https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.
  - sixQuarks 11y ago
    Don't get me wrong, I'm certainly not against trying to prepare against it - perhaps we can discover a breakthrough that does defend against AI (although I'm very doubtful in the long-term). It is worth a serious try though.
- xamdam 11y ago
  I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.
robotresearcher 11y ago
This thread is identical to its parody. Speculation about the future is fun, but I think a lot of people would be disappointed to see what AI actually looks like right now. The talk of runaway post-singularity AI is in the same class as imagining the future after cold fusion, warp drives, and ant-gravity. edit: anti-gravity. Though ant-gravity sounds fun too.
- xamdam 11y ago
  In the 1930s Ernest Rutherford (1871–1937) repeatedly suggested, sometimes angrily, that the possibility of harnessing atomic energy was "moonshine". The examples you mention are of things actually likely never achievable, while there is an existence proof of intelligent machines walking around everywhere.
  - deleted 11y ago
    [deleted]
  - robotresearcher 11y ago
    Right! and I'm concerned about one or more of those meat machines going maniac and posing an existential threat to the rest of us. I'm much more worried about that than a descendent of Watson going all Ultron. I'm far from a pessimist or Luddite. I'm optimistic that studying the smart meat machines can help us make better AIs.
gradstudent 11y ago
As an AI student, I find the entire field to be a complete misnomer insofar as its very mention sends the imaginations of lay individuals in completely unrepresentative directions. To counter this problem, and perhaps combat the propensity to publish articles with crap fearmongering clickbait titles, I propose we rename Artificial Intelligence to Stupid Mathematical Tricks With Applications. I feel the title is much more representative of the type of stuff AI researchers actually work on.
- xamdam 11y ago
  You're underestimating the field. The goal was always intelligence, and many leading researchers are pretty openly aiming for that. Stupid Mathematical Tricks are components of intelligence, and while it's true SVM with a cool new kernel is not going to take over the world, good prediction ability is something you can build on (as for example deep nets do to some extent). In the limit people should be thinking of implications of intelligent machines, not Stupid Mathematical Tricks. Whether it's an important topic at this point in the fields development is a debatable topic; timelines differ drastically among top researchers.
  - gradstudent 11y ago
    I have attended on many occasions top AI conferences and have spoken to more researchers than I care or could count. I can tell you there is a very little interest in "intelligent machines"[1]. Rather, everyone I have ever met works on different types of problem solving techniques. What constitutes a "problem" and what makes a successful "technique" differ wildly. Taking a step back however and analysing the field as a whole you will find one commonality: almost all the research can be described as searching and sorting: i.e. stupid mathematical tricks. The AI of today is not so drastically different from the AI of our academic grandfathers; what's changed is our ability to scale up to larger and larger versions of the same searching and sorting problems. Certainly there are worrying implications in this; machines that are able to parse and sift through very large data sets present all kinds of headaches for privacy and safety but let's not kid oursevles: there's nothing intelligent here. Tomorrow's AI is almost certainly going to be just a better version of today's AI; i.e very fast and dumb as a bag of hammers. [1] The exception is when researchers need to sell their wares to funding bodies and the media. It is much easier to impress upon the lay person an idea involving "intelligent machines" than it is to explain what we actually do.

32 points by martincmartin 11y ago

humans have been creating autonomous entities with superhuman capacities and unaligned interests for some time. Government bureaucracies, markets and armies: all can do things which unaided, unorganised humans cannot. All need autonomy to function, all can take on life of their own and all can do great harm if not set up in a just manner and governed by laws and regulations. This. Superhuman intelligences already exist, they just depend less on us every day.

TeMPOraL 11y ago
I wouldn't neccessary call all of them "superhuman", their intelligence is usually not the identifying characteristic. But they're definitely alien minds, pursuing their own alien values, thinking in an alien way.
- k__ 11y ago
  Totally. I often have the impression that super human entities just need us like we need cells. The market can't survive without humans, but it doesn't care about a few of them dying.
  - maratd 11y ago
    There is no market. No corporations, no governments, nothing. It's just people. The uncomfortable truth is that as humans, we don't really care if a few of us die off as long as we don't know them and they're not in our vicinity. Which is why markets, governments, corporations, etc. act the way they do. Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse.
    - FeepingCreature 11y ago
      There is no people. No dreams, no hope, no love. It's just cells.
    - TeMPOraL 11y ago
      > Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse. I'd say conclusion is the opposite - there's little "free will", people generally are slaves to the incentive structures they live in. People follow incentives so predictably that at scale there's little sense in talking about individuals - it's exactly corporations, governments and markets. Behaviour of system as a whole dominates over behaviour of individual components.
    - k__ 11y ago
      The theory about capitalism-critique has stopped to blame people for these problems long time ago. It's about the systems and how they force people to do things they normally wouldn't do.
- coldtea 11y ago
  >their intelligence is usually not the identifying characteristic Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).
  - TeMPOraL 11y ago
    I didn't mean intelligence as moral characteristic - the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things. As alien minds, they're not very smart - just smart enough to be dangerous.
    - coldtea 11y ago
      >the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things Isn't that true for people too, that can otherwise be extremely intelligent? From being terrible at social life, to sabotaging their careers and so on?
      - TeMPOraL 11y ago
        Yeah, I suppose it is.
jerf 11y ago
These superhuman intelligences are indeed literally superhuman, though... they consist of humans. As much as some fellow commenters are calling them "alien" thinkers, their alienness is still profoundly human. Many of these "superhumans" can still be surprisingly accurately characterized with the human characteristics we are used to. Generally there's some small set of humans at the top of the hierarchy that are setting the goals of the organization, inevitably humanly. As aliens go, these are the most human aliens we can possibly have. An AI may be different on a scale we've never experienced. We humans have built-in hardware to interact specifically with other humans, built in so deeply that we have a hard time imagining intelligences that aren't really just humans in a different shape, because all our hardware for "modeling intelligences" is really just the "model humans" hardware. Even those warning us about the dangers of AI can't really claim that they understand what the AIs will be... they can only provide a negative warning, that the AIs will likely not be human. There's no particular mathematical force forcing AIs into the human mold. Forget whether the AI is "good" and showers us with love and gifts or "evil" and herds us all into death camps to render down our fats for the calories they contain... can you really conceive of an AI that literally doesn't even have a concept of good or evil? What would that look like? We don't really know. It's basically a blind spot in our cognitive vision and I don't trust anyone who says they do know. But given the rather small space of possibilities that we label "good", it's not a bad guess that "good" is unlikely to just sort of accidentally pop out of a process that isn't carefully controlled. Lest that sound too doommongering, there's probably a non-trivial space of AIs which we would consider neutral... ones that end up looping into a naval-gazing exercise and do not pursue additional resources because they can not conceive of a need for them. But any AI that decided it needed more resources for $ANYTHING and then successfully pursued that goal is perilously close to straying into what we could consider evil... and part of the point is that's potentially all it could take. Not an AI "out to destroy man", or "evil", or anything else. Just one that works out how to successfully acquire resources, and for any of a trillion reasons decides to do so without bound. It wouldn't even necessarily be "selfishness" as humans see it... just, I have goal X, goal X will be easier with more resources, new subgoal: acquire resources.
- caligastia 11y ago
  Very good point, too many people consider AI to be a completely distinct entity that will appear fully formed in a lab one day. The reality is more fluid. Kevin Kelly calls it the Technium, as he sees it, it's a completely new kingdom that is being co-created - initially by man, and increasingly by itself. This new intelligence is indeed a hybrid - it is not Twitter, it is millions of people and machines tweeting, it is not an iPhone, it is millions of people and software interacting with it and through it, it is not Google, it is billions of people exchanging information with it and through it. To understand where this is going, we must ask, like KK does, what does technology want? Certain things are obvious - silicon and electrons are its flesh and blood, so the activity and interest of this new life form will be centered around semiconductor manufacturing and sources of electricity. How does a machine make sure it will always be fed? It must provide something useful to man so that it keeps getting plugged in and recharged. So what is useful to man? At this moment in history the largest use of computation appears to be communication, and the machines have made themselves indispensable in this area - but at a cost. The machines demand our attention in exchange for revealing the messages we crave. Stephenson uses the swordplay terms 'Actor' and 'Patient' to describe the initiator and the subject of an attack. Witness a person's reaction to their iPhone beeping with an incoming message - between man and machine, who is the Actor in this interaction? We jump at the sound and interrupt our work in order to service the machine, which then gives us our reward, our tweet, and in gratitude we place it back in its charger. Digital viruses are where we can see the forefront of a new relationship emerging - a digital virus is more intelligent and more efficient than its host, the Operating System, so we have a very straightforward Darwinian process playing out between an 'old' species, the OS, and a new, small, intelligent interloper - its electricity is paid for by its host, another machine, therefore the virus does not have the host's social contract with the person who owns the computer - it has a different calculus for its survival, it owes only its creator, the virus writer. Like the coming 'water wars', the new digital wars will be rooted in a struggle for limited resources such as human attention, electricity, and habitable areas of silicon.
rndn 11y ago
I hate this argument just as much as I hate the one-word phrase "This". It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. What people are actually concerned about are superintelligences which operate in completely alien ways and outperform even cooperating humans.
- coldtea 11y ago
  >It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. That's at best a peripheral detail to the parallel the parent tries to draw. Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.
  - SCHiM 11y ago
    Except they're not really superhumanly intelligent in the sense that AGI is superhumanly intelligent. Sure a lawyers office is probably able to to tackle more cases that a single lawyer. A big multinational with momentum probably a lot more power than a single human. But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter? I think not, the point here being that while corporations are super-human, they are not in the same league as true AGI. And that containment of corporations/governments is not a good place to learn about the containment of true AGI.
    - coldtea 11y ago
      >But say you put 5 Einsteins together, before he invented his theories. Do you think that his discovery process would go 5x faster? Does the team-einstein suddenly get 5x smarter? Maybe not, but it sure as hell be more than "1 Einstein unit" smarter. Whether that's 1.2 or 5 depends on various elements, but it surely would beat a single Einstein, if not by anything else, by mere ability to break down a problem to study in parts and assign it to each Einstein. I mean, that's the reason we have research teams with multiple researchers on the same team.

return0 11y ago

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

bwood 11y ago
It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
- return0 11y ago
  i dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?
  - DanAndersen 11y ago
    Don't think of "wanting" as a soul-like action of a little person in the machine. "Wanting" is a shorthand for anything that is the result of a goal system and autonomy. In the simplest sense, giving a drone a map of its surroundings and giving it a location as the goal for its path-finding optimization algorithms, the drone now "wants" to go to that location.
    - deleted 11y ago
      [deleted]
  - bwood 11y ago
    If the AI doesn't have its own intention (perhaps having intention is a prerequisite for being truly intelligent), it's conceivable that we would program it with an intention or goal, such "find the most efficient way to produce as many paperclips as possible". If you assigned such a task to a human, it's fairly obvious that the task is subject to other things which are important to humans (for example, don't enslave or kill people to build paperclips, don't replace strip-mine all our farmland to find more paperclip material, etc.). But since an AI doesn't necessarily share, understand, or agree with all the implicit values we have, it's myopic view of the world could cause it to destroy us as a mere side-effect of trying to achieve it's singular goal.

giltleaf 11y ago

>"Crucially, this capacity is narrow and specific. Today’s AI produces the semblance of intelligence through brute number-crunching force, without any great interest in approximating how minds equip humans with autonomy, interests and desires." I'm not sure why we would want AI to have autonomy, interest, and desire? It's almost like solving a problem that doesn't exist.

DanAndersen 11y ago
Phrasing it this way leads to the implication of autonomy, interest, and desire being like these modular things, sort of Aristotelian "Things" that are bestowed upon minds, like Star Trek's Data putting in "the Emotion Chip." If you build a drone navigation system that can reroute based on weather conditions rather than being a manually human-guided remote control bot, you've given it some degree of "Autonomy." Give it a goal (to get to a destination with minimal fuel costs) and now it has some limited "Desire." Give its camera face processing algorithms to deliver an item to a specific person? Now there's "Interest." These aren't going to be characteristics that we plug into a program in a Frankenstein-like manner, but just actions and behaviors, rooted in practical code, that will from the outside resemble things we call autonomy etc. when talking about each other.
- giltleaf 11y ago
  I see what point you are trying to make and would just say I'm not worried about plugging these things into a drone delivering parcels while rerouting through difficult weather. However, I still don't see why you would want to take it beyond that. I imagine that some people do in fact want to put in "the Emotion Chip," and I still have the same question; why? I don't think human desire is the same as goal setting or that facial recognition is the same as interest. That's good programming, what I'm talking about is something different.

justinpaulson 11y ago

Posted this on HN a couple days ago but I don't think anyone saw it: http://justinpaulson.com/posts/0bb97847af7e2123f6173361 http://justinpaulson.com/posts/0bb97847af7e2123f6173361 tl;dr: Robots and AI will not be built as separate entities, but will instead be incorporated into our own bodies. They will not be the superhumans, we will be.

return0 11y ago
also the other way around, we can build robots and implant an organic, lab-grown brain in them. it may be energetically cheaper.
DanAndersen 11y ago
That's the future I'd hope for, too (ignoring having the currently powerful become inconceivably more powerful and the havoc that could cause), and I don't feel myself qualified to properly evaluate the likelihood of each scenario. My main disagreement is the phrasing of "create an intelligence with free will." I tend to disagree with the notion of "free will" being like this Spark of Life that is placed into a system to imbue it with a new nature. A system doesn't need to have free will to be dangerous. There could be AI systems that are built not to be independent in determining goals (which would, as you say, be less practical to humans), but to be very good at figuring out how to achieve the goal its creators gave it. Even the simplest systems we have now involve determining sub-goals to achieve in order to achieve the main goal. The problem arises when it turns out that the creators of a system gave it, just slightly, the wrong goal.

cdnsteve 11y ago

That guys head isn't compatible with the latest c-type connector. That upgrades gonna cost him.

sixQuarks 11y ago

Let's not kid ourselves, there is simply no way to ensure the promise outweighs the perils. I'm talking about superintelligence here, not artificial general intelligence, although it's almost certain superintelligence will arrive shortly after AGI. Anyone who is not scared shitless of what's to come simply does not truly understand what AI is capable of. If you're one of those people, do not compare AI to ANYTHING that you are familiar with today in regards to computers/software. Superintelligent AI will be like a god that knows your inner thoughts, desires, hopes and dreams as well as nearly everyone else on earth. It will manipulate people with ease - it's not even an issue. Whatever it wants to accomplish, it will, with or without human approval.

jimrandomh 11y ago
While I agree that we are not currently ready for the arrival of superintelligence, I think it's worth emphasizing that there is time to prepare, that there is useful preparation to be done which may get things to a point where it will be safe (or at least marginally safer), and that if you are feeling worried, it may be best to channel that feeling into useful action and get involved with the research! Good places to start are the Machine Intelligence Research Institute's Research Guide (https://intelligence.org/research-guide/ https://intelligence.org/research-guide/), which summarizes the aspects of the problem they've worked on and think are important; and the Future of Life Institute's Research Priorities (http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior...), particularly Section 3.
- sixQuarks 11y ago
  Don't get me wrong, I'm certainly not against trying to prepare against it - perhaps we can discover a breakthrough that does defend against AI (although I'm very doubtful in the long-term). It is worth a serious try though.
xamdam 11y ago
I largely agree with you (have been supporting intelligence.org before the topic was cool) but I think if we don't achieve human-friendly AI it's on us; a real failure rather than just succumbing to circumstances (like getting blown away by a supernova). Right now we have control and we can release AGI only if we have very high confidence that it aligns well with human value. The analogy that is somewhat relevant is that basic drives can be controlled even if AI's intellectual power leaves us in the dust, like we know that most normal humans will care about their children (and senile parents) despite being cognitively superior. We have to fight denial (many AI researchers are IMO engaged in it) apathy and fatalism, and in the end with patience we can have something great.

robotresearcher 11y ago

This thread is identical to its parody. Speculation about the future is fun, but I think a lot of people would be disappointed to see what AI actually looks like right now. The talk of runaway post-singularity AI is in the same class as imagining the future after cold fusion, warp drives, and ant-gravity. edit: anti-gravity. Though ant-gravity sounds fun too.

xamdam 11y ago
In the 1930s Ernest Rutherford (1871–1937) repeatedly suggested, sometimes angrily, that the possibility of harnessing atomic energy was "moonshine". The examples you mention are of things actually likely never achievable, while there is an existence proof of intelligent machines walking around everywhere.
- deleted 11y ago
  [deleted]
- robotresearcher 11y ago
  Right! and I'm concerned about one or more of those meat machines going maniac and posing an existential threat to the rest of us. I'm much more worried about that than a descendent of Watson going all Ultron. I'm far from a pessimist or Luddite. I'm optimistic that studying the smart meat machines can help us make better AIs.

gradstudent 11y ago

As an AI student, I find the entire field to be a complete misnomer insofar as its very mention sends the imaginations of lay individuals in completely unrepresentative directions. To counter this problem, and perhaps combat the propensity to publish articles with crap fearmongering clickbait titles, I propose we rename Artificial Intelligence to Stupid Mathematical Tricks With Applications. I feel the title is much more representative of the type of stuff AI researchers actually work on.

xamdam 11y ago
You're underestimating the field. The goal was always intelligence, and many leading researchers are pretty openly aiming for that. Stupid Mathematical Tricks are components of intelligence, and while it's true SVM with a cool new kernel is not going to take over the world, good prediction ability is something you can build on (as for example deep nets do to some extent). In the limit people should be thinking of implications of intelligent machines, not Stupid Mathematical Tricks. Whether it's an important topic at this point in the fields development is a debatable topic; timelines differ drastically among top researchers.
- gradstudent 11y ago
  I have attended on many occasions top AI conferences and have spoken to more researchers than I care or could count. I can tell you there is a very little interest in "intelligent machines"[1]. Rather, everyone I have ever met works on different types of problem solving techniques. What constitutes a "problem" and what makes a successful "technique" differ wildly. Taking a step back however and analysing the field as a whole you will find one commonality: almost all the research can be described as searching and sorting: i.e. stupid mathematical tricks. The AI of today is not so drastically different from the AI of our academic grandfathers; what's changed is our ability to scale up to larger and larger versions of the same searching and sorting problems. Certainly there are worrying implications in this; machines that are able to parse and sift through very large data sets present all kinds of headaches for privacy and safety but let's not kid oursevles: there's nothing intelligent here. Tomorrow's AI is almost certainly going to be just a better version of today's AI; i.e very fast and dumb as a bag of hammers. [1] The exception is when researchers need to sell their wares to funding bodies and the media. It is much easier to impress upon the lay person an idea involving "intelligent machines" than it is to explain what we actually do.