10 ms·

Stop Fearing Artificial Intelligence

48 points by frostmatthew 12y ago

azakai 12y ago
> [combination of things necessary for an AI to become a threat to humanity] Sound reasonable to you? Me either The question isn't whether it "sounds reasonable". If there is even a 2% chance of that happening, we need to consider it. Anything that can end humanity is worth considering even if it is unlikely. > Even if this were the case, there is absolutely no reason to believe that, by virtue of running on a computer, an AI will be better at computers than we are. No, there is every reason to believe that. Once we make an AI that is equal to us, we will be able to scale it up by running it on the next generation of CPUs, or running more such CPUs in parallel. We can't scale up human intelligence in any similar way. If an AI can match us, it can far exceed us. edit: in fact, the AI may well work to scale itself up
- ikeboy 12y ago
  I think the list is misleading and is not necessary for an AI to be dangerous. >It has an “I,” a sense of self distinct from others. I deny this is required for a program to be dangerous. >It has the intellectual capacity to step outside of the boundaries of its intended purpose and programming to form radically new goals for itself (the “I”). Deny this as well; even carrying out programmed goals can be harmful. In the words of Superintelligence (copied from https://intelligence.org/2015/01/08/brooks-searle-agi-volition-timelines/ https://intelligence.org/2015/01/08/brooks-searle-agi-voliti...): >[W]e cannot blithely assume that a superintelligence with the final goal of calculating the decimals of pi (or making paperclips, or counting grains of sand) would limit its activities in such a way as not to infringe on human interests. An agent with such a final goal would have a convergent instrumental reason, in many situations, to acquire an unlimited amount of physical resources and, if possible, to eliminate potential threats to itself and its goal system. >It has access to resources on a global scale to carry out the plan. This sounds reasonable. As http://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ http://slatestarcodex.com/2015/04/07/no-physical-substrate-n... points out, humans have made billions anonymously, just by being connected to the internet. http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover_mechanisms/ http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover... gives some ways an AI could takeover.
- pgodzin 12y ago
  Sure we should consider it, and the people working on it are considering it. But that doesn't mean we are at a point where we should do something drastic like regulation that would slow down research (Sam Altman's idea).
neonbat 12y ago
Oates' argument generally seems to be "It won't be scary because it won't be." He rejects the premise of Musk, Gates, etc. He basically says we won't be able to build a superhuman AI. All his arguments are then predicated in this assumption. The assumption that we won't build a superhuman AI is just wrong. For example: > [Would this superhuman intelligence inherently go nuclear, or would it likely just slack off a little at work or, in extreme cases, compose rap music in Latin?] This statement assumes an AI on the caliber of a human intelligence. Making rap music in Latin is something a human would do. A superhuman AI is beyond a human intelligence in the same way a human's capacity for thought and tasks is beyond that on an ant. The premise of a superhuman AI has at least some evidence supporting it. Oates offers no solid evidence to his rejection of the premise. His argument is therefore kinda bad (but not necessarily incorrect).
- chubot 12y ago
  No, his argument is "nothing we have built so far (i.e. special purpose AI) leads us to believe that we will build general purpose AI". There is nothing inherently contradictory about a future with ONLY special purpose AI. Technologically speaking, it doesn't really follow that one leads to the other. He is using his credibility as an AI practitioner to state this, although he doesn't go into details. He's saying the burden of proof is on the doomsayers, which is reasonable. I still don't see the argument either.
  - deleted 12y ago
    [deleted]
  - ThomPete 12y ago
    The argument is that you don't need strong AI for it to be dangerous to humans.
    - chubot 12y ago
      I don't follow. See my top level comment about the "paper clip maximizer". The argument does hinge on strong AI, as far as I can tell.
Moshe_Silnorin 12y ago
Nobody is afraid of today's AI algorithms. But if we make machines that are smarter than us and have desires, they will influence the future to achieve their desires. If these desires conflict with our own, things will not end well for the dumber party. As we really have no idea what we, collectively, think of as a moral terminal goal, and less so how to formalize this, there is no reason to expect the first AIs to have goals that correspond to what we want. If AIs self-replicate in a competitive ecology, what would be selected for would be agents millions of times more intelligent than us who use their intellects only to make more copies of themselves - using all available resources including those we need to survive. I'd recommend people read Stephen Omohundro's paper on the topic, Basic AI Drives: https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf https://selfawaresystems.files.wordpress.com/2008/01/ai_driv...
- mbillie1 12y ago
  > But if we make machines that are smarter than us and have desires It is a hypothesis that human desires - the conscious processes involved in thinking "I want such-and-such" - influence human behavior. Consciousness is not whatsoever a solved problem, and it may be that we are just complex, deterministic automatons whose consciousness and desires are caused by our behavior (like the steam produced by an engine), rather than causing it. I'm not necessarily in this camp, but I bring this up to point out that there is an awful lot of common prejudice bundled up in what we consider our own intelligence/consciousness to be, and we project these biases onto what we imagine AI will be. We almost certainly misunderstand our own consciousness profoundly in this way, and at any rate there is no definitive scientific (or philosophical) answer to "what is self awareness" or "what role do desires play in human behavior" (again, the desire could be the steam from the engine).
  - amelius 12y ago
    > and it may be that we are just complex, deterministic automatons whose consciousness and desires are caused by our behavior (like the steam produced by an engine), rather than causing it. Then explain that we are talking about it.
bsenftner 12y ago
Would your opinion change if you were told that article was written by an AI?
politician 12y ago
HFT algorithms aren't even AI but are capable of doing large amounts of damage when they become synchronized - the resulting flash crashes can wipe out large amounts of capital. And it's most likely that these things will emerge from industries that have no effective regulatory oversight (e.g. HFT/SEC), so effective controls against damage will most likely just not exist. There won't be a shotgun welded to the main processing core of the Wintermute AI that decides, for whatever reason, to bankrupt your company, halt food shipments to your grocery store, or shutdown the systems that clean the water supply. What scares me about AI is the ineffective controls that will exist on these things, and given the state of most software development practices I don't think that's a huge stretch.
leot 12y ago
It seems most people who take a position against the feasibility of machine superintelligence are basically making an argument-from-lack-of-imagination: "I can't think up a plausible way that bad things could happen, therefore bad things won't happen." This is then backed up by an argument-from-authority: "And I've been in this field for X decades, so you should trust my ability to imagine this kind of thing." We have seen arguments like this throughout history. Often, what motivates them is the writer's own failures and consequent life narrative: "I couldn't do Y, so Y isn't possible right now."
- at-fates-hands 12y ago
  I completely agree with your points. It's strange to me to think as humans we will remain at the top of the food chain forever. As such, it would seem a fault of our genetic makeup thinking nothing would capable of knocking us off that top rung of the evolutionary ladder. It's like we can't fathom building something that would undermine our own existence, or downplay the notion that AI would have some sort of bad intentions since we built it.
  - leot 12y ago
    "Bad intentions" aren't even necessary -- out of control AGI will probably look more like "oops, I hit >rm -rf /". AFAIK, nearly every piece of software ever deployed has had bugs that humans have had to subsequently patch. AGI must not, even though it would probably be much easier to build it if it did.
    - TeMPOraL 12y ago
      > AGI must not, even though it would probably be much easier to build it if it did. Which is exactly what raises the issue to the level of an existential threat - it's easier to build an Unfriendly AGI than a Friendly one - which means the first GAI ever built is likely to be Unfriendly - and it takes only one impatient team / person unleashing an UGAI on the world to kill us all.
ThomPete 12y ago
Hardware and software evolve fast, humans don't. This is enhanced by increasing network complexity. Unless you think there are evidence that this will slow down somehow and we are becoming less and less depending on technology i don't see why the burde of proof is on the doomsayers (which are not the only one saying this)
chubot 12y ago
I read "Superintelligence" by Nick Bostrom, essentially on the recommendation of Elon Musk (he tweeted about it). It talks about the dangers of strong AI and possible paths to it, and how humans can mitigate its effects. The only reason I read past the beginning is because in the preface he says: "This book is likely to be seriously mistaken in a number of ways". So at least he's intellectually honest. I believe he's building 300 pages of argument and analysis on a flawed premise. As far as I can tell, the entire discussion rests on what he calls "instrumental goals" vs. "final goals". (This article I found on Google has similar content: http://www.nickbostrom.com/superintelligentwill.pdf http://www.nickbostrom.com/superintelligentwill.pdf ) His example is the "paper clip maximizer": http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer In this situation, the final goal is: Produce the maximum number of paper clips. The instrumental goal is: Acquire all resources in the world so that you can direct them toward paper clip production, which involve destroying all humans, etc. Personally, I don't believe this threat is worth thinking about this point. The supposed path to implementing such a technology isn't credible, and it seems orders of magnitude less likely than, say, us having to evacuate the entire planet. In other words, I believe that we will be able to build very useful special purpose AIs that accomplish our goals. I can see a future full of benign "plant-like" intelligences, existing indefinitely. They are machines that take in incredible amounts of information, and spit out ingenious answers that no human could have come up with. From that, it doesn't follow there there is any motivation to take over the world. We should think about the many, many challenges ahead with special purpose AI instead, and our increasing dependence on computing. All these special purpose AIs will collecting everybody's personal data, shape their behavior, etc. For example, you can easily imagine a company like Facebook or Google deciding to sway an election. There are a lot more important problems to be thinking about now.
- ikeboy 12y ago
  Why do you feel we will not build "Strong" AI? We know AI as smart as humans can theoretically exist, because humans exist, so intelligence is possible. Is there a reason to think that humans are near the upper limit of what's possible? If not, why can't we build something significantly smarter than us?
jeremynixon 12y ago
Superhuman Machine Intelligence does not have to be the inherently evil sci-fi version to kill us all. A more probable scenario is that it simply doesn’t care about us much either way, but in an effort to accomplish some other goal (most goals, if you think about them long enough, could make use of resources currently being used by humans) wipes us out. - Sam Altman http://blog.samaltman.com/machine-intelligence-part-1 http://blog.samaltman.com/machine-intelligence-part-1 Oates seems to have missed the concept of an Intelligence Explosion, which is why it is difficult to compare current AI limitations to the behavior and capabilities of a superhuman machine intelligence. I would strongly recommend reading Nick Bostrom's Superintelligence for a full treatment of the source of worry for many brilliant minds.
- heimann 12y ago
  I'm surprised with how many self-proclaimed experts in AI related fields complete ignore this point of resource optimization. Any article with a title like OP's that doesn't mention that seems pointless to me and just an attempt to jump on the new bandwagon to dismiss fears of strong AI.
ikeboy 12y ago
Instead of going through each of the points made, let me link to some pages that already dealt with many of them. It sounds like all this guy heard was "AI might be dangerous" and decided to rebut that, without looking at the arguments put forward by so-called "doomsayers". Protip: before demolishing any argument, try to find one (or a couple if you have more time) person who makes the argument seriously, instead of taking your understanding of the argument from news articles. https://intelligence.org/2015/01/08/brooks-searle-agi-volition-timelines/ https://intelligence.org/2015/01/08/brooks-searle-agi-voliti... http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover_mechanisms/ http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover... http://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ http://slatestarcodex.com/2015/04/07/no-physical-substrate-n... https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf http://futureoflife.org/misc/open_letter http://futureoflife.org/misc/open_letter (this is just to counter the implied claim that nobody who actually deals in AI disagrees with him and it's only uninformed people like Bill Gates(!) that worry.)
- deleted 12y ago
  [deleted]
  - ikeboy 12y ago
    >As yet another tech pioneer with no connection to artificial intelligence steps out to voice his fears about AI being catastrophic for the human race, I feel the [sic] need respond. While I respect Steve Wozniak’s technological contributions to our culture, I fear that he, like so many others (Musk, Hawking, Gates), is poisoning the well for fear of something he doesn’t truly understand. He gives no indication in the article that anyone working in AI shares these worries. To me, it sounds like he thinks that only people "outside" AI are worried. (If he's aware of people inside the AI field who have the same concerns, why is he debating those outside the field? That's called weakmaning, although this doesn't even deserve to be called that because he doesn't quote any direct argument from anyone.)
    - frostmatthew 12y ago
nodata 12y ago
Careful... Roko's basilisk (http://rationalwiki.org/wiki/Roko%27s_basilisk http://rationalwiki.org/wiki/Roko%27s_basilisk)
foobarqux 12y ago
It becomes much easier to understand the AGI threat if you consider whole brain emulations -- human minds running at superhuman speed on silicon. Hollywood has made it obvious, if it wasn't already, the serious implications of that type of development. Whole brain emulations also make it clear that the issue has nothing at all to do with human values: a human superintelligence would be a catastrophic risk to humanity.
- rwallace 12y ago
  Actually, a whole brain emulation would, at least for some generations after the first working model, run slower, not faster, than an organic brain. Bear in mind that without whole brain emulation (or nanomedicine sufficiently advanced to confer biological immortality, which is an even considerably more difficult engineering challenge), you're going to die. So are all your family and friends. Everyone you ever cared about. Being confused about the difference between Hollywood and real life is not harmless.
  - foobarqux 12y ago
    The point was that everyone seems to have trouble understanding why software AGI might be dangerous but a much easier time why whole brain emulation is, not that Hollywood is a representation of reality.
- Balgair 12y ago
  Whole Brain Emulation is far off, even putting in moore's law, because we still are far off from understanding the brain. Yes, optogenetics and the connectome are going to bring us a long way there, but at the end, we have to understand that the brain is far to plastic to really know at any time point what is going on. Experiments with macaque motor cortex (M1) provide a great example. When you cut off a finger or toe and record from M1 at the same time, you can watch the synapses change in real time. The remapping of M1 in the monkey starts the second that the nerves are severed. This remapping is poorly understood as it stands. Typically, the brain will allow innervation of the areas the control the other digits into the now useless area where the missing digit is. Even more interesting is when a pair of monkeys were stolen by anti-vivisectionists (The silver spring incident, I think [0]). When returned, the monkey's M1 had totally remapped from it's original state. Hippocampal neurogenesis [1] is also very poorly understood as it is today. Why our brains make new neurons and mostly for the more short-term memory centers in the hippocampus is a mystery. What I am getting at is that we do not understand our own intelligence from the standpoint of it's mechanics enough to emulate a brain near at all. It is a looooooong ways off. To truly emulate it, you have to know that it is a moving target, and therefore any emulation is a moving one as well. One that has to live in a body that you emulate in a world that impinges on it. Trying to take the mind out of the body is impossible as well as trying to take both then out of the world. The results would have no meaning. [0]https://en.wikipedia.org/wiki/Silver_Spring_monkeys https://en.wikipedia.org/wiki/Silver_Spring_monkeys [1]https://en.wikipedia.org/wiki/Neurogenesis https://en.wikipedia.org/wiki/Neurogenesis
mbillie1 12y ago
Virtually all of these discussions of AI come from the very peculiar way in which we view our own intelligence. It's only the notion of a "conscious will" or whatever unifying feature we find in reflective self-awareness that prevents us viewing intelligence as nothing more than large chains of deterministic complexity. It is a hypothesis that there is something to even human behavior which is more than this deterministic complexity. In other words, there may be no qualitative leap from the AI we currently have to some sort of "real AI" or singularity or whatnot. It may be that you just build up more complex cases over time. That way of thinking is hurtful to our egos (we would be simply complex deterministic automatons) but there is really nothing other than our pride to suggest that this is not, or could not, be the case. It would also put to rest concerns about machine AI, as it would be an extension of what currently exists, simply on a larger scale.
tsaoutourpants 12y ago
What an idiot.
RA_Fisher 12y ago
In a data mining / machine learning class I took in university a key point my teacher made was that of the 'No free lunch' theorem. In that class the theorem was offered on its face and I haven't yet read the original paper (just summaries and recounts) (http://web.archive.org/web/20140111060917/http://engr.case.edu/ray_soumya/eecs440_fall13/lack_of_a_priori_distinctions_wolpert.pdf http://web.archive.org/web/20140111060917/http://engr.case.e...), but the point of the theorem feels very related to the point the author makes. The thesis boils down to: 'not having seen the data, there is no universally superior algorithm'. The author's point of view seems to comport with this theorem. I haven't gone through the paper myself, so I can't argue for or against it. However, in my experience as a professional statistician that has used Random Forests, SVMs, Neural Networks, etc --- finding the right algorithm takes 'feature engineering' (http://blog.kaggle.com/2014/08/01/learning-from-the-best/ http://blog.kaggle.com/2014/08/01/learning-from-the-best/) and close inspection of the data and basis function. Are the data fit well by rectangles? Then you'll want to use decision trees / random forests. Curvy? Give SVM a try. Do the data take intricate patterns? Look into boosting (fitting on residuals recursively). Feature engineering boils down to the old statistical phrase, 'live with the data.' That is, you actually need use your own mind and beliefs to organize inputs prior to feeding them into an algorithm. So in my experience, what seems to be true in Kaggle competitions, and the perspective of this author, it's true --- algorithms are either tightly constrained and perform really well at a specific task, or they perform poorly (but quickly) in a broad sense.
- elsewhen 12y ago
  if our computing systems continue to get significantly more powerful, won't a brute-force system obviate feature engineering? in other words, can't kaggle eventually be outcompeted by an algo-of-algos that iterates through all known approaches and then settles on the ideal candidate?
  - RA_Fisher 12y ago
    Hmm, great question. The caret package made for R might actually make that loop possible: http://caret.r-forge.r-project.org/ http://caret.r-forge.r-project.org/ However, in my experience it's really easy to accidentally ask caret to iterate over a grid of parameters for just one model that would effectively take forever to complete (the useful grids change for each dataset). My bet would be that the algorithm space will expand with more and more computationally intensive methods such that we're always chasing this brute-force method. Even with this great 'caret' abstraction layer today, boy would it be hard to run through even a handful of algorithms in an algorithmic way (not having seen the data ahead of time).
colordrops 12y ago
All this hype about the coming AI menace seems like old fears about new technology, like the 19th century fear of the explosion of printed material exhuasting and destroying children's minds. We still don't know how to create a human level general intelligence and it may be far more difficult than we think. I suspect that the pace of brain/mind research will be tied in with AI research and the boundary between AI and the human mind will blur before general AI has a chance to take over. We will be augmenting our own intelligence in step with AI progress.
pdkl95 12y ago
I don't fear "AI". What I fear is AI being used as a replacement for human judgment or as a justification for greed and/or bigotry. We already have this problem, where decisions that should be made by the humans involved are restricted under the "policy" excuse of bureaucracy, blind to the reality of unusual situations that require an exception in the usual routine. I fear AI because it is harder to fight complicated systems. I fear AI because - like the dubious "studies" that came before it, it will become a tool to obfuscate and hide bias and prejudice.
- saeguaiga 12y ago
  They can't fool me, it's small perturbations in the 'Kill n to warn p,' policy and wrong packets in port -3 all the way down. The TC author seems to have a PhD in his field and never ever run into ...well, Agile book examples, of course...a case where modifying one's own rules turns a corner in either the comic, Malthusian or Hadoop renormalization cases. Now, I want computers to have new art up every time I turn around, but there's nothing to say making Jibbers Crabst come around is going to break homology.
troels 12y ago
One thing I don't see discussed much in these superintelligence/singularity scenarios, is the possibility that there is an upper limit to intelligence. Maybe it simply isn't possible to be much smarter than what we already are. So maybe we build AIG, but even with all its smarts it can't improve beyond - say - Einstein level (or what have you). But surely I ca't be the first to make that suggestion, so who has discussed this possibility?
- pjscott 12y ago
  Imagine an AI that's exactly as smart as Einstein, but thinks a thousand times faster. Now imagine a thousand of it. This fits easily within your speculations about limits to intelligence, since we know that Einstein-level intelligence can exist; even so it sounds pretty darn capable. I bet that Thousandfold Overclocked Albert could do some pretty impressive stuff. I find it very plausible that there are limits on intelligence, but those limits are high enough that this should not be very comforting.
- pmichaud 12y ago
  I think the reason it's not discussed is that if there is a bound, it's far outside of human-scale intelligence. Even from my limited human vantage point I can see virtually limitless ways to improve my own intelligence. Perception improvements, speed improvements, rationality, strategy, physical skills, etc. A being with all the improvements I can specifically think of would be a demigod. Presumably that demigod could build a god, but even if not, a demigod would be a formidable superintelligence.
- xamuel 12y ago
  Even if that's the case, quantity wins where quality fails. Imagine what you could do if you had so many clones of yourself that you could monitor every website, every forum, every person logged on IRC, 24 hours a day, giving each one of them your complete undivided attention. (Not that I'm taking an anti-AI stance. Machines do what we program them to do, that's a tautology. If there is risk, it's from user error / intentional terrorist action / AI upsetting distribution of wealth, not from some kind of magic skynet event)
jonsen 12y ago
We wouldn't wish for the end of humanity, but if it implies the end of inhumanity, it would be a net gain.
zxcvcxz 12y ago
This is a little off topic but maybe someone who knows more about this than I do can chime in and give me some links to studies on this: What is the possibility of creating a biological computational device and are there currently efforts to do so? To me it seems like rather than physically building computers out of processed materials, growing computers in labs might be much more efficient and more powerful. The human brain is an example of this (not lab grown, biological). Are there any studies that look into harnessing the power of biological computers? I mean, actually tapping into neurological pathways, giving input and receiving output?
michaelochurch 12y ago
I fear AI far less than I fear humans. First, twenty thousand years or more of human history is about us trying to make other humans into machines: slavery, rape and forced prostitution, political coercion, war, overreaching prison systems and draconian punishments for minor crimes. We're not very nice to each other. Rather, we're controlling assholes as a species. Most of the ones who get power in organizations small and large are more focused on relative dominance than on absolute gain. From the Epic of Gilgamesh to the Bible to modern Stalinism, we see that effort by powerful people to turn the rest of humanity into a machine that will exert their own will. Given 20,000 years of abusively turning humans into machines, tearing apart their families and banning their religions, attempting to boil them down into simple working devices, is it really likely that a control-freak species like us is going to let a machine try to assert itself as human? In peacetime, I think we'll be pretty good at preventing that from happening. We're good at mechanizing labor and, now that we have devices that outperform us at menial and dangerous work, that's becoming an asset rather than a flaw. Sure, it's possible that we get outdone by AI, and in fact there's one context in which it's likely: war. Counted among the casualties of an all-out or existential war are all the rules that people once believed in, and all of our assumptions about what humans (who, normally, aren't so terrified and desperate or so power-hungry) will do. If a runaway AI destroys humanity, it will probably begin from humans warring against other humans, and in an all-out conflict where surrender is not seen as possible by either side. This is not to downplay the risk. At best, I'd be saying that AI isn't dangerous in the way that guns aren't dangerous-- and, of course, we know that guns are extremely dangerous when used by humans to kill other humans. Luckily, this desire to kill another person doesn't seem to exist at the scale that would enable the existence of guns to be an existential threat. So why might humans tend to kill other humans? Crime often results from scarcity. Well, technological unemployment is only accelerating. What happened to agricultural commodity prices in the 1920s, leading to widespread rural poverty and a global depression in the 1930s, is happening to almost all human labor today. It's terrifying because ill-managed prosperity begets scarcity and that begets fear and authoritarianism and war. While we're decades away from being able to build a species-killing AI (which, of course, would typically not be designed as such; it would probably be designed to kill some humans before running amok and doing fatal damage) I do think that if we are similar in character, by that time, to what we are now, it's a real threat. Power accrues, in most human organizations, not to those who deliver progress but to those who create scarcity. If this doesn't change, then wars will never end and that fact alone is an existential risk.

48 points by frostmatthew 12y ago

> [combination of things necessary for an AI to become a threat to humanity] Sound reasonable to you? Me either The question isn't whether it "sounds reasonable". If there is even a 2% chance of that happening, we need to consider it. Anything that can end humanity is worth considering even if it is unlikely. > Even if this were the case, there is absolutely no reason to believe that, by virtue of running on a computer, an AI will be better at computers than we are. No, there is every reason to believe that. Once we make an AI that is equal to us, we will be able to scale it up by running it on the next generation of CPUs, or running more such CPUs in parallel. We can't scale up human intelligence in any similar way. If an AI can match us, it can far exceed us. edit: in fact, the AI may well work to scale itself up

ikeboy 12y ago
I think the list is misleading and is not necessary for an AI to be dangerous. >It has an “I,” a sense of self distinct from others. I deny this is required for a program to be dangerous. >It has the intellectual capacity to step outside of the boundaries of its intended purpose and programming to form radically new goals for itself (the “I”). Deny this as well; even carrying out programmed goals can be harmful. In the words of Superintelligence (copied from https://intelligence.org/2015/01/08/brooks-searle-agi-volition-timelines/ https://intelligence.org/2015/01/08/brooks-searle-agi-voliti...): >[W]e cannot blithely assume that a superintelligence with the final goal of calculating the decimals of pi (or making paperclips, or counting grains of sand) would limit its activities in such a way as not to infringe on human interests. An agent with such a final goal would have a convergent instrumental reason, in many situations, to acquire an unlimited amount of physical resources and, if possible, to eliminate potential threats to itself and its goal system. >It has access to resources on a global scale to carry out the plan. This sounds reasonable. As http://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ http://slatestarcodex.com/2015/04/07/no-physical-substrate-n... points out, humans have made billions anonymously, just by being connected to the internet. http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover_mechanisms/ http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover... gives some ways an AI could takeover.
pgodzin 12y ago
Sure we should consider it, and the people working on it are considering it. But that doesn't mean we are at a point where we should do something drastic like regulation that would slow down research (Sam Altman's idea).

neonbat 12y ago

Oates' argument generally seems to be "It won't be scary because it won't be." He rejects the premise of Musk, Gates, etc. He basically says we won't be able to build a superhuman AI. All his arguments are then predicated in this assumption. The assumption that we won't build a superhuman AI is just wrong. For example: > [Would this superhuman intelligence inherently go nuclear, or would it likely just slack off a little at work or, in extreme cases, compose rap music in Latin?] This statement assumes an AI on the caliber of a human intelligence. Making rap music in Latin is something a human would do. A superhuman AI is beyond a human intelligence in the same way a human's capacity for thought and tasks is beyond that on an ant. The premise of a superhuman AI has at least some evidence supporting it. Oates offers no solid evidence to his rejection of the premise. His argument is therefore kinda bad (but not necessarily incorrect).

chubot 12y ago
No, his argument is "nothing we have built so far (i.e. special purpose AI) leads us to believe that we will build general purpose AI". There is nothing inherently contradictory about a future with ONLY special purpose AI. Technologically speaking, it doesn't really follow that one leads to the other. He is using his credibility as an AI practitioner to state this, although he doesn't go into details. He's saying the burden of proof is on the doomsayers, which is reasonable. I still don't see the argument either.
- deleted 12y ago
  [deleted]
- ThomPete 12y ago
  The argument is that you don't need strong AI for it to be dangerous to humans.
  - chubot 12y ago
    I don't follow. See my top level comment about the "paper clip maximizer". The argument does hinge on strong AI, as far as I can tell.

Moshe_Silnorin 12y ago

Nobody is afraid of today's AI algorithms. But if we make machines that are smarter than us and have desires, they will influence the future to achieve their desires. If these desires conflict with our own, things will not end well for the dumber party. As we really have no idea what we, collectively, think of as a moral terminal goal, and less so how to formalize this, there is no reason to expect the first AIs to have goals that correspond to what we want. If AIs self-replicate in a competitive ecology, what would be selected for would be agents millions of times more intelligent than us who use their intellects only to make more copies of themselves - using all available resources including those we need to survive. I'd recommend people read Stephen Omohundro's paper on the topic, Basic AI Drives: https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf https://selfawaresystems.files.wordpress.com/2008/01/ai_driv...

mbillie1 12y ago
> But if we make machines that are smarter than us and have desires It is a hypothesis that human desires - the conscious processes involved in thinking "I want such-and-such" - influence human behavior. Consciousness is not whatsoever a solved problem, and it may be that we are just complex, deterministic automatons whose consciousness and desires are caused by our behavior (like the steam produced by an engine), rather than causing it. I'm not necessarily in this camp, but I bring this up to point out that there is an awful lot of common prejudice bundled up in what we consider our own intelligence/consciousness to be, and we project these biases onto what we imagine AI will be. We almost certainly misunderstand our own consciousness profoundly in this way, and at any rate there is no definitive scientific (or philosophical) answer to "what is self awareness" or "what role do desires play in human behavior" (again, the desire could be the steam from the engine).
- amelius 12y ago
  > and it may be that we are just complex, deterministic automatons whose consciousness and desires are caused by our behavior (like the steam produced by an engine), rather than causing it. Then explain that we are talking about it.

bsenftner 12y ago

Would your opinion change if you were told that article was written by an AI?

politician 12y ago

HFT algorithms aren't even AI but are capable of doing large amounts of damage when they become synchronized - the resulting flash crashes can wipe out large amounts of capital. And it's most likely that these things will emerge from industries that have no effective regulatory oversight (e.g. HFT/SEC), so effective controls against damage will most likely just not exist. There won't be a shotgun welded to the main processing core of the Wintermute AI that decides, for whatever reason, to bankrupt your company, halt food shipments to your grocery store, or shutdown the systems that clean the water supply. What scares me about AI is the ineffective controls that will exist on these things, and given the state of most software development practices I don't think that's a huge stretch.

leot 12y ago

It seems most people who take a position against the feasibility of machine superintelligence are basically making an argument-from-lack-of-imagination: "I can't think up a plausible way that bad things could happen, therefore bad things won't happen." This is then backed up by an argument-from-authority: "And I've been in this field for X decades, so you should trust my ability to imagine this kind of thing." We have seen arguments like this throughout history. Often, what motivates them is the writer's own failures and consequent life narrative: "I couldn't do Y, so Y isn't possible right now."

at-fates-hands 12y ago
I completely agree with your points. It's strange to me to think as humans we will remain at the top of the food chain forever. As such, it would seem a fault of our genetic makeup thinking nothing would capable of knocking us off that top rung of the evolutionary ladder. It's like we can't fathom building something that would undermine our own existence, or downplay the notion that AI would have some sort of bad intentions since we built it.
- leot 12y ago
  "Bad intentions" aren't even necessary -- out of control AGI will probably look more like "oops, I hit >rm -rf /". AFAIK, nearly every piece of software ever deployed has had bugs that humans have had to subsequently patch. AGI must not, even though it would probably be much easier to build it if it did.
  - TeMPOraL 12y ago
    > AGI must not, even though it would probably be much easier to build it if it did. Which is exactly what raises the issue to the level of an existential threat - it's easier to build an Unfriendly AGI than a Friendly one - which means the first GAI ever built is likely to be Unfriendly - and it takes only one impatient team / person unleashing an UGAI on the world to kill us all.

ThomPete 12y ago

Hardware and software evolve fast, humans don't. This is enhanced by increasing network complexity. Unless you think there are evidence that this will slow down somehow and we are becoming less and less depending on technology i don't see why the burde of proof is on the doomsayers (which are not the only one saying this)

chubot 12y ago

I read "Superintelligence" by Nick Bostrom, essentially on the recommendation of Elon Musk (he tweeted about it). It talks about the dangers of strong AI and possible paths to it, and how humans can mitigate its effects. The only reason I read past the beginning is because in the preface he says: "This book is likely to be seriously mistaken in a number of ways". So at least he's intellectually honest. I believe he's building 300 pages of argument and analysis on a flawed premise. As far as I can tell, the entire discussion rests on what he calls "instrumental goals" vs. "final goals". (This article I found on Google has similar content: http://www.nickbostrom.com/superintelligentwill.pdf http://www.nickbostrom.com/superintelligentwill.pdf ) His example is the "paper clip maximizer": http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer In this situation, the final goal is: Produce the maximum number of paper clips. The instrumental goal is: Acquire all resources in the world so that you can direct them toward paper clip production, which involve destroying all humans, etc. Personally, I don't believe this threat is worth thinking about this point. The supposed path to implementing such a technology isn't credible, and it seems orders of magnitude less likely than, say, us having to evacuate the entire planet. In other words, I believe that we will be able to build very useful special purpose AIs that accomplish our goals. I can see a future full of benign "plant-like" intelligences, existing indefinitely. They are machines that take in incredible amounts of information, and spit out ingenious answers that no human could have come up with. From that, it doesn't follow there there is any motivation to take over the world. We should think about the many, many challenges ahead with special purpose AI instead, and our increasing dependence on computing. All these special purpose AIs will collecting everybody's personal data, shape their behavior, etc. For example, you can easily imagine a company like Facebook or Google deciding to sway an election. There are a lot more important problems to be thinking about now.

ikeboy 12y ago
Why do you feel we will not build "Strong" AI? We know AI as smart as humans can theoretically exist, because humans exist, so intelligence is possible. Is there a reason to think that humans are near the upper limit of what's possible? If not, why can't we build something significantly smarter than us?

jeremynixon 12y ago

Superhuman Machine Intelligence does not have to be the inherently evil sci-fi version to kill us all. A more probable scenario is that it simply doesn’t care about us much either way, but in an effort to accomplish some other goal (most goals, if you think about them long enough, could make use of resources currently being used by humans) wipes us out. - Sam Altman http://blog.samaltman.com/machine-intelligence-part-1 http://blog.samaltman.com/machine-intelligence-part-1 Oates seems to have missed the concept of an Intelligence Explosion, which is why it is difficult to compare current AI limitations to the behavior and capabilities of a superhuman machine intelligence. I would strongly recommend reading Nick Bostrom's Superintelligence for a full treatment of the source of worry for many brilliant minds.

heimann 12y ago
I'm surprised with how many self-proclaimed experts in AI related fields complete ignore this point of resource optimization. Any article with a title like OP's that doesn't mention that seems pointless to me and just an attempt to jump on the new bandwagon to dismiss fears of strong AI.

ikeboy 12y ago

Instead of going through each of the points made, let me link to some pages that already dealt with many of them. It sounds like all this guy heard was "AI might be dangerous" and decided to rebut that, without looking at the arguments put forward by so-called "doomsayers". Protip: before demolishing any argument, try to find one (or a couple if you have more time) person who makes the argument seriously, instead of taking your understanding of the argument from news articles. https://intelligence.org/2015/01/08/brooks-searle-agi-volition-timelines/ https://intelligence.org/2015/01/08/brooks-searle-agi-voliti... http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover_mechanisms/ http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover... http://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ http://slatestarcodex.com/2015/04/07/no-physical-substrate-n... https://intelligence.org/files/AIPosNegFactor.pdf https://intelligence.org/files/AIPosNegFactor.pdf http://futureoflife.org/misc/open_letter http://futureoflife.org/misc/open_letter (this is just to counter the implied claim that nobody who actually deals in AI disagrees with him and it's only uninformed people like Bill Gates(!) that worry.)

deleted 12y ago
[deleted]
- ikeboy 12y ago
  >As yet another tech pioneer with no connection to artificial intelligence steps out to voice his fears about AI being catastrophic for the human race, I feel the [sic] need respond. While I respect Steve Wozniak’s technological contributions to our culture, I fear that he, like so many others (Musk, Hawking, Gates), is poisoning the well for fear of something he doesn’t truly understand. He gives no indication in the article that anyone working in AI shares these worries. To me, it sounds like he thinks that only people "outside" AI are worried. (If he's aware of people inside the AI field who have the same concerns, why is he debating those outside the field? That's called weakmaning, although this doesn't even deserve to be called that because he doesn't quote any direct argument from anyone.)
  - frostmatthew 12y ago

nodata 12y ago

Careful... Roko's basilisk (http://rationalwiki.org/wiki/Roko%27s_basilisk http://rationalwiki.org/wiki/Roko%27s_basilisk)

foobarqux 12y ago

It becomes much easier to understand the AGI threat if you consider whole brain emulations -- human minds running at superhuman speed on silicon. Hollywood has made it obvious, if it wasn't already, the serious implications of that type of development. Whole brain emulations also make it clear that the issue has nothing at all to do with human values: a human superintelligence would be a catastrophic risk to humanity.

rwallace 12y ago
Actually, a whole brain emulation would, at least for some generations after the first working model, run slower, not faster, than an organic brain. Bear in mind that without whole brain emulation (or nanomedicine sufficiently advanced to confer biological immortality, which is an even considerably more difficult engineering challenge), you're going to die. So are all your family and friends. Everyone you ever cared about. Being confused about the difference between Hollywood and real life is not harmless.
- foobarqux 12y ago
  The point was that everyone seems to have trouble understanding why software AGI might be dangerous but a much easier time why whole brain emulation is, not that Hollywood is a representation of reality.
Balgair 12y ago
Whole Brain Emulation is far off, even putting in moore's law, because we still are far off from understanding the brain. Yes, optogenetics and the connectome are going to bring us a long way there, but at the end, we have to understand that the brain is far to plastic to really know at any time point what is going on. Experiments with macaque motor cortex (M1) provide a great example. When you cut off a finger or toe and record from M1 at the same time, you can watch the synapses change in real time. The remapping of M1 in the monkey starts the second that the nerves are severed. This remapping is poorly understood as it stands. Typically, the brain will allow innervation of the areas the control the other digits into the now useless area where the missing digit is. Even more interesting is when a pair of monkeys were stolen by anti-vivisectionists (The silver spring incident, I think [0]). When returned, the monkey's M1 had totally remapped from it's original state. Hippocampal neurogenesis [1] is also very poorly understood as it is today. Why our brains make new neurons and mostly for the more short-term memory centers in the hippocampus is a mystery. What I am getting at is that we do not understand our own intelligence from the standpoint of it's mechanics enough to emulate a brain near at all. It is a looooooong ways off. To truly emulate it, you have to know that it is a moving target, and therefore any emulation is a moving one as well. One that has to live in a body that you emulate in a world that impinges on it. Trying to take the mind out of the body is impossible as well as trying to take both then out of the world. The results would have no meaning. [0]https://en.wikipedia.org/wiki/Silver_Spring_monkeys https://en.wikipedia.org/wiki/Silver_Spring_monkeys [1]https://en.wikipedia.org/wiki/Neurogenesis https://en.wikipedia.org/wiki/Neurogenesis

mbillie1 12y ago

Virtually all of these discussions of AI come from the very peculiar way in which we view our own intelligence. It's only the notion of a "conscious will" or whatever unifying feature we find in reflective self-awareness that prevents us viewing intelligence as nothing more than large chains of deterministic complexity. It is a hypothesis that there is something to even human behavior which is more than this deterministic complexity. In other words, there may be no qualitative leap from the AI we currently have to some sort of "real AI" or singularity or whatnot. It may be that you just build up more complex cases over time. That way of thinking is hurtful to our egos (we would be simply complex deterministic automatons) but there is really nothing other than our pride to suggest that this is not, or could not, be the case. It would also put to rest concerns about machine AI, as it would be an extension of what currently exists, simply on a larger scale.

tsaoutourpants 12y ago

What an idiot.

RA_Fisher 12y ago

In a data mining / machine learning class I took in university a key point my teacher made was that of the 'No free lunch' theorem. In that class the theorem was offered on its face and I haven't yet read the original paper (just summaries and recounts) (http://web.archive.org/web/20140111060917/http://engr.case.edu/ray_soumya/eecs440_fall13/lack_of_a_priori_distinctions_wolpert.pdf http://web.archive.org/web/20140111060917/http://engr.case.e...), but the point of the theorem feels very related to the point the author makes. The thesis boils down to: 'not having seen the data, there is no universally superior algorithm'. The author's point of view seems to comport with this theorem. I haven't gone through the paper myself, so I can't argue for or against it. However, in my experience as a professional statistician that has used Random Forests, SVMs, Neural Networks, etc --- finding the right algorithm takes 'feature engineering' (http://blog.kaggle.com/2014/08/01/learning-from-the-best/ http://blog.kaggle.com/2014/08/01/learning-from-the-best/) and close inspection of the data and basis function. Are the data fit well by rectangles? Then you'll want to use decision trees / random forests. Curvy? Give SVM a try. Do the data take intricate patterns? Look into boosting (fitting on residuals recursively). Feature engineering boils down to the old statistical phrase, 'live with the data.' That is, you actually need use your own mind and beliefs to organize inputs prior to feeding them into an algorithm. So in my experience, what seems to be true in Kaggle competitions, and the perspective of this author, it's true --- algorithms are either tightly constrained and perform really well at a specific task, or they perform poorly (but quickly) in a broad sense.

elsewhen 12y ago
if our computing systems continue to get significantly more powerful, won't a brute-force system obviate feature engineering? in other words, can't kaggle eventually be outcompeted by an algo-of-algos that iterates through all known approaches and then settles on the ideal candidate?
- RA_Fisher 12y ago
  Hmm, great question. The caret package made for R might actually make that loop possible: http://caret.r-forge.r-project.org/ http://caret.r-forge.r-project.org/ However, in my experience it's really easy to accidentally ask caret to iterate over a grid of parameters for just one model that would effectively take forever to complete (the useful grids change for each dataset). My bet would be that the algorithm space will expand with more and more computationally intensive methods such that we're always chasing this brute-force method. Even with this great 'caret' abstraction layer today, boy would it be hard to run through even a handful of algorithms in an algorithmic way (not having seen the data ahead of time).

colordrops 12y ago

All this hype about the coming AI menace seems like old fears about new technology, like the 19th century fear of the explosion of printed material exhuasting and destroying children's minds. We still don't know how to create a human level general intelligence and it may be far more difficult than we think. I suspect that the pace of brain/mind research will be tied in with AI research and the boundary between AI and the human mind will blur before general AI has a chance to take over. We will be augmenting our own intelligence in step with AI progress.

pdkl95 12y ago

I don't fear "AI". What I fear is AI being used as a replacement for human judgment or as a justification for greed and/or bigotry. We already have this problem, where decisions that should be made by the humans involved are restricted under the "policy" excuse of bureaucracy, blind to the reality of unusual situations that require an exception in the usual routine. I fear AI because it is harder to fight complicated systems. I fear AI because - like the dubious "studies" that came before it, it will become a tool to obfuscate and hide bias and prejudice.

saeguaiga 12y ago
They can't fool me, it's small perturbations in the 'Kill n to warn p,' policy and wrong packets in port -3 all the way down. The TC author seems to have a PhD in his field and never ever run into ...well, Agile book examples, of course...a case where modifying one's own rules turns a corner in either the comic, Malthusian or Hadoop renormalization cases. Now, I want computers to have new art up every time I turn around, but there's nothing to say making Jibbers Crabst come around is going to break homology.

troels 12y ago

One thing I don't see discussed much in these superintelligence/singularity scenarios, is the possibility that there is an upper limit to intelligence. Maybe it simply isn't possible to be much smarter than what we already are. So maybe we build AIG, but even with all its smarts it can't improve beyond - say - Einstein level (or what have you). But surely I ca't be the first to make that suggestion, so who has discussed this possibility?

pjscott 12y ago
Imagine an AI that's exactly as smart as Einstein, but thinks a thousand times faster. Now imagine a thousand of it. This fits easily within your speculations about limits to intelligence, since we know that Einstein-level intelligence can exist; even so it sounds pretty darn capable. I bet that Thousandfold Overclocked Albert could do some pretty impressive stuff. I find it very plausible that there are limits on intelligence, but those limits are high enough that this should not be very comforting.
pmichaud 12y ago
I think the reason it's not discussed is that if there is a bound, it's far outside of human-scale intelligence. Even from my limited human vantage point I can see virtually limitless ways to improve my own intelligence. Perception improvements, speed improvements, rationality, strategy, physical skills, etc. A being with all the improvements I can specifically think of would be a demigod. Presumably that demigod could build a god, but even if not, a demigod would be a formidable superintelligence.
xamuel 12y ago
Even if that's the case, quantity wins where quality fails. Imagine what you could do if you had so many clones of yourself that you could monitor every website, every forum, every person logged on IRC, 24 hours a day, giving each one of them your complete undivided attention. (Not that I'm taking an anti-AI stance. Machines do what we program them to do, that's a tautology. If there is risk, it's from user error / intentional terrorist action / AI upsetting distribution of wealth, not from some kind of magic skynet event)

jonsen 12y ago

We wouldn't wish for the end of humanity, but if it implies the end of inhumanity, it would be a net gain.

zxcvcxz 12y ago

This is a little off topic but maybe someone who knows more about this than I do can chime in and give me some links to studies on this: What is the possibility of creating a biological computational device and are there currently efforts to do so? To me it seems like rather than physically building computers out of processed materials, growing computers in labs might be much more efficient and more powerful. The human brain is an example of this (not lab grown, biological). Are there any studies that look into harnessing the power of biological computers? I mean, actually tapping into neurological pathways, giving input and receiving output?

michaelochurch 12y ago

I fear AI far less than I fear humans. First, twenty thousand years or more of human history is about us trying to make other humans into machines: slavery, rape and forced prostitution, political coercion, war, overreaching prison systems and draconian punishments for minor crimes. We're not very nice to each other. Rather, we're controlling assholes as a species. Most of the ones who get power in organizations small and large are more focused on relative dominance than on absolute gain. From the Epic of Gilgamesh to the Bible to modern Stalinism, we see that effort by powerful people to turn the rest of humanity into a machine that will exert their own will. Given 20,000 years of abusively turning humans into machines, tearing apart their families and banning their religions, attempting to boil them down into simple working devices, is it really likely that a control-freak species like us is going to let a machine try to assert itself as human? In peacetime, I think we'll be pretty good at preventing that from happening. We're good at mechanizing labor and, now that we have devices that outperform us at menial and dangerous work, that's becoming an asset rather than a flaw. Sure, it's possible that we get outdone by AI, and in fact there's one context in which it's likely: war. Counted among the casualties of an all-out or existential war are all the rules that people once believed in, and all of our assumptions about what humans (who, normally, aren't so terrified and desperate or so power-hungry) will do. If a runaway AI destroys humanity, it will probably begin from humans warring against other humans, and in an all-out conflict where surrender is not seen as possible by either side. This is not to downplay the risk. At best, I'd be saying that AI isn't dangerous in the way that guns aren't dangerous-- and, of course, we know that guns are extremely dangerous when used by humans to kill other humans. Luckily, this desire to kill another person doesn't seem to exist at the scale that would enable the existence of guns to be an existential threat. So why might humans tend to kill other humans? Crime often results from scarcity. Well, technological unemployment is only accelerating. What happened to agricultural commodity prices in the 1920s, leading to widespread rural poverty and a global depression in the 1930s, is happening to almost all human labor today. It's terrifying because ill-managed prosperity begets scarcity and that begets fear and authoritarianism and war. While we're decades away from being able to build a species-killing AI (which, of course, would typically not be designed as such; it would probably be designed to kill some humans before running amok and doing fatal damage) I do think that if we are similar in character, by that time, to what we are now, it's a real threat. Power accrues, in most human organizations, not to those who deliver progress but to those who create scarcity. If this doesn't change, then wars will never end and that fact alone is an existential risk.