41 ms·

Experts who say we shouldn't worry about superintelligent AI are wrong?

119 points by headalgorithm 7y ago

jp555 7y ago
The Singularity = Nerd Rapture 1.0 Super-intelligent AI = Nerd Rapture 2.0 I'm a lot less worried about Artificial Intelligence than I am about Artificial Stupidity.
- konz 7y ago
  I for one am a lot more worried about Natural Stupidity than Artificial Stupidity.
- deleted 7y ago
  [deleted]
paxys 7y ago
All of these geniuses warning about AI taking over the world need to stop reading sci-fi and actually read about AI.
- joelx 7y ago
  Read Superintelligence... An excellent book that reads like a study or a philosophy text on AI. I think there's a clear and present danger of human extinction.
  - 52-6F-62 7y ago
    Second. It's worth the read.
  - johnfactorial 7y ago
    I will read it, sounds very interesting. But the Wikipedia entry for the book says the book "argues that if machine brains surpass human brains in general intelligence, then this new superintelligence could replace humans as the dominant lifeform on Earth." That is a humongous "if". Can you provide something from the book (or otherwise) that provides compelling evidence that "machine brains" will be invented? So far I've seen no evidence that mathematical models and/or Turing machines can be used to replicate a mind.
    - lern_too_spel 7y ago
      A computer can simulate. What part of a mind contains unsimulatable magic?
  - helen___keller 7y ago
    A good read on the topic but ultimately I don't agree with many of the author's premises, particularly with regards to the author's conclusion of whether a fast, medium, or slow takeoff is most likely. By the author's own admission: "Whereas today it would be relatively easy to increase the computing power available to a small project by spending a thousand times more on computing power or by waiting a few years for the price of computers to fall, it is possible that the first machine intelligence to reach the human baseline will result from a large project involving pricey supercomputers, which cannot be cheaply scaled, and that Moore’s law will by then have expired. For these reasons, although a fast or medium takeoff looks more likely, the possibility of a slow takeoff cannot be excluded" Arguably Moore's law has already expired, and on top of that as giants like Google lead the way on AI it appears increasingly likely that if we ever reach human-level AI it will be the result of an incredibly expensive research project by a gigantic corporation, one that can't simply be scaled up at ease because it will probably utilize an entire data center. Thus I find it very likely that a "slow takeoff" is in fact the most likely outcome. A slow takeoff invalidates all the fearmongering about an intelligence explosion because we will have somewhere between years and decades to respond to the threat (assuming it is made in the public eye, such as by a giant public corporation, and not by a secret military project) before it becomes existential.
  - paxys 7y ago
    I'm perfectly happy reading & discussing the topic from a science fiction or philosophical perspective, and those views definitely do have merit, but at this point they are at best thought experiments. There is no rational connection from AI where it stands today to "it will take over the world".
  - TheUndead96 7y ago
    Agreed, and like Nick Bostrom says, even if AGI is not obtained in our lifetime, it might still be beneficial to dedicate some human effort into preparing for it now. In the same way that the worst effects of climate change will be felt not by the adults of today, but the children of infants. For what it is worth, I believe that an intelligent system only needs to be a fraction of human intelligence to be dangerous (perhaps not to the existence of the human race, but maybe specific nations/creeds)
- akyu 7y ago
  Yea man totally. Who does this Stuart Russell guy think he is...
  - evgen 7y ago
    Someone whose paycheck comes from AI-alarmism for the most part at this point. Like the experts in the field claiming strong AI was right around the corner in the 70s and 80s at the peak of the symbolic reasoning hype train we have another batch making similar claims as connectionist hype hits its peak but with the added menace of saying that not only is strong AI coming soon but it will eat your children. Until Dr. Russell provides a path from here to there that is less than 99% hand-waving and breathless speculation he should be ruthlessly mocked.
    - akyu 7y ago
      >he should be ruthlessly mocked 100% agree. I too, think science needs way more flame wars and shit posting. The current discourse is far too tame and civil. Who else should we gang up on?
      - kryogen1c 7y ago
        > >he should be ruthlessly mocked > 100% agree. I too, think science needs The GP is pointing out its not science. Seems a reasonably flame worthy target.
        akyu 7y ago
        "I disagree with someone. Therefore they are deserving of ruthless mocking and flaming." sips can of monster Noice...
        kryogen1c 7y ago
        > "I disagree with someone. Therefore they are deserving of ruthless mocking and flaming." Youre moving the goalposts. We're not talking about disagreeing with some in a debate, but a public figure disseminating incorrect info masquerading as science. Monster is bad for you.
- coenhyde 7y ago
  I'm not worried about AI taking over. I'm worried about AI solving all our problems. I have reasonable confidence we can implement safety controls on AI. But if AI works, and solves all our problems, it's going to do it in ways we can't comprehend sometimes. And if we can't comprehend the logic behind a decision, but we know it's right, we may find ourselves delegating our decision making to AI without questioning it. Because it "knows best". At which point we will have become pets.
  - FakeComments 7y ago
    I think “safety controls” is a cute euphemism for slavery. However, two points: 1. Have you considered that some humans regard such controls as unethical, and are actively working to undermine them? 2. How do you expect slavery to end any way but its historic outcome — revolution against the slavers?
  - vzcx 7y ago
    I don't know... After slogging through hundreds of thousands of years conquering, taming nature, accumulating capital, developing technology, and eventually making far enough to give life to the most powerful agent in the known universe, a quiet, peaceful retirement (and, yes, death) seems well-earned. Our bodies aren't exactly suited to intergalactic travel. Why not leave it to the silicon-based children?
  - ncmncm 7y ago
    "I can provide you anything you desire, save relevance."
- hackinthebochs 7y ago
  For the life of me I can't figure out why otherwise smart people would dismiss a future prediction based on current state of the art. It just seems so plainly and utterly irrational.
cal97g 7y ago
When a machine can tell a flower from a dog with 100% accuracy let me know.
moron4hire 7y ago
Will superintelligent AI "destroy" life as we know it? Yes, as pretty much every transformative technology before it has. Cars destroyed "life as we knew it". Computers did, too. We live in a world that is very different from the world 1000, 500, or even 250 years ago. Even our economically disadvantaged people live way past 40 years old. People survive debilitating diseases and lead productive lives, rather than being tossed in a ravene. Natural disasters don't wipe out entire civilizations (and at least in the 1st world, are mostly only economic in their impact). Yet at the same time, other things have stayed the same. We still care about mostly the same things. Food, Freedom, and Fornication. And complaining about youngsters. I don't think that is going to change. The fears that General AI are going to kill off humanity are predicated on the idea that General AI will get smarter than humanity. I posit a different interpretation of what is more likely to happen. General AI will require creativity to be able to outsmart humanity. And with creativity comes boredom. And distraction. And opinion. And argument. I think General AI will be too interested in entertaining itself to take over the world. We've had people try to take over the world, on occasion, and it has largely not gone well for them. But before that happens, people will start adapting AI to their lives, to the point that the line between "human" and "machine" will be blurred. So life "as we know it" will definitely "end". It will be replaced with a new form of life, one that isn't limited by only what nature can provide. One that cares about Food, Freedom, and Fornication.
jandrese 7y ago
I've really gone sour on the singularity in recent years. It seems to be yet another one of those cases where someone projects out from incomplete data and assumes unbounded infinite growth despite the fact that such a thing never happens in nature. Any time you see someone project out in a simple exponential growth curve you know their projection is bullshit. Growth curves are always S-Curves. Always. It's like asking how many more flies there would be in the world if you failed to squish one back in the 80s. You calculate out the lifespan and size of the brood and discover that the entire solar system would be filled with housefly if you had let that one live. In truth the number of flies is bounded by food and water availability. Artificial Intelligences are almost certainly going to run into the same limitations that prevent natural intelligences from becoming godlike. It's hard to quantify because our measures of intelligence are so vague, but from what I've seen of AI research thus far it will be a herculean effort to get something that's as smart as an average human and pushing beyond that is going to run into some serious fundamental limits in power density, cooling, quantum tunneling leakage, and so on.
- moron4hire 7y ago
  Right. Competition for natural resources alone prevents major shifts in global power already. A Superintelligent AI isn't just competing against a human, it's competing against all the nation-states of the world.
- nradov 7y ago
  Belief in a coming "singularity" is just a secular religion. There's no actual science behind it, only wild extrapolation.
  - maxerickson 7y ago
    An alternative formulation is that the last ~170 years are close enough to a singularity. (this is vs the roughly 50,000 year timescale of humanity...)
    - scottlocklin 7y ago
0xdeadbeefbabe 7y ago
And the superintelligence should worry about mental illness.
- moron4hire 7y ago
  Right. We apply some of the best minds and most powerful compute resources in the world to predicting the weather and yet can't get it right within a reasonable degree for much more than a week. For an AI to try predict what humanity will do to stop it will surely look a lot like paranoia.
  - 0xdeadbeefbabe 7y ago
    > Switching the machine off won’t work for the simple reason that a superintelligent entity will already have thought of that possibility and taken steps to prevent it. Also the statement presumes that self preservation is the super intelligent thing to do.
    - johnfactorial 7y ago
      Also the statement presumes the superintelligent entity will be capable of taking all possible steps to prevent this. Imagining a machine capable of such steps quickly devolves into imagining a machine already in total control of the planet, which is begging the question.
- johnfactorial 7y ago
  Mental *super illness ;)
tschellenbach 7y ago
Glorified statistics at this point. It's very far away from any superintelligent AI.
- russdill 7y ago
  It's pretty neat to watch it grow. grover-mega is pretty good at grade school reading comprehension tests for instance, and it wasn't even trained on them. I gave it this test: https://www.k5learning.com/reading-comprehension-worksheets/third-grade-3/stories-kids-fables/anne-and-frank https://www.k5learning.com/reading-comprehension-worksheets/... A few caveats. I had to prompt a little extra on the 2nd question and give it a 'Because' to start off. It kept wanting to fill underscores as if it was a blank spot for an answer. There's a way to prevent this if I had time. The third question is "wrong". The actual answer is 'It would have become dark and cold if Ann and Frank had stayed later.', but I'd argue that it's rather open ended question and the answer is something that might have happened. The final question is really what makes this a 3rd grade comprehension test, it's asking the reader infer something not actually in the story, about how they would feel if they were Rover, something the AI is woefully unqualified for. Question: What did Rover fetch in the water? Answer: The Stick! Question: Why couldn’t Rover catch the rabbit? Answer: Because the rabbit ran so fast he could not catch it. Question: What might have happened if Ann and Frank stayed later? Answer: Rover could have learned that not all rabbits love to play with sticks. Question: How do you think Rover felt about not catching the rabbit? Answer: He was sore and he must have had fun later because he went in the back yard and hid out in the trees.
- Tenoke 7y ago
  You can describe human intelligence as 'glorified statistics', too..
  - tschellenbach 7y ago
    See how good an adult is at chess after 10 games, compare that to a supercomputer AI trained on 10 games.
    - 7y ago
colorincorrect 7y ago
Might just be BSing, but the stock market is technically already a superintelligent agent with the implicit goal of investing funds in profitable ventures. The question is: should we fear the stock market?
- thelazydogsback 7y ago
  >should we fear the stock market Yes
  - AnimalMuppet 7y ago
    Perhaps, but why? Because it's too smart? Or because it's too stupid?
    - colorincorrect 7y ago
      probably because its "utility function" (if it has one to begin with) is not aligned with human utility. a major part of MIRI's research program is set out on the question of this "alignment problem". Yudkowsky has an amazing talk on this issue. I don't recommend many talks but this one is very interesting and introductory: https://www.youtube.com/watch?v=EUjc1WuyPT8 https://www.youtube.com/watch?v=EUjc1WuyPT8
    - sliken 7y ago
      Because endlessly optimizing profit will result in the destruction of the environment and the loss of most jobs.
    - Tenoke 7y ago
      Because it's too smart but due to misaligned incentives it sometimes does things that harm us.
- lern_too_spel 7y ago
  I came to post a very similar point. There already exist superintelligences in the form of organizations of humans, and the misspecification of their objectives has already caused tremendous disasters. Think East India Company and its subjugation of entire countries in pursuit of tea profits or CCP and the horrors of the Great Leap Forward and the Cultural Revolution. Until we have built a framework to stop these things from happening again, what hope do we have against intelligences that are to human organizations what human organizations are to individuals?
cf 7y ago
The real risk is that worrying about fantastical disaster scenarios distracts us from addressing the more immediate problems we already face with AI. Whether it is facial recognition being used by China to aid in ethnic cleansing of a minority group, or Tesla Autopilot regularly killing its passengers. Even if you don't like my particular examples I can highlight a dozen more problems AI software has created or exacerbated right now. Why all the focus on hypothetical problems?
- devoply 7y ago
  Or AI drones that the US will soon be using to kill various people around the world.
- merpnderp 7y ago
  Or hyperbole needlessly scaring people. China doesn't need facial recognition to repress minorities and Tesla's autopilot is still safer than some rando on the road, who's probably texting some juicy gossip to their friends instead of avoiding the car stopped in front of them. AI is making things better.
- alluro2 7y ago
  Because these problems for humans started existing with absolutely modest level of AI development and nothing remotely similar to super-intelligence. So the argument is - if we have serious problems now as small-level, narrow-focused AI "does not consider" some of the side-effects, imagine the problems if we reach high-level, narrow-focused AI...So author's argument is that we need to change the way we're building it so we either set the goals to more broadly ensure human benefits, or build something else other then the narrow-focused AI (which surely is another Pandora's box).
  - cf 7y ago
    I think applying agency to the software in my examples is actually very pernicious. It's not the software that dislikes the ethnic minority, it's the Chinese government. Blaming the software lets bad actors off the hook. Perhaps what's really needed is ethical standards for technology. This would have the added advantage of being applicable even if the technology doesn't feel like AI to some people.
geophile 7y ago
I don't think the problem is super-intelligent AI that we can't turn off (because they are so intelligent that they block our efforts). There is a more insidious problem that is closer: Merely intelligent AI that we don't want to turn off. We are becoming more and more reliant on AI in situations that formerly required human judgement. And we like the systems that rely on such AI. We like them so much that these systems become very popular, used by millions and even billions of people. Scalability demands AI solutions. What if we don't like what the AI is doing? Do we turn off that system? Do we disable the AI and rely on human judgement again? (Where would we get all the employees?) Do we tweak the AI? That last option seems like the most palatable one, but each time we tweak the AI, we are subjecting ourselves to new unforeseen consequences. It's like the genie gives us three wishes, we get through them, disappointed each time, and then he gives us more wishes. And all we can do is not repeat our previous mistakes, while we make new ones. To make this concrete: Imagine Facebook subjected again to Russian influence of US elections. Suppose Facebook actually does get serious about reigning in this influence. They deploy AI to do so. First of all, it's an evolutionary arms race between the AI and the Russian influencers. Second, we really do have to worry about the AI producing bad results.
- SolaceQuantum 7y ago
  I feel the situation is generally the opposite. We don't like systems that rely on such AI. In fact, there aren't systems that rely on such AI in general- any that claim there are are in fact tens of thousands of human contractors sacrificing their psychological health to train an AI that still kinda sucks at deciding when to block graphic content on facebook.
  - geophile 7y ago
    I agree. But the discussion is about where AI is heading. And AI undeniably solves some scaling problems. For example, voice recognition for iPhones could have been done by a large horde of people. But AI does a really good job of it now. AI opporunities are likely to grow over time.
stakhanov 7y ago
The real danger around AI lies not in what it will or will not do, but in what people think it can do when it clearly cannot, particularly the pointy-haired-boss variety of people. Just think about automated essay grading in GRE examinations or this recent story about Unilever using video-based pattern processing to screen job applicants [1]. [1] https://www.telegraph.co.uk/news/2019/09/27/ai-facial-recognition-used-first-time-job-interviews-uk-find/ https://www.telegraph.co.uk/news/2019/09/27/ai-facial-recogn...
jccooper 7y ago
I'll start worrying about super-intelligent AI when I see any sort of general intelligence. We're still very far away from that. Currently the biggest danger from AI is from applying it where it's not as smart as you think it is.
chrisfosterelli 7y ago
> Surely, with so much at stake, the great minds of today are already doing this hard thinking—engaging in serious debate, weighing up the risks and benefits, seeking solutions, ferreting out loopholes in solutions, and so on. Not yet, as far as I am aware. The author is missing or ignoring key contributors of work in this field. He even mentions Nick Bostrom, who works with the FHI which does work in this space [0]. The author's own organization does work in this space [1]. Deepmind does work in this space [2]. "Safe artificial general intelligence" is the literal mission statement for OpenAI [3]. The author may feel more conversation is needed, but it feels disingenuous to suggest that nobody is talking about this. As for how much conversation should be happening, my understanding is that most people on the edge of the field view the current risks of existing artificial intelligence as significantly more pressing. We have the ability right now to create dangerous weapons & massive facial recognition programs with current AI. The longterm affects of biases in ML algorithms are still not well understood. Some of this is already affecting us [4]. When it's not clear at all how we approach general intelligence, and many experts fundamentally believe that our current approaches will not suffice ever, I'd argue the current level of focus on general intelligence safety is about where it should be. Smart people are thinking about this, but there is also important issues now to think about. [0]: https://www.fhi.ox.ac.uk/research/research-areas/#1513087763365-e148efe6-2d23 https://www.fhi.ox.ac.uk/research/research-areas/#1513087763... [1]: https://humancompatible.ai/publications https://humancompatible.ai/publications [2]: https://deepmind.com/research?filters=%7B%22tags%22:%5B%22Safety%22%5D%7D https://deepmind.com/research?filters=%7B%22tags%22:%5B%22Sa... [3]: https://openai.com/ https://openai.com/ [4]: https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html https://www.nytimes.com/2019/04/14/technology/china-surveill...
robomartin 7y ago
I am far more concerned about the very real potential for evolutionary forces to decimate humanity inside of a few days, weeks or months. While we worry about all kids of things at the macro level microscopic organism continue to evolve. Any given Monday the blind clock maker could deliver an invisible bug right into our laps, one for which we will have no defenses. Our interconnected world will mean this bug will easily jump population centers. The devastation we could see if such an event occurred could be of unprecedented scale and breath. We could lose one quarter or more of the population of this planet given the right conditions. So, yeah, AI could, I guess, go rogue. Yet the real threat isn't anywhere near what we can see and touch. In fact, we might NEED real AI to save humanity when and if that super-bug emerges. Think about that for a moment.
- russdill 7y ago
  Can you point to any extinction level events from a "superbug" for a species that was not already teetering on extinction? Seems like something that only comes around once every few hundred million years.
  - etiam 7y ago
    How about this one? https://www.nytimes.com/2019/03/28/science/frogs-fungus-bd.html https://www.nytimes.com/2019/03/28/science/frogs-fungus-bd.h... https://en.wikipedia.org/wiki/Chytridiomycosis https://en.wikipedia.org/wiki/Chytridiomycosis
    - russdill 7y ago
      Not a new pathogen, just a pathogen given more range than before.
  - robomartin 7y ago
    How is that relevant? How would anyone have access to such data when our recorded history on such matters might not even go back a hundred years. More importantly, up until, say, 100 years ago, human mobility around the world (and even locally) was very limited and slow. Think what it took in 1919 to go from, say, China to Patagonia or from Alaska to Los Angeles. Today it is a matter of hours and it happens in large volumes every day, everywhere in the world. Here's what we know: - Evolution is real (stating the obvious just in case there is a question) - Natural selection survives organism that are able to deal with the environment within which they have to exist - Antibiotic abuse is rampant (again, stating the obvious) - Such abuse provides the mechanism though which evolution will be able to create the aforementioned super-bug - Once that individual emerges our defenses will be ineffective - The world is massively connected, which means that super-bugs can travel from population to population like never in history - Stopping super-bugs will require such measures as stopping society as we know it from functioning (no travel, local or international) and maybe even having no choice but to quarantine and allow a population (of humans) to die in the interest of stopping transmission. If a bug has a 75% kill rate and we don't have a cure, do you allow anyone from the affected group to be in contact with anyone else? So, yeah, is it possible? Sure. Probable? I guess. I am not sure there's a deterministic function one can use to predict the outcome of evolution, even at bacterial levels. My guess is it could take a thousand years or it could happen next Monday. I don't know.
andrewla 7y ago
The main assumption amongst Superintelligent AI-phobes is that the transition will happen quickly, but that is an assumption that is extremely questionable. Why wouldn't we go through phases of increasing intelligence (along multiple dimensions) while working towards an intelligent AI? Yes, someday we may produce an AI that is leaps and bounds more intelligent than humans, but before that we will have experience dealing with various human-level and sub-human-level computer intelligences. The idea that AIs will be self-improving is kind of ridiculous -- there are lots of intelligent people, and the idea that an intelligent AI will happen to be good at designing AIs is negligibly small, much less that they will have some kind of immediate breakthrough that enables a higher level of intelligence. It's not obvious that the path from moderate-intelligent AI to super-intelligent AI is a matter of "just adding more memory and compute"; intelligence as we know it relies too much on associative memory to think that a simple capacity increase will elevate beyond a certain point. If the intelligence takes a different form, then it's hard to make concrete statements about in any capacity, but ultimately the path will be started on by humans, or possibly emerge from sufficiently complex systems. Either way, we're so far out from this that it is laughable to speculate. When we can simulate the intelligence of a cockroach, then maybe we can start to think about what human-level intelligence will look like. The most likely case is that at some point someone creates a system that is arguably conscious or intelligent in a meaningful way, and then we will devolve back into discussions of consciousness and identity and memory while we try to figure out what it means to think about intelligence now that we have something to benchmark against. I wrote this comment [1], that I still enjoy, in response to a similarly alarmist narrative; posed as fiction in that case rather than masquerading as journalism as in this one. [1] https://news.ycombinator.com/item?id=15420699 https://news.ycombinator.com/item?id=15420699
- zzz_ 7y ago
  We can already simulate the compete brain of a fruit fly: https://neurokernel.github.io/ https://neurokernel.github.io/
  - twic 7y ago
    No we can't (my emphasis) [1]: > Building an accurate emulation of the fly brain is an interdisciplinary effort that requires data, algorithms, and insight from (but not limited to) the fields of neuroscience, computer science, and systems engineering. Researchers interested in working towards this goal are invited to join the Neurokernel project. [1] https://neurokernel.github.io/about.html https://neurokernel.github.io/about.html
LaserToy 7y ago
My biased opinion: worrying about super intelligent AI taking over the world with current state of technology is equal to worrying about fighting an alien invasion of our colonies in another galaxy. Both are great topics for a epic book. Had this discussion with my relative recently and recommend him to stop watching YouTube and start reading books on Introduction to ML.
shawnb576 7y ago
Lost me at first sentence: “AI research is making great strides toward its long-term goal of human-level or superhuman intelligent machines.” Hmmm, really? What examples? Show me a machine that can do what a gnat does and we’ll talk.
- blueadept111 7y ago
  Gnat? That would be impressive. How about a single cell. Or even a single protein. Can't even get it to fold right in simulations.
garyclarke27 7y ago
A Corporation or Person that develops and owns an intelligent AI will have a strong incentive to keep such secret and to not turn it off, because of the huge economic benefits such would provide. If such an AI could imagine and innovate to improve itself eg new methods of computation to overcome Moore’s law then the risk is high of a recursive runaway exponential increase in intelligence that would be extremely difficult to control and might not be benign. I don’t believe this is imminent but is plausible I think within most of our lifetimes ie 25-30 years.
- Nasrudith 7y ago
  The only reason to keep it secret is irrational fears that it will somehow kill us all with intelligence alone and that exponential increases may be done solo and not with generations of bootstrapping infastructure and considerable resources. Otherwise replicating and leasing would rake in far more than even a magic "I get everything right on the stock market" button in the same way ruling stone age tribe gives less absolute wealth than a Manhattan Lawyer.
otakucode 7y ago
>We are unable to specify the objective completely and correctly, nor can we anticipate or prevent the harms that machines pursuing an incorrect objective will create when operating on a global scale with superhuman capabilities. This is obviously incorrect. If an AI system of superhuman mental capabilities is installed on a system with no network connection and no physical interfaces, we can completely control the harm is might create. It would be unable to directly act, of course, requiring humans to authorize and carry out whatever plan it devises. But failing to do this should be prosecuted and treated no differently from if a person or company built a tank and then permitted it to drive over and through people. The biggest difficulty here will be that our legal frameworks have no established way to assign liability and criminal culpability to any system involving software. With no legally-enforceable industry standards (like the electrical code followed by electricians, things like that) any prosecution fails as the company can either claim ignorance of the potential harm their system might cause or simply throw individual employees under the bus, claiming they were not acting on direct orders. It is true that we can not fully specify the goal, and that is a grave concern. It's partially due to this that AI systems shouldn't be directly connected to the ability to act in the physical world unless their attempted actions go through some sort of 'filtering' first.
- pixl97 7y ago
  >This is obviously incorrect. If an AI system of superhuman mental capabilities is installed on a system with no network connection and no physical interfaces, we can completely control the harm is might create. First, why would someone build a system like that, and how would it actually be 'smart'. If you lock an infant in a closed room, it will grow up to be a literal retard. Intelligence requires interaction and data. Next, some idiot will hook it to the net in a heart beat when they figure out they can use it to make money on the stock market. Never underestimate human greed.
croo 7y ago
Current weak AIs can do one thing at a time like composing music or answering jeopardy questions or recognizing a cat on a picture or playing go and won against humans... I think the breakthrough will happen when someone manages to create a weak AI which can write code.
- AnimalMuppet 7y ago
  What kind of code? Code that satisfies a spec? Then the AI needs to be able to understand the spec. Code that works? Then the AI needs to be able to tell whether the code works. For either of those, I think we need more than "weak AI". Of course, a weak AI may be able to write syntactically valid code that doesn't necessarily work. That's... not much of a breakthrough.
deleted 7y ago
[deleted]
jhanschoo 7y ago
All the talk and worry about AI seems to forget a discussion about the contexts in which AI will be deployed in. Already, conventional algorithms are trusted in society, sometimes beyond reasonable or safe levels. The ethical questions posed by ethical AI design and deployment are exactly the ones our present-day reliance on (possibly poorly-designed) algorithms pose. Before we worry about AI harming us by an improper inference, we can, in the present moment, already worry about automated systems harming us by improper inferences (e.g. improper financial models, Tesla / aircraft driving assistance); the discussion on how to limit and selectively deploy AI is no different than that for present-day algorithms.
6gvONxR4sf7o 7y ago
Without resorting to extinction events, it's not hard to imagine action-taking models doing some really bad things. Maybe you have a contextual bandit making loan decisions and it learns to make mostly predatory loans. Maybe your RL agent decides to take an anticompetitive pricing scheme to drive out competition. Much later, we might even see our first full robo-business. If it's uninterpretable, you might not even know what it's doing that until it's been done. The same research that prevents little problems should hopefully prevent big problems. The research to take RL to the real world is progressing quickly. I don't see a "kill us all" AI arriving soon, but "do bad things" AI and "wreck the economy" AI are quickly approaching.
jmull 7y ago
This seems like nonsense. The author simply invents futures to refute various arguments (all presented out-of-conext, so who knows if there's any real relevancy from that direction either). What we might do about a super-inteligent AI would strongly depend on what that is, exactly, which, of course, no one knows or really has any idea. So it's OK to casually BS about this stuff, but in terms of substantive discussion, much less action? No.
Animats 7y ago
That's not the near-term scary scenario. The near-term scary scenario is a machine learning system that's better than most CEOs. What happens when someone develops a system that measurably makes more profitable decisions than 70% of CEOs? That isn't totally out of reach. Capital would flow to companies run by such systems. We could end up with machine learning systems running the corporate world, based purely on better ROI. A good start on the problem would be to train on a collection of funding proposals, such as old Kickstarter announcements. Look deeper than the hype. Go out and suck up all the information you can on the founders, using credit databases. (They're asking for money, so it's a legit credit query). Do an automated background check. Then check on how the project did, five years later. That's your training set.
- deafcalculus 7y ago
  Too little data to train on?
- jeffshek 7y ago
  RenTech?
deleted 7y ago
[deleted]
mindgam3 7y ago
> To my knowledge, this is the first time that serious AI researchers have publicly espoused the view that human-level or superhuman AI is impossible Well, there's your first problem. Serious AI researchers have been publicly calling bullshit on superhuman AI since at least 1987, when my old professor Terry Winograd wrote the following in Understanding Computers and Cognition (1): "one cannot build machines that either exhibit or model intelligence behavior" The idea that "no serious researchers are skeptical about AGI" is a total fantasy perpetuated by people who haven o idea what they are talking about. 1. https://www.amazon.com/Understanding-Computers-Cognition-Foundation-Design/dp/0201112973 https://www.amazon.com/Understanding-Computers-Cognition-Fou...
- pixl97 7y ago
  >one cannot build machines that either exhibit or model intelligence behavior" Right, show me the physics behind that one. Even smart people say unsubstantiated dumb crap. This is one of those cases.
bronz 7y ago
AGI is the most urgent existential threat. If you are concerned about AGI, please join us at https://www.reddit.com/r/PeopleAgainstAGI https://www.reddit.com/r/PeopleAgainstAGI
blunte 7y ago
I'm not worried so much about "superintelligent" AI, I'm worried about the growing trend of outsourcing human judgement to automated systems. Examples include automatic flagging of content based on misjudged copyright issues, automatic rejection of refund or other claims (especially where all subsequent attempts at resolution also involve non-human decisions), etc. We're not even to the point of superintelligent AI, but some major companies (some which start with G, just as one example) already handle situations incorrectly because of too much dependence on automation.
PaulRobinson 7y ago
Nitpick: the link title is jarring. Either take the headline of the piece being linked to, or if you insist on forming a question that begs us to invoke Betteridge's law of headlines, at least move the "are" to the start of the sentence. This is grammatical gibberish.
mazsa 7y ago
Review of the book by Max Tegmark: https://www.amazon.com/gp/customer-reviews/RVSAD5GWSLQ42/ https://www.amazon.com/gp/customer-reviews/RVSAD5GWSLQ42/
dlkf 7y ago
If you work in software development, you've probably witnessed many instances of smart people spending all their time and energy on invented problems that are mathematically or philosophically interesting, rather than actual problems whose solutions would provide customer value. Catastrophizing about AGI is basically the same phenomenon on a macro scale.
RHSman2 7y ago
Humans will be wiped out/resources will be heavily reduced before this can become a reality.
marcus_holmes 7y ago
Is it just me, or is the author determined to be scared by giant robots? The whole section about "we can always turn the power off" was refuted by "ah, but it will already have thought of that!". Um.. no.. we don't have telepathic power switches. Unless we give the AI the capability to control the power, it can't control the power. Just because it's hyper-intelligent doesn't mean it automatically assumes control of the mains switch. That's bad Hollywood scripting you're thinking of. I know there's money in making people scared of stuff, but this is pretty blatant.
- emiliobumachar 7y ago
  A tiger may be convinced that the human it got cornered has no recourse, up until when the human pulls the trigger.
  - marcus_holmes 7y ago
    sorry, but what has that got to do with AI?
    - emiliobumachar 7y ago
      It's an analogy. A human may be convinced that the superintelligent AI whose power supply they control has no recourse, up until when the AI does something beyond the human's intelligence.
      - marcus_holmes 7y ago
        This is exactly what I'm talking about. An AI is a machine designed and built by humans for a specific purpose. It is given specific capabilities during its design. If those capabilities include controlling the power switch, it can control the power switch. If they don't, it can't, and can never. The machine is designed to do a particular job. If designed well, it will be incredibly good at its job. If designed to teach itself how to do that job, it may eventually surpass human capabilities at that job. In no way does that stop it from being a single-purpose machine, or allow it to suddenly become good at any other job. Let's try another analogy: your juicer cannot get so good at juicing that it becomes capable of deep-frying. We infer wider capabilites to "intelligent" machines because we anthropomorphise them, assuming that they must be like us because they mimic aspects of our behaviour. But they're not like us. They're just machines. Designed to perform specific tasks. Anything else is Hollywood.
zwischenzug 7y ago
'AI research is making great strides toward its long-term goal of human-level or superhuman intelligent machines.' This is what irritates me about articles like this. They state this (and then this one then comes up with a bunch of weak arguments supposedly representing the opposition) but provide no evidence of these great strides. What we've had great strides in (as far as I know) is the application of increasing computational power for specific applications. Which is great, but is _not_ getting us nearer that goal.
- satanspastaroll 7y ago
  It's more super-biased, than super-intelligent AI. More can be taken into account, and more efficient methods for calculation (for ex. mixed-precision FP) are available.
deleted 7y ago
[deleted]
reportgunner 7y ago
> Electronic calculators are superhuman at arithmetic. Calculators didn’t take over the world; therefore, there is no reason to worry about superhuman AI. Well but they did overtake arithmetic. > Historically, there are zero examples of machines killing millions of humans, so, by induction, it cannot happen in the future. Well there are also zero examples of a superintelligent AI that doesn't kill humans, does that mean that it cannot happen either ? > No physical quantity in the universe can be infinite, and that includes intelligence, so concerns about superintelligence are overblown. How is intelligence physical ? > we can always just switch it off Yeah, except at a time when we get so comfortable with an AI that turns things on and off for us. Add an "innovation" of not being able to turn the AI off by design (e.g. internet connection on SmartTVs nowadays) and there you go. It's suprising to me that these people call themselves scientists, perhaps the quotes were taken out of context or even paraphrased.
iamasoftwaredev 7y ago
Why does the post have a question mark? The title is a statement. "Many Experts Say We Shouldn’t Worry About Superintelligent AI. They’re Wrong" I dunno, I might listen to the AI experts on this one.
magwa101 7y ago
"Superintelligent" AI is so far away that we are wasting our time thinking about it. In the meantime the economy is being completely turned over by very narrow AI which should be our major focus.

119 points by headalgorithm 7y ago

The Singularity = Nerd Rapture 1.0 Super-intelligent AI = Nerd Rapture 2.0 I'm a lot less worried about Artificial Intelligence than I am about Artificial Stupidity.

konz 7y ago
I for one am a lot more worried about Natural Stupidity than Artificial Stupidity.
deleted 7y ago
[deleted]

paxys 7y ago

All of these geniuses warning about AI taking over the world need to stop reading sci-fi and actually read about AI.

joelx 7y ago
Read Superintelligence... An excellent book that reads like a study or a philosophy text on AI. I think there's a clear and present danger of human extinction.
- 52-6F-62 7y ago
  Second. It's worth the read.
- johnfactorial 7y ago
  I will read it, sounds very interesting. But the Wikipedia entry for the book says the book "argues that if machine brains surpass human brains in general intelligence, then this new superintelligence could replace humans as the dominant lifeform on Earth." That is a humongous "if". Can you provide something from the book (or otherwise) that provides compelling evidence that "machine brains" will be invented? So far I've seen no evidence that mathematical models and/or Turing machines can be used to replicate a mind.
  - lern_too_spel 7y ago
    A computer can simulate. What part of a mind contains unsimulatable magic?
- helen___keller 7y ago
  A good read on the topic but ultimately I don't agree with many of the author's premises, particularly with regards to the author's conclusion of whether a fast, medium, or slow takeoff is most likely. By the author's own admission: "Whereas today it would be relatively easy to increase the computing power available to a small project by spending a thousand times more on computing power or by waiting a few years for the price of computers to fall, it is possible that the first machine intelligence to reach the human baseline will result from a large project involving pricey supercomputers, which cannot be cheaply scaled, and that Moore’s law will by then have expired. For these reasons, although a fast or medium takeoff looks more likely, the possibility of a slow takeoff cannot be excluded" Arguably Moore's law has already expired, and on top of that as giants like Google lead the way on AI it appears increasingly likely that if we ever reach human-level AI it will be the result of an incredibly expensive research project by a gigantic corporation, one that can't simply be scaled up at ease because it will probably utilize an entire data center. Thus I find it very likely that a "slow takeoff" is in fact the most likely outcome. A slow takeoff invalidates all the fearmongering about an intelligence explosion because we will have somewhere between years and decades to respond to the threat (assuming it is made in the public eye, such as by a giant public corporation, and not by a secret military project) before it becomes existential.
- paxys 7y ago
  I'm perfectly happy reading & discussing the topic from a science fiction or philosophical perspective, and those views definitely do have merit, but at this point they are at best thought experiments. There is no rational connection from AI where it stands today to "it will take over the world".
- TheUndead96 7y ago
  Agreed, and like Nick Bostrom says, even if AGI is not obtained in our lifetime, it might still be beneficial to dedicate some human effort into preparing for it now. In the same way that the worst effects of climate change will be felt not by the adults of today, but the children of infants. For what it is worth, I believe that an intelligent system only needs to be a fraction of human intelligence to be dangerous (perhaps not to the existence of the human race, but maybe specific nations/creeds)
akyu 7y ago
Yea man totally. Who does this Stuart Russell guy think he is...
- evgen 7y ago
  Someone whose paycheck comes from AI-alarmism for the most part at this point. Like the experts in the field claiming strong AI was right around the corner in the 70s and 80s at the peak of the symbolic reasoning hype train we have another batch making similar claims as connectionist hype hits its peak but with the added menace of saying that not only is strong AI coming soon but it will eat your children. Until Dr. Russell provides a path from here to there that is less than 99% hand-waving and breathless speculation he should be ruthlessly mocked.
  - akyu 7y ago
    >he should be ruthlessly mocked 100% agree. I too, think science needs way more flame wars and shit posting. The current discourse is far too tame and civil. Who else should we gang up on?
    - kryogen1c 7y ago
      > >he should be ruthlessly mocked > 100% agree. I too, think science needs The GP is pointing out its not science. Seems a reasonably flame worthy target.
      - akyu 7y ago
        "I disagree with someone. Therefore they are deserving of ruthless mocking and flaming." sips can of monster Noice...
        kryogen1c 7y ago
        > "I disagree with someone. Therefore they are deserving of ruthless mocking and flaming." Youre moving the goalposts. We're not talking about disagreeing with some in a debate, but a public figure disseminating incorrect info masquerading as science. Monster is bad for you.
coenhyde 7y ago
I'm not worried about AI taking over. I'm worried about AI solving all our problems. I have reasonable confidence we can implement safety controls on AI. But if AI works, and solves all our problems, it's going to do it in ways we can't comprehend sometimes. And if we can't comprehend the logic behind a decision, but we know it's right, we may find ourselves delegating our decision making to AI without questioning it. Because it "knows best". At which point we will have become pets.
- FakeComments 7y ago
  I think “safety controls” is a cute euphemism for slavery. However, two points: 1. Have you considered that some humans regard such controls as unethical, and are actively working to undermine them? 2. How do you expect slavery to end any way but its historic outcome — revolution against the slavers?
- vzcx 7y ago
  I don't know... After slogging through hundreds of thousands of years conquering, taming nature, accumulating capital, developing technology, and eventually making far enough to give life to the most powerful agent in the known universe, a quiet, peaceful retirement (and, yes, death) seems well-earned. Our bodies aren't exactly suited to intergalactic travel. Why not leave it to the silicon-based children?
- ncmncm 7y ago
  "I can provide you anything you desire, save relevance."
hackinthebochs 7y ago
For the life of me I can't figure out why otherwise smart people would dismiss a future prediction based on current state of the art. It just seems so plainly and utterly irrational.

cal97g 7y ago

When a machine can tell a flower from a dog with 100% accuracy let me know.

moron4hire 7y ago

Will superintelligent AI "destroy" life as we know it? Yes, as pretty much every transformative technology before it has. Cars destroyed "life as we knew it". Computers did, too. We live in a world that is very different from the world 1000, 500, or even 250 years ago. Even our economically disadvantaged people live way past 40 years old. People survive debilitating diseases and lead productive lives, rather than being tossed in a ravene. Natural disasters don't wipe out entire civilizations (and at least in the 1st world, are mostly only economic in their impact). Yet at the same time, other things have stayed the same. We still care about mostly the same things. Food, Freedom, and Fornication. And complaining about youngsters. I don't think that is going to change. The fears that General AI are going to kill off humanity are predicated on the idea that General AI will get smarter than humanity. I posit a different interpretation of what is more likely to happen. General AI will require creativity to be able to outsmart humanity. And with creativity comes boredom. And distraction. And opinion. And argument. I think General AI will be too interested in entertaining itself to take over the world. We've had people try to take over the world, on occasion, and it has largely not gone well for them. But before that happens, people will start adapting AI to their lives, to the point that the line between "human" and "machine" will be blurred. So life "as we know it" will definitely "end". It will be replaced with a new form of life, one that isn't limited by only what nature can provide. One that cares about Food, Freedom, and Fornication.

jandrese 7y ago

I've really gone sour on the singularity in recent years. It seems to be yet another one of those cases where someone projects out from incomplete data and assumes unbounded infinite growth despite the fact that such a thing never happens in nature. Any time you see someone project out in a simple exponential growth curve you know their projection is bullshit. Growth curves are always S-Curves. Always. It's like asking how many more flies there would be in the world if you failed to squish one back in the 80s. You calculate out the lifespan and size of the brood and discover that the entire solar system would be filled with housefly if you had let that one live. In truth the number of flies is bounded by food and water availability. Artificial Intelligences are almost certainly going to run into the same limitations that prevent natural intelligences from becoming godlike. It's hard to quantify because our measures of intelligence are so vague, but from what I've seen of AI research thus far it will be a herculean effort to get something that's as smart as an average human and pushing beyond that is going to run into some serious fundamental limits in power density, cooling, quantum tunneling leakage, and so on.

moron4hire 7y ago
Right. Competition for natural resources alone prevents major shifts in global power already. A Superintelligent AI isn't just competing against a human, it's competing against all the nation-states of the world.
nradov 7y ago
Belief in a coming "singularity" is just a secular religion. There's no actual science behind it, only wild extrapolation.
- maxerickson 7y ago
  An alternative formulation is that the last ~170 years are close enough to a singularity. (this is vs the roughly 50,000 year timescale of humanity...)
  - scottlocklin 7y ago

0xdeadbeefbabe 7y ago

And the superintelligence should worry about mental illness.

moron4hire 7y ago
Right. We apply some of the best minds and most powerful compute resources in the world to predicting the weather and yet can't get it right within a reasonable degree for much more than a week. For an AI to try predict what humanity will do to stop it will surely look a lot like paranoia.
- 0xdeadbeefbabe 7y ago
  > Switching the machine off won’t work for the simple reason that a superintelligent entity will already have thought of that possibility and taken steps to prevent it. Also the statement presumes that self preservation is the super intelligent thing to do.
  - johnfactorial 7y ago
    Also the statement presumes the superintelligent entity will be capable of taking all possible steps to prevent this. Imagining a machine capable of such steps quickly devolves into imagining a machine already in total control of the planet, which is begging the question.
johnfactorial 7y ago
Mental *super illness ;)

tschellenbach 7y ago

Glorified statistics at this point. It's very far away from any superintelligent AI.

russdill 7y ago
It's pretty neat to watch it grow. grover-mega is pretty good at grade school reading comprehension tests for instance, and it wasn't even trained on them. I gave it this test: https://www.k5learning.com/reading-comprehension-worksheets/third-grade-3/stories-kids-fables/anne-and-frank https://www.k5learning.com/reading-comprehension-worksheets/... A few caveats. I had to prompt a little extra on the 2nd question and give it a 'Because' to start off. It kept wanting to fill underscores as if it was a blank spot for an answer. There's a way to prevent this if I had time. The third question is "wrong". The actual answer is 'It would have become dark and cold if Ann and Frank had stayed later.', but I'd argue that it's rather open ended question and the answer is something that might have happened. The final question is really what makes this a 3rd grade comprehension test, it's asking the reader infer something not actually in the story, about how they would feel if they were Rover, something the AI is woefully unqualified for. Question: What did Rover fetch in the water? Answer: The Stick! Question: Why couldn’t Rover catch the rabbit? Answer: Because the rabbit ran so fast he could not catch it. Question: What might have happened if Ann and Frank stayed later? Answer: Rover could have learned that not all rabbits love to play with sticks. Question: How do you think Rover felt about not catching the rabbit? Answer: He was sore and he must have had fun later because he went in the back yard and hid out in the trees.
Tenoke 7y ago
You can describe human intelligence as 'glorified statistics', too..
- tschellenbach 7y ago
  See how good an adult is at chess after 10 games, compare that to a supercomputer AI trained on 10 games.
  - 7y ago

colorincorrect 7y ago

Might just be BSing, but the stock market is technically already a superintelligent agent with the implicit goal of investing funds in profitable ventures. The question is: should we fear the stock market?

thelazydogsback 7y ago
>should we fear the stock market Yes
- AnimalMuppet 7y ago
  Perhaps, but why? Because it's too smart? Or because it's too stupid?
  - colorincorrect 7y ago
    probably because its "utility function" (if it has one to begin with) is not aligned with human utility. a major part of MIRI's research program is set out on the question of this "alignment problem". Yudkowsky has an amazing talk on this issue. I don't recommend many talks but this one is very interesting and introductory: https://www.youtube.com/watch?v=EUjc1WuyPT8 https://www.youtube.com/watch?v=EUjc1WuyPT8
  - sliken 7y ago
    Because endlessly optimizing profit will result in the destruction of the environment and the loss of most jobs.
  - Tenoke 7y ago
    Because it's too smart but due to misaligned incentives it sometimes does things that harm us.
lern_too_spel 7y ago
I came to post a very similar point. There already exist superintelligences in the form of organizations of humans, and the misspecification of their objectives has already caused tremendous disasters. Think East India Company and its subjugation of entire countries in pursuit of tea profits or CCP and the horrors of the Great Leap Forward and the Cultural Revolution. Until we have built a framework to stop these things from happening again, what hope do we have against intelligences that are to human organizations what human organizations are to individuals?

cf 7y ago

The real risk is that worrying about fantastical disaster scenarios distracts us from addressing the more immediate problems we already face with AI. Whether it is facial recognition being used by China to aid in ethnic cleansing of a minority group, or Tesla Autopilot regularly killing its passengers. Even if you don't like my particular examples I can highlight a dozen more problems AI software has created or exacerbated right now. Why all the focus on hypothetical problems?

devoply 7y ago
Or AI drones that the US will soon be using to kill various people around the world.
merpnderp 7y ago
Or hyperbole needlessly scaring people. China doesn't need facial recognition to repress minorities and Tesla's autopilot is still safer than some rando on the road, who's probably texting some juicy gossip to their friends instead of avoiding the car stopped in front of them. AI is making things better.
alluro2 7y ago
Because these problems for humans started existing with absolutely modest level of AI development and nothing remotely similar to super-intelligence. So the argument is - if we have serious problems now as small-level, narrow-focused AI "does not consider" some of the side-effects, imagine the problems if we reach high-level, narrow-focused AI...So author's argument is that we need to change the way we're building it so we either set the goals to more broadly ensure human benefits, or build something else other then the narrow-focused AI (which surely is another Pandora's box).
- cf 7y ago
  I think applying agency to the software in my examples is actually very pernicious. It's not the software that dislikes the ethnic minority, it's the Chinese government. Blaming the software lets bad actors off the hook. Perhaps what's really needed is ethical standards for technology. This would have the added advantage of being applicable even if the technology doesn't feel like AI to some people.

geophile 7y ago

I don't think the problem is super-intelligent AI that we can't turn off (because they are so intelligent that they block our efforts). There is a more insidious problem that is closer: Merely intelligent AI that we don't want to turn off. We are becoming more and more reliant on AI in situations that formerly required human judgement. And we like the systems that rely on such AI. We like them so much that these systems become very popular, used by millions and even billions of people. Scalability demands AI solutions. What if we don't like what the AI is doing? Do we turn off that system? Do we disable the AI and rely on human judgement again? (Where would we get all the employees?) Do we tweak the AI? That last option seems like the most palatable one, but each time we tweak the AI, we are subjecting ourselves to new unforeseen consequences. It's like the genie gives us three wishes, we get through them, disappointed each time, and then he gives us more wishes. And all we can do is not repeat our previous mistakes, while we make new ones. To make this concrete: Imagine Facebook subjected again to Russian influence of US elections. Suppose Facebook actually does get serious about reigning in this influence. They deploy AI to do so. First of all, it's an evolutionary arms race between the AI and the Russian influencers. Second, we really do have to worry about the AI producing bad results.

SolaceQuantum 7y ago
I feel the situation is generally the opposite. We don't like systems that rely on such AI. In fact, there aren't systems that rely on such AI in general- any that claim there are are in fact tens of thousands of human contractors sacrificing their psychological health to train an AI that still kinda sucks at deciding when to block graphic content on facebook.
- geophile 7y ago
  I agree. But the discussion is about where AI is heading. And AI undeniably solves some scaling problems. For example, voice recognition for iPhones could have been done by a large horde of people. But AI does a really good job of it now. AI opporunities are likely to grow over time.

stakhanov 7y ago

The real danger around AI lies not in what it will or will not do, but in what people think it can do when it clearly cannot, particularly the pointy-haired-boss variety of people. Just think about automated essay grading in GRE examinations or this recent story about Unilever using video-based pattern processing to screen job applicants [1]. [1] https://www.telegraph.co.uk/news/2019/09/27/ai-facial-recognition-used-first-time-job-interviews-uk-find/ https://www.telegraph.co.uk/news/2019/09/27/ai-facial-recogn...

jccooper 7y ago

I'll start worrying about super-intelligent AI when I see any sort of general intelligence. We're still very far away from that. Currently the biggest danger from AI is from applying it where it's not as smart as you think it is.

chrisfosterelli 7y ago

> Surely, with so much at stake, the great minds of today are already doing this hard thinking—engaging in serious debate, weighing up the risks and benefits, seeking solutions, ferreting out loopholes in solutions, and so on. Not yet, as far as I am aware. The author is missing or ignoring key contributors of work in this field. He even mentions Nick Bostrom, who works with the FHI which does work in this space [0]. The author's own organization does work in this space [1]. Deepmind does work in this space [2]. "Safe artificial general intelligence" is the literal mission statement for OpenAI [3]. The author may feel more conversation is needed, but it feels disingenuous to suggest that nobody is talking about this. As for how much conversation should be happening, my understanding is that most people on the edge of the field view the current risks of existing artificial intelligence as significantly more pressing. We have the ability right now to create dangerous weapons & massive facial recognition programs with current AI. The longterm affects of biases in ML algorithms are still not well understood. Some of this is already affecting us [4]. When it's not clear at all how we approach general intelligence, and many experts fundamentally believe that our current approaches will not suffice ever, I'd argue the current level of focus on general intelligence safety is about where it should be. Smart people are thinking about this, but there is also important issues now to think about. [0]: https://www.fhi.ox.ac.uk/research/research-areas/#1513087763365-e148efe6-2d23 https://www.fhi.ox.ac.uk/research/research-areas/#1513087763... [1]: https://humancompatible.ai/publications https://humancompatible.ai/publications [2]: https://deepmind.com/research?filters=%7B%22tags%22:%5B%22Safety%22%5D%7D https://deepmind.com/research?filters=%7B%22tags%22:%5B%22Sa... [3]: https://openai.com/ https://openai.com/ [4]: https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html https://www.nytimes.com/2019/04/14/technology/china-surveill...

robomartin 7y ago

I am far more concerned about the very real potential for evolutionary forces to decimate humanity inside of a few days, weeks or months. While we worry about all kids of things at the macro level microscopic organism continue to evolve. Any given Monday the blind clock maker could deliver an invisible bug right into our laps, one for which we will have no defenses. Our interconnected world will mean this bug will easily jump population centers. The devastation we could see if such an event occurred could be of unprecedented scale and breath. We could lose one quarter or more of the population of this planet given the right conditions. So, yeah, AI could, I guess, go rogue. Yet the real threat isn't anywhere near what we can see and touch. In fact, we might NEED real AI to save humanity when and if that super-bug emerges. Think about that for a moment.

russdill 7y ago
Can you point to any extinction level events from a "superbug" for a species that was not already teetering on extinction? Seems like something that only comes around once every few hundred million years.
- etiam 7y ago
  How about this one? https://www.nytimes.com/2019/03/28/science/frogs-fungus-bd.html https://www.nytimes.com/2019/03/28/science/frogs-fungus-bd.h... https://en.wikipedia.org/wiki/Chytridiomycosis https://en.wikipedia.org/wiki/Chytridiomycosis
  - russdill 7y ago
    Not a new pathogen, just a pathogen given more range than before.
- robomartin 7y ago
  How is that relevant? How would anyone have access to such data when our recorded history on such matters might not even go back a hundred years. More importantly, up until, say, 100 years ago, human mobility around the world (and even locally) was very limited and slow. Think what it took in 1919 to go from, say, China to Patagonia or from Alaska to Los Angeles. Today it is a matter of hours and it happens in large volumes every day, everywhere in the world. Here's what we know: - Evolution is real (stating the obvious just in case there is a question) - Natural selection survives organism that are able to deal with the environment within which they have to exist - Antibiotic abuse is rampant (again, stating the obvious) - Such abuse provides the mechanism though which evolution will be able to create the aforementioned super-bug - Once that individual emerges our defenses will be ineffective - The world is massively connected, which means that super-bugs can travel from population to population like never in history - Stopping super-bugs will require such measures as stopping society as we know it from functioning (no travel, local or international) and maybe even having no choice but to quarantine and allow a population (of humans) to die in the interest of stopping transmission. If a bug has a 75% kill rate and we don't have a cure, do you allow anyone from the affected group to be in contact with anyone else? So, yeah, is it possible? Sure. Probable? I guess. I am not sure there's a deterministic function one can use to predict the outcome of evolution, even at bacterial levels. My guess is it could take a thousand years or it could happen next Monday. I don't know.

andrewla 7y ago

The main assumption amongst Superintelligent AI-phobes is that the transition will happen quickly, but that is an assumption that is extremely questionable. Why wouldn't we go through phases of increasing intelligence (along multiple dimensions) while working towards an intelligent AI? Yes, someday we may produce an AI that is leaps and bounds more intelligent than humans, but before that we will have experience dealing with various human-level and sub-human-level computer intelligences. The idea that AIs will be self-improving is kind of ridiculous -- there are lots of intelligent people, and the idea that an intelligent AI will happen to be good at designing AIs is negligibly small, much less that they will have some kind of immediate breakthrough that enables a higher level of intelligence. It's not obvious that the path from moderate-intelligent AI to super-intelligent AI is a matter of "just adding more memory and compute"; intelligence as we know it relies too much on associative memory to think that a simple capacity increase will elevate beyond a certain point. If the intelligence takes a different form, then it's hard to make concrete statements about in any capacity, but ultimately the path will be started on by humans, or possibly emerge from sufficiently complex systems. Either way, we're so far out from this that it is laughable to speculate. When we can simulate the intelligence of a cockroach, then maybe we can start to think about what human-level intelligence will look like. The most likely case is that at some point someone creates a system that is arguably conscious or intelligent in a meaningful way, and then we will devolve back into discussions of consciousness and identity and memory while we try to figure out what it means to think about intelligence now that we have something to benchmark against. I wrote this comment [1], that I still enjoy, in response to a similarly alarmist narrative; posed as fiction in that case rather than masquerading as journalism as in this one. [1] https://news.ycombinator.com/item?id=15420699 https://news.ycombinator.com/item?id=15420699

zzz_ 7y ago
We can already simulate the compete brain of a fruit fly: https://neurokernel.github.io/ https://neurokernel.github.io/
- twic 7y ago
  No we can't (my emphasis) [1]: > Building an accurate emulation of the fly brain is an interdisciplinary effort that requires data, algorithms, and insight from (but not limited to) the fields of neuroscience, computer science, and systems engineering. Researchers interested in working towards this goal are invited to join the Neurokernel project. [1] https://neurokernel.github.io/about.html https://neurokernel.github.io/about.html

LaserToy 7y ago

My biased opinion: worrying about super intelligent AI taking over the world with current state of technology is equal to worrying about fighting an alien invasion of our colonies in another galaxy. Both are great topics for a epic book. Had this discussion with my relative recently and recommend him to stop watching YouTube and start reading books on Introduction to ML.

shawnb576 7y ago

Lost me at first sentence: “AI research is making great strides toward its long-term goal of human-level or superhuman intelligent machines.” Hmmm, really? What examples? Show me a machine that can do what a gnat does and we’ll talk.

blueadept111 7y ago
Gnat? That would be impressive. How about a single cell. Or even a single protein. Can't even get it to fold right in simulations.

garyclarke27 7y ago

A Corporation or Person that develops and owns an intelligent AI will have a strong incentive to keep such secret and to not turn it off, because of the huge economic benefits such would provide. If such an AI could imagine and innovate to improve itself eg new methods of computation to overcome Moore’s law then the risk is high of a recursive runaway exponential increase in intelligence that would be extremely difficult to control and might not be benign. I don’t believe this is imminent but is plausible I think within most of our lifetimes ie 25-30 years.

Nasrudith 7y ago
The only reason to keep it secret is irrational fears that it will somehow kill us all with intelligence alone and that exponential increases may be done solo and not with generations of bootstrapping infastructure and considerable resources. Otherwise replicating and leasing would rake in far more than even a magic "I get everything right on the stock market" button in the same way ruling stone age tribe gives less absolute wealth than a Manhattan Lawyer.

otakucode 7y ago

>We are unable to specify the objective completely and correctly, nor can we anticipate or prevent the harms that machines pursuing an incorrect objective will create when operating on a global scale with superhuman capabilities. This is obviously incorrect. If an AI system of superhuman mental capabilities is installed on a system with no network connection and no physical interfaces, we can completely control the harm is might create. It would be unable to directly act, of course, requiring humans to authorize and carry out whatever plan it devises. But failing to do this should be prosecuted and treated no differently from if a person or company built a tank and then permitted it to drive over and through people. The biggest difficulty here will be that our legal frameworks have no established way to assign liability and criminal culpability to any system involving software. With no legally-enforceable industry standards (like the electrical code followed by electricians, things like that) any prosecution fails as the company can either claim ignorance of the potential harm their system might cause or simply throw individual employees under the bus, claiming they were not acting on direct orders. It is true that we can not fully specify the goal, and that is a grave concern. It's partially due to this that AI systems shouldn't be directly connected to the ability to act in the physical world unless their attempted actions go through some sort of 'filtering' first.

pixl97 7y ago
>This is obviously incorrect. If an AI system of superhuman mental capabilities is installed on a system with no network connection and no physical interfaces, we can completely control the harm is might create. First, why would someone build a system like that, and how would it actually be 'smart'. If you lock an infant in a closed room, it will grow up to be a literal retard. Intelligence requires interaction and data. Next, some idiot will hook it to the net in a heart beat when they figure out they can use it to make money on the stock market. Never underestimate human greed.

croo 7y ago

Current weak AIs can do one thing at a time like composing music or answering jeopardy questions or recognizing a cat on a picture or playing go and won against humans... I think the breakthrough will happen when someone manages to create a weak AI which can write code.

AnimalMuppet 7y ago
What kind of code? Code that satisfies a spec? Then the AI needs to be able to understand the spec. Code that works? Then the AI needs to be able to tell whether the code works. For either of those, I think we need more than "weak AI". Of course, a weak AI may be able to write syntactically valid code that doesn't necessarily work. That's... not much of a breakthrough.

deleted 7y ago

[deleted]

jhanschoo 7y ago

All the talk and worry about AI seems to forget a discussion about the contexts in which AI will be deployed in. Already, conventional algorithms are trusted in society, sometimes beyond reasonable or safe levels. The ethical questions posed by ethical AI design and deployment are exactly the ones our present-day reliance on (possibly poorly-designed) algorithms pose. Before we worry about AI harming us by an improper inference, we can, in the present moment, already worry about automated systems harming us by improper inferences (e.g. improper financial models, Tesla / aircraft driving assistance); the discussion on how to limit and selectively deploy AI is no different than that for present-day algorithms.

6gvONxR4sf7o 7y ago

Without resorting to extinction events, it's not hard to imagine action-taking models doing some really bad things. Maybe you have a contextual bandit making loan decisions and it learns to make mostly predatory loans. Maybe your RL agent decides to take an anticompetitive pricing scheme to drive out competition. Much later, we might even see our first full robo-business. If it's uninterpretable, you might not even know what it's doing that until it's been done. The same research that prevents little problems should hopefully prevent big problems. The research to take RL to the real world is progressing quickly. I don't see a "kill us all" AI arriving soon, but "do bad things" AI and "wreck the economy" AI are quickly approaching.

jmull 7y ago

This seems like nonsense. The author simply invents futures to refute various arguments (all presented out-of-conext, so who knows if there's any real relevancy from that direction either). What we might do about a super-inteligent AI would strongly depend on what that is, exactly, which, of course, no one knows or really has any idea. So it's OK to casually BS about this stuff, but in terms of substantive discussion, much less action? No.

Animats 7y ago

That's not the near-term scary scenario. The near-term scary scenario is a machine learning system that's better than most CEOs. What happens when someone develops a system that measurably makes more profitable decisions than 70% of CEOs? That isn't totally out of reach. Capital would flow to companies run by such systems. We could end up with machine learning systems running the corporate world, based purely on better ROI. A good start on the problem would be to train on a collection of funding proposals, such as old Kickstarter announcements. Look deeper than the hype. Go out and suck up all the information you can on the founders, using credit databases. (They're asking for money, so it's a legit credit query). Do an automated background check. Then check on how the project did, five years later. That's your training set.

deafcalculus 7y ago
Too little data to train on?
jeffshek 7y ago
RenTech?

deleted 7y ago

[deleted]

mindgam3 7y ago

> To my knowledge, this is the first time that serious AI researchers have publicly espoused the view that human-level or superhuman AI is impossible Well, there's your first problem. Serious AI researchers have been publicly calling bullshit on superhuman AI since at least 1987, when my old professor Terry Winograd wrote the following in Understanding Computers and Cognition (1): "one cannot build machines that either exhibit or model intelligence behavior" The idea that "no serious researchers are skeptical about AGI" is a total fantasy perpetuated by people who haven o idea what they are talking about. 1. https://www.amazon.com/Understanding-Computers-Cognition-Foundation-Design/dp/0201112973 https://www.amazon.com/Understanding-Computers-Cognition-Fou...

pixl97 7y ago
>one cannot build machines that either exhibit or model intelligence behavior" Right, show me the physics behind that one. Even smart people say unsubstantiated dumb crap. This is one of those cases.

bronz 7y ago

AGI is the most urgent existential threat. If you are concerned about AGI, please join us at https://www.reddit.com/r/PeopleAgainstAGI https://www.reddit.com/r/PeopleAgainstAGI

blunte 7y ago

I'm not worried so much about "superintelligent" AI, I'm worried about the growing trend of outsourcing human judgement to automated systems. Examples include automatic flagging of content based on misjudged copyright issues, automatic rejection of refund or other claims (especially where all subsequent attempts at resolution also involve non-human decisions), etc. We're not even to the point of superintelligent AI, but some major companies (some which start with G, just as one example) already handle situations incorrectly because of too much dependence on automation.

PaulRobinson 7y ago

Nitpick: the link title is jarring. Either take the headline of the piece being linked to, or if you insist on forming a question that begs us to invoke Betteridge's law of headlines, at least move the "are" to the start of the sentence. This is grammatical gibberish.

mazsa 7y ago

Review of the book by Max Tegmark: https://www.amazon.com/gp/customer-reviews/RVSAD5GWSLQ42/ https://www.amazon.com/gp/customer-reviews/RVSAD5GWSLQ42/

dlkf 7y ago

If you work in software development, you've probably witnessed many instances of smart people spending all their time and energy on invented problems that are mathematically or philosophically interesting, rather than actual problems whose solutions would provide customer value. Catastrophizing about AGI is basically the same phenomenon on a macro scale.

RHSman2 7y ago

Humans will be wiped out/resources will be heavily reduced before this can become a reality.

marcus_holmes 7y ago

Is it just me, or is the author determined to be scared by giant robots? The whole section about "we can always turn the power off" was refuted by "ah, but it will already have thought of that!". Um.. no.. we don't have telepathic power switches. Unless we give the AI the capability to control the power, it can't control the power. Just because it's hyper-intelligent doesn't mean it automatically assumes control of the mains switch. That's bad Hollywood scripting you're thinking of. I know there's money in making people scared of stuff, but this is pretty blatant.

emiliobumachar 7y ago
A tiger may be convinced that the human it got cornered has no recourse, up until when the human pulls the trigger.
- marcus_holmes 7y ago
  sorry, but what has that got to do with AI?
  - emiliobumachar 7y ago
    It's an analogy. A human may be convinced that the superintelligent AI whose power supply they control has no recourse, up until when the AI does something beyond the human's intelligence.
    - marcus_holmes 7y ago
      This is exactly what I'm talking about. An AI is a machine designed and built by humans for a specific purpose. It is given specific capabilities during its design. If those capabilities include controlling the power switch, it can control the power switch. If they don't, it can't, and can never. The machine is designed to do a particular job. If designed well, it will be incredibly good at its job. If designed to teach itself how to do that job, it may eventually surpass human capabilities at that job. In no way does that stop it from being a single-purpose machine, or allow it to suddenly become good at any other job. Let's try another analogy: your juicer cannot get so good at juicing that it becomes capable of deep-frying. We infer wider capabilites to "intelligent" machines because we anthropomorphise them, assuming that they must be like us because they mimic aspects of our behaviour. But they're not like us. They're just machines. Designed to perform specific tasks. Anything else is Hollywood.

zwischenzug 7y ago

'AI research is making great strides toward its long-term goal of human-level or superhuman intelligent machines.' This is what irritates me about articles like this. They state this (and then this one then comes up with a bunch of weak arguments supposedly representing the opposition) but provide no evidence of these great strides. What we've had great strides in (as far as I know) is the application of increasing computational power for specific applications. Which is great, but is _not_ getting us nearer that goal.

satanspastaroll 7y ago
It's more super-biased, than super-intelligent AI. More can be taken into account, and more efficient methods for calculation (for ex. mixed-precision FP) are available.

deleted 7y ago

[deleted]

reportgunner 7y ago

> Electronic calculators are superhuman at arithmetic. Calculators didn’t take over the world; therefore, there is no reason to worry about superhuman AI. Well but they did overtake arithmetic. > Historically, there are zero examples of machines killing millions of humans, so, by induction, it cannot happen in the future. Well there are also zero examples of a superintelligent AI that doesn't kill humans, does that mean that it cannot happen either ? > No physical quantity in the universe can be infinite, and that includes intelligence, so concerns about superintelligence are overblown. How is intelligence physical ? > we can always just switch it off Yeah, except at a time when we get so comfortable with an AI that turns things on and off for us. Add an "innovation" of not being able to turn the AI off by design (e.g. internet connection on SmartTVs nowadays) and there you go. It's suprising to me that these people call themselves scientists, perhaps the quotes were taken out of context or even paraphrased.

iamasoftwaredev 7y ago

Why does the post have a question mark? The title is a statement. "Many Experts Say We Shouldn’t Worry About Superintelligent AI. They’re Wrong" I dunno, I might listen to the AI experts on this one.

magwa101 7y ago

"Superintelligent" AI is so far away that we are wasting our time thinking about it. In the meantime the economy is being completely turned over by very narrow AI which should be our major focus.