4 ms·
Don't these arguments kind of assume a few things though? Like, 'strong' AI being implemented into things that really don't need it? I mean, the paperclip exa
by CM30 11y ago
Don't these arguments kind of assume a few things though? Like, 'strong' AI being implemented into things that really don't need it? I mean, the paperclip example in the Medium article... did that factory ever need AI involved? Couldn't a simple computer program no more advanced than those of today do the exact same job... without the risk of all that 'thinking and making copies of itself' stuff?
Maybe I'm being naive here, but you don't need strong AI in everything.
And they also kind of assume that there's only one AI at a time (what if two factories are run by AIs designed to build different things, and they both end up in this sort of situation at the same time?), that everything is connected to the internet in some way, that humanity couldn't simply wipe out anything that poses a threat (and no, an AI in a factory environment wouldn't get access to anything that would allow for nuclear/chemical/biological weapons). Just seems like a lot of this speculation is based around a society that's making tons of careless mistakes one after another. But maybe I'm missing something here.
- cousin_it 11y agoDon't focus on one scenario leading to disaster. There are many scenarios pointing roughly in the same direction. Here's the argument in its shortest form: The current situation, where unmodified humans are the smartest creatures around and call the shots, is unstable. It could go several different ways depending on the desires and quirks of the first superhuman intelligence to appear. It could happen via math-based AI, nature-imitating AI, self-improving mind uploads, biological intelligence amplification, or other means. We're making fast progress on all these fronts, so I'd be surprised if at least one of those didn't happen within the next century, possibly much sooner. Since it's an arms race between many competing organizations, it's unrealistic to expect that all superhuman intelligences will be kept powerless. So one way or another, it will happen. How do we ensure that humanity and its values survive the transition?
- click170 11y ago> How do we ensure that humanity and its values survive the transition? I think a better question is do we deserve to.
- MrQuincle 11y agoVery true words. I think it's even dangerous to focus on scenarios that leads to disasters. I'd like to write this up in more detail one time, but it's summarized as a self-fulfilling prophecy. How would evil AI be prevented by a company from Musk, let's call it aiFriend? - Analyse what evil AI would constitute - Build forms of evil AI to understand them better, understand how to counter them - "Offense is the best defence" thoughts might lead to good AI bot development fighting evil AI bots - Development in tools to fool evil AI bots to make them make the wrong decisions - Development of all the human capabilities of deceitfulness to help good AI counter evil AI. Will Musk create evil AI by focusing on the dark side of a technology and bring about unintentionally that what he fears so much?
- soared 11y agoThis comment is straight fear mongering and should not be taken seriously. > Analyse what evil AI would constitute No. There is no such thing as evil AI. Thats implying an AI can be good or evil, which is an absurd starting point. Then to claim someone would build an AI to offensively attack and fight evil AI bots? I can't down vote, so take this comment as a replacement: Get the fuck out of here.
- JoshTriplett 11y ago> I mean, the paperclip example in the Medium article... did that factory ever need AI involved? That example got a bit lost in translation. The standard example of the paperclipping AI is not about building an AI to run a paperclip factory. It's about providing an appropriate value system for an AI, so that it can evaluate courses of action. The standard example: you use a paperclip as an example for an AI as the smallest unit of incremental value, such that having a paperclip is epsilon better than not having a paperclip. The AI files that information away, and tiles the universe with paperclips. Related examples include using a smiling human face as an example of happiness and having the AI tile the universe with the smallest possible object that matches its "smiling human face" recognizer. > And they also kind of assume that there's only one AI at a time Yes. If you're going to build a strong AI, then either you got its value system right, and it will take over the world (in the good, "solves all problems at once" way), or you got its value system wrong, and it will destroy the world. There is no plausible scenario in which we build strong AI (as in, capable of self-improvement to better satisfy its value function) and it doesn't take over the world, or in which it allows other strong AIs to exist that are not effectively subroutines of itself (any AI that shares the same value system is the same AI, and any AI that doesn't share the same value system is too dangerous to exist). > that everything is connected to the internet in some way, that humanity couldn't simply wipe out anything that poses a threat (and no, an AI in a factory environment wouldn't get access to anything that would allow for nuclear/chemical/biological weapons). You can't keep a strong AI capable of self-improvement in a "box". Random example: a security conference this year demonstrated using nothing but DRAM to successfully transmit short-range GSM signals. That's something humans thought up and implemented. That's leaving aside intentional human intervention or development-and-release. Not least of which because a strong AI done right provides a massive reduction in both existential risk for humanity and day-to-day risk for humans.