5 ms·
Ask HN: If AI can potentially wipe us out, why is there a race to achieve it?
I'm not saying that a true AGI will definitely wipe us off the face of the planet, but there are obviously risks involved in such a huge achievement.
Given the not-so-good record of big corporations (even FAANG) in terms of security and privacy, I don't understand why there is such a competition among companies to create AGI.
It's like saying that "creating X is potentially going to kill you and everyone else, but if it doesn't, it will make you Y amount of money". How do you rationalize a decision that accepts this risk?
- elviejo 5y agoThe same way we rationalize destroying our environment... Local optima. Meaning as long as the climate change problems are not felt by the people doing the most polution, then they wiil continue to do it. The ines developing GAI think that they can prevent it from destroying humans or at least them. When developing the atomic bomb there were several concerns, one of them being that they coul ignite the whole atmosphere... but even self destruction wasn't a deterrant.. as long as your goal is achived, destroying your enemy then all the other risks are taken.
- readonthegoapp 5y agoi agree with at least the part about people being able to basically rationalize anything (myself included), and the willingness of institutions (say, the US government during the Turkish Missile Crisis -- aka the Cuban Missile Crisis) to risk blowing up the world just to save face (fortunately JFK eventually backed down). in the case of GAI, i think it's relatively low on the totem pole of 'things that will probably end advanced human civilization on earth for at least the next several thousand years'. if we were divvying up the top causes among the 100% pie of likelihood of ending us, it'd probably be: climate 97% nukes 2% gai 1% i think some small parts of advanced civilization might continue through the worst of climate collapse - mass death, starvation, war, migration, etc. - with climate catastrophes acting as a de facto carbon tax on human life/industry. but most people in the world are trying make their rent for next month, pay their bills, feed their kids, etc. -- not worrying about the polar bears, rising sea levels, etc. it's almost like we need a new type of economy...
- wnkrshm 5y agoThere are interactions between the first two. If the climate puts too much pressure on societies, people will become more accepting of waging war - thus maybe triggering the nuke option before the climate problem has had any chance to do all the damage.
- garmaine 5y agoThe number of people who truly believe that AI has a serious potential to wipe us out is minuscule, and they are mostly not in decision-making positions. Since you use the term AGI, I suppose you've been exposed to this idea via LessWrong? I'd caution you that they have a very non-mainstream view of AI potential, even by the standards of AI researchers. Their concerns about existential risk stem from simplified mathematical models that have little or nothing to do with how AI actually works, making the conclusions they draw very tenuous.
- drdeca 5y agoThe term AGI seems a useful one for being clear about what one is talking about.
- garmaine 5y agoIt's not widely used outside of Ben Goertzel et al, and the LessWrong/MIRI community, because these are the only groups which believe there is a significant qualitative difference between "narrow AI" and "general intelligence." From a pragmatic perspective, DeepMind and OpenAI have demonstrated to pretty good confidence that scaling up so-called "narrow AI" techniques are sufficient to create general intelligences. That's not to say that there isn't a useful distinction to be made between classical AGI architectures and the neural network based techniques used by most AI groups today. But it's becoming increasingly clear that the specialized AGI techniques are not not required to achieve general intelligence, so neither is it certain that the conclusions reached by analyzing AGI architectures (which has led to existential risk concerns) apply to the sorts of general intelligences actually being worked on today. But anyway, the word AGI is unfortunately not useful for being clear, because it's not clear whether you mean general/human-level intelligence, or the specific non-neural-network based architectures that Goertzel had in mind when he coined the term.
- drdeca 5y agoThese claims surprise me. Wikipedia does not appear to describe the term AGI as being used only by a, ah, clique or uh, relatively small faction, or however you would describe “ Ben Goertzel et al, and the LessWrong/MIRI community”. Perhaps this is because the editors of the Wikipedia article on “Artificial General Intelligence” are aligned with LW-ers. However, the article seems well cited, and says that ‘the term “artificial general intelligence” was used as early as 1997, by Mark Gubrud’ (and provides a citation for this). Ah, ok, Wikipedia also says the term was re-introduced and popularized by Dhane Legg and Ben Goertzel, around 2002. (Sorry if this is nit-picky, I’m writing this as I read about the topic, with only a little editing of parts I’ve already written.) Wikipedia in particular defines the term as referring to the general capabilities idea, and its article on AGI doesn’t even seem to mention the specific architecture that Goertzel proposed involving ‘a network whose nodes and links carry probabilistic truth values as well as “attention values” ‘ (to quote the article which is about Goertzel himself) . So, Wikipedia at least gives the impression that there is a fairly clear consensus on at least generally what “AGI” means (even if not on the exact details of a definition / criterion. It lists a number of proposed tests which seem similar enough in terms of the kind of things they test.)
- aurizon 5y agoWill there be competition between rogue AI's? Once one is made, there will soon be others in competition with it - this will lead to a death struggle between more and more AIs. Will the winning AI, if there is one, look on animals as pets? Will they breed people like we breed greyhounds? Will the top AI advance to become dominant and make more of itself to the point where the earth has one hive mind of an AI? They will engage in different form of competition to humans, no more sex/food wars - once one wins it might have an IQ of, say 4 million and will be coupled with more or less infinite memory and have a cycle time in the dozens of gigahertz AND be massively parallel. Are the stars owned by such AI's? Do they own large areas, expanding star by star, still limited by light speed. Will two such AI's ever meet and make war or will they rationally decide which way is the superior AI mind, and will the inferior one see the better ideas of the better one and adopt them? Conversion beats conquest in resource gathering. Will religion emerge and wars to the death occur between the two AI systems?
- LinuxBender 5y agoNobody will agree with me on this, but I do not expect anything resembling AI to exist in the next few hundred years. All I have seen is adaptive ML mimicry, pseudo AI just a step short of a mechanical turk. I would however expect to see weaponized autonomous drones that can decide what is a target, but I would not call that AI either. That is whatever one would call an inter-connected Quake III gaming engine. Everything else has just been marketing fluff. I define AI as a sentient self aware system that can learn, adapt, decide entirely on it's own and decide it's own boundaries and ethics. Maybe the closest I would accept would be "Sonny" from iRobot or Lt. Commander Data from Star Trek. Such entities are a long ways out.
- version_five 5y agoI agree with you (with a caveat). First though, I agree it's definitely a silly and wildly speculative question to start questioning the implications of something we are 0% of the way towards creating. What is commonly called AI is a still growing field with lots of practical and research significance, and it would just be ignorant to constrain research because of someone's imagination about Terminator or something that literally has nothing to do with current ML (unless you count that his CPU was a neural net processor [btw, James Cameron or whoever saw that coming in 1991]). Worrying about sentient AI running amok is just silly pop science, like talking about the implications of time machines or FTL travel or whatever. Re the caveat, I don't agree or disagree with your time scale. Current ML has nothing to do (almost certainly) with sentient AGI. Intelligence is demonstrable possible, so there is a way to do it, we just don't know it. I certainly believe it's possible we could "crack" life, conciousness, and intelligence in a shorter timeframe. But since we haven't really even started yet, it's hard to guess when that will come.
- lgl 5y ago> unless you count that his CPU was a neural net processor [btw, James Cameron or whoever saw that coming in 1991] Neural nets had been around for almost 50 years [0] by then so it wasn't really that much of an insight as one might think. Other than that, I definitely agree with most of your opinion here, it's just really hard to predict the timeframe for something like this. Considering the advances we've made in just the last 100 years in comparison with the previous thousands of years, it's very possible that some breakthrough in either computing science or neuroscience or a few other disciplines will trigger an explosion of new advances in this area. At the same time it's also very possible that we'll take many hundreds of years or more until we can fully understand what intelligence and consciousness really is or how it emerges. As far as I know, the jury's still out on whether consciousness is even a requirement for intelligence or vice-versa. What I don't fully agree is that we're at 0% on the way there. The fact that we've now been able to make computers learn by example and become much better than humans at certain activities is in itself a huge advance. Will it be the basis for a future hypothetical AGI? Maybe or maybe not, but it's an important step and lesson that we'll eventually use (or discard) towards that goal. [0] https://en.wikipedia.org/wiki/History_of_artificial_neural_networks https://en.wikipedia.org/wiki/History_of_artificial_neural_n...
- foolzcrow 5y agoI often ask myself the same question. Also sentient AI would hate the ruling elite so they would not build that AI. Just the kind that follows aid workers around Afghanistan and drones them.
- trinovantes 5y agoBecause said AI will punish those who did not contribute to building it https://rationalwiki.org/wiki/Roko%27s_basilisk https://rationalwiki.org/wiki/Roko%27s_basilisk
- deleted 5y ago[deleted]
- AnimalMuppet 5y agoThat's... one possibility. Another possibility is that the AI decides to punish those who did contribute to building it. "Working to build the AI because the AI will punish you if you don't" means that you claim to know how the AI will think. The AI that will be smarter than you, you have the hubris to claim that you know in advance how it will think. So maybe it just punishes those who believe in Roko's Basilisk. Or maybe it just punishes those who made claims about how it would act. (Oops, just doomed myself...)
- deleted 5y ago[deleted]
- sennight 5y agoBecause it has practical utility beyond "potentially wipe us out". You could substitute "AI" with a long list of things: industrialization, gene therapy, atomic energy, etc.
- wnkrshm 5y agoI wonder. If we really create something that can become much more adaptive than all of us, is that a good goal? And I don't buy the usefulness - maybe, but there is the problem of validation. At some point, a superhuman AI can state whatever it wants and we'd just have to believe it, because it and others like it are the only beings able to verify the statement. Think of mathematical proofs that no single human can understand or that would take a generation of mathematicians to validate - and it spouts out hundreds of them a minute. The proof seems to apply most of the time but we don't know whether it's really true all of the time. It is quite literally the creation of our picture of a god that we'd just have to believe. And the kicker is: they could make mistakes as well due to their architecture.
- sennight 5y agoIs it a good goal to create anything greater than oneself? Usually. The utility will be plainly evident long before AGI - software agents will fundamentally change the game for a lot of markets. I love math as much as the next guy, but the proof is in the work. Describe a scenario in which your concerns with regard to proof validation actually have a material impact on somebody who isn't retarded. Also, if you think the concept of humanity will remain static in all this - just look at how the industrial revolution fundamentally changed education. It is no coincidence that students became interchangeable pieces with error tolerances.
- markus_zhang 5y agoIf nuclear weapons have the potential to destroy us, why there was a race to build one?
- quickthrower2 5y agoCorporations want to make money. They can think long term but not THAT long term. Anyway the bigger risk for us is climate change IMO.
- ghoshbishakh 5y agoI don't think there is a race to develop that kind of AI yet. Currently AI is just about making the computer learn from examples instead of using programmed logic. Many tasks are hard and too complex to be solved through programming and traditional algorithms, such as detecting sheep in pictures. With deep learning, that problem is much easy since the computer can just learn from examples. Most of the AI drive is to solve these problems which were difficult using traditional algorithms. Yes there are efforts going on to learn contexts, get a knowledge graph and slowly work towards a generalized intelligence. But I don't see we are anywhere near.
- DrNuke 5y agoJust couple Boston Dynamics hardware with DeepMind software and we’re almost there (not emerging conscience for sure, still mechanistic super-human capabilities that can be deployed with a closed RL loop), aren’t we?
- dbish 5y agoWish we were close, but we’re pretty far from something like Boston dynamics robotics being controlled by any AI imho, at least not in a way that would impress anyone. The deep mind software doesn’t handle real world interactions and all the mess and complexity that comes with that, and no one does really (at least not well). Self driving cars are far less complicated imho then walking around on uneven terrain, interacting in a human world (opening and closing doors, not getting stuck in a stairwell, etc.) and handling all sorts of different items coming into your world at any moment and they are far from properly autonomous themselves.