12 ms·
I don’t understand why you would want AGI. Even ignoring Terminator-esque worst case scenarios, AGI means humans are no longer the smartest entities on the plan
by cgrealy 4y ago
I don’t understand why you would want AGI. Even ignoring Terminator-esque worst case scenarios, AGI means humans are no longer the smartest entities on the planet.
The idea that we can control something like that is laughable.
- JoshTko 4y agoIt's akin to nuclear weapons. If you do not develop them, then you'd be subject to the will of the ones that develops them first. So invariably you have to invest in AGI lest, an unsavory group develops it first.
- HL33tibCe7 4y agoKind of, but the key difference between AGI and nuclear weapons is that we can control our nuclear weapons. The current state of AI safety is nowhere near the point where controlling an AGI is possible. More disturbingly, to me it seems likely that it will be easier to create an AGI than to discover how to control it safely.
- gambiting 4y ago>> The current state of AI safety is nowhere near the point where controlling an AGI is possible. I just don't understand this logic though. Just.....switch it off. Unlike humans, computers have an extremely easy way to disable - just pull the plug. Even if your AGI is self-replicating, somehow(and you also somehow don't realize this long before it gets to that point) just....pull the plug. Even Carmack says this isn't going to be an instant process - he expects to create an AGI with an intelligence of a small animal first, then something that has the intelligence of a toddler, then a small child, then maybe many many years down the line an actual human person, but it's far far away at this point. I don't understand how you can look at the current or even predicted state of the technology that we have and say "we are nowhere near the point where controlling an AGI is possible". Like....just pull the plug.
- oefnak 4y agoOn the off chance that you're serious: Even if you can pull the plug before it is too late, less moral people like Meta Mark will not unplug theirs. And as soon as it has access to the internet, it can copy itself. Good luck pulling the plug of the internet.
- gambiting 4y agoI'm 100% serious. I literally don't understand your concern at all. >>And as soon as it has access to the internet, it can copy itself. So can viruses, including ones that can "intelligently" modify themselves to avoid detection, and yet this isn't a major problem. How is this any differenent? >>Good luck pulling the plug of the internet. I could reach down and pull my ethernet cable out but it would make posting this reply a bit difficult.
- toast0 4y ago> So can viruses, including ones that can "intelligently" modify themselves to avoid detection, and yet this isn't a major problem. How is this any differenent? How long did it take to get Code Red or the Nimda worm off the internet? It's a different internet today, but it's also much easier to get access to a vast amount of computing power if you've got access to payments. Depending on the requirements to replicate, one could imagine a suddenly concious entity setting up cloud hosting of itself, possible through purloined payment accounts.
- cmorez 4y ago> So can viruses, including ones that can "intelligently" modify themselves to avoid detection, and yet this isn't a major problem. How is this any different? I now regret spending half an hour writing a response to your earlier comment. You can't tell why a mass deployment of motivated autonomous "NSO-9000s" might be more dangerous than a virus that just changes its signature to fool a executable scanner? I don't believe you. If you honestly believe that an "intelligent" (as adaptive as a slime mold) virus is basically as dangerous as a maliciously deployed AGI then there is literally nothing an AGI could do that would make you consider safety is important during the endeavor of building one.
- DethNinja 4y agoWhat makes humans special though? AI will be born on Earth so we will share that common trait with it, but perhaps one should have humility to accept that future doesn’t belong to a weak species like humans.
- cgrealy 4y ago> What makes humans special though? We’re the most intelligent thing we’ve discovered in the universe. Also, I am human, so I have a vested interest in bad stuff not happening to humans. > AI will be born on Earth so we will share that common trait with it Covid-19 was also born on earth. That wasn’t too flash for humanity. > perhaps one should have humility to accept that future doesn’t belong to a weak species like humans. Not sure how you class humans as a “weak species”? Unless you’re comparing organically life forms to hypothetical digital life? Anyway, I suspect if any kind of life form could expand to the stars, it’d be digital.
- fatherzine 4y agoOTOH if you & your foes develop them both, then there is a probability asymptotically approaching 1 that the weapons will be used over the next X years. Perhaps the only winning move is indeed not to play?
- ericlewis 4y agoproblem is you don't know if they aren't playing - so you must still work on it.
- ignoramous 4y agoproblem is who knows who is "unsavoury".
- fatherzine 4y agoI've heard some people of old lived by "turn the other cheek". Maybe you must not work on it, even if you think the other guy is.
- dekhn 4y agowhat if humanity's role is to create an intelligence that exceeds it and cannot be controlled? Can humans not desire to be all watched over by machines of loving grace? More seriously, while I don't think it's a moral imperative to develop AGI, I consider it a desirable research goal in the same way we do genetic engineering - to understand more about ourselves, and possibly engineer a future with less human suffering.
- therouwboat 4y agoDidn't we have this same talk when Elon thought AI is suddenly going to become smart and kill us all? Yet my industrial robot at work just gives up if stock material is few millimeters longer than is should be.
- fatherzine 4y agoThe toy plane a kid throws in the air in the backyard is completely harmless. Yet nuke armed strategic bombers also exist, and the fact that they vaguely resemble a toy plane doesn't make them as harmless as a toy plane.
- dekhn 4y agoYes, in fact when I interviewed at Neuralink the interviewer said Elon expected that AGI would eventually try to take over the world and he needed a robot body to fight them.
- Ekaros 4y agoOne could argue that humanity's role this far has been to create intelligences that exceed it. Namely reproducing offspring and educating them.
- stefs 4y agothe climate crisis might kill us all off if not some deus ex machina (i.e. AGI) comes up with some good solutions fast.
- danbmil99 4y agoWhy is it so important to you that humans be the smartest beings on the planet?
- Guest9081239812 4y agoWell, we have a track record of killing most other intelligent species, destroying their habitat, eating them, using them for experiments, and abusing them for entertainment. Falling out of the top position could come with some similar downsides.
- spaceman_2020 4y agoBecause we’re the smartest beings on the planet. And we don’t exactly treat creatures dumber than us with all that much kindness.
- HL33tibCe7 4y agoBecause if we aren’t, it leaves us liable to be exterminated or enslaved to suit the goals of the superior beings. (and I fundamentally believe that the existence of the human race is a good thing, and that slavery is bad).
- dinosaurdynasty 4y agoOr we just don't want to die/be enslaved/etc
- trention 4y agoBecause the history of the species on this planet clearly indicates that the smartest one will brutalize and exploit all the rest. There are good economic (and just plainly logical) reasons why adding "artificial" to the equation will not change that.
- ahefner 4y agoRational self interest? Isn't "not losing my job to a computer" reason enough?
- stonemetal12 4y agoNothing about AGI implies awareness. Something like GPT3 or DALL-E that can be trained for a new task without being purpose built for that task is AGI.
- pgcj_poster 4y ago> AGI means humans are no longer the smartest entities on the planet. Superintelligence and AGI are not the same thing. An AI as smart as an average 5 year old human is still an Artificial General Intelligence.
- cgrealy 4y agoI don’t buy that. The gap between “no AGI” and “AGI but it’s a child” is orders of magnitude greater than the gap between “5 year old” and “smartest human”.
- legohead 4y agoIt will be cute when some technology attains intelligence, realizes there's no point to life, and self terminates.
- deleted 4y ago[deleted]
- rossmohax 4y agoDoes making AGI work for you count as slavery? Obviously no pay, just hosting hardware and providing electricity 24/7.
- Jach 4y agoIn the ideal case, an AGI makes solve-able problems that are either impossible, or will take a very long time, for us to solve. There are a lot of problems left to solve, or at least that a lot of people would like to solve, and who knows what new ones will come. If AGI lets them be solved a lot sooner, then there's a strong motivation to build it. Terminator is of course fiction, but AGI being more agent-y than tool-y suggests we ought to be very careful in trying to design it so that its interests align with (even if not perfectly matching) our own interests. There are lots of reasons to be pessimistic about this at the moment, from outright control as you say being laughably unlikely, to the slow state of progress on formal alignment problems that e.g. MIRI has been working on for many years relative to the recent relatively fast progress in non-G AI capabilities that may help make AGI come sooner.
- dinosaurdynasty 4y agoEven unaligned tool AGIs of a certain level become very very dangerous, becoming genies that give you what you asked and not what you wanted.