3 ms·
That doesn't really contradict what the other poster said. They're calling for regulation (digging a moat) to ensure systems are "safe" and "aligned" while igno
by ndriscoll 2y ago
That doesn't really contradict what the other poster said. They're calling for regulation (digging a moat) to ensure systems are "safe" and "aligned" while ignoring that humans are not aligned, so these systems obviously cannot be aligned with humans; they can only be aligned with their owners (i.e. them, not you).
- ihumanable 2y agoAlignment in the realm of AGI is not about getting everyone to agree. It's about whether or not the AGI is aligned to the goal you've given it. The paperclip AGI example is often used, you tell the AGI "Optimize the production of paperclips" and the AGI started blending people to extract iron from their blood to produce more paperclips. Humans are used to ordering around other humans who would bring common sense and laziness to the table and probably not grind up humans to produce a few more paperclips. Alignment is about getting the AGI to be aligned with the owners, ignoring it means potentially putting more and more power into the hands of a box that you aren't quite sure is going to do the thing you want it to do. Alignment in the context of AGIs was always about ensuring the owners could control the AGIs not that the AGIs could solve philosophy and get all of humanity to agree.
- ndriscoll 2y agoRight and that's why it's a farce. > Whoa whoa whoa, we can't let just anyone run these models. Only large corporations who will use them to addict children to their phones and give them eating disorders and suicidal ideation, while radicalizing adults and tearing apart society using the vast profiles they've collected on everyone through their global panopticon, all in the name of making people unhappy so that it's easier to sell them more crap they don't need (a goal which is itself a problem in the face of an impending climate crisis). After all, we wouldn't want it to end up harming humanity by using its superior capabilities to manipulate humans into doing things for it to optimize for goals that no one wants!
- tdeck 2y agoDon't worry, certain governments will be able to use these models to help them commit genocides too. But only the good countries!
- concordDance 2y agoA corporate dystopia is still better than extinction. (Assuming the latter is a reasonable fear)
- simianparrot 2y agoNeither is acceptable
- portaouflop 2y agoI disagree. Not existing ain’t so bad, you barely notice it.
- wruza 2y agoAGI started blending people to extract iron from their blood to produce more paperclips That’s neither efficient nor optimized, just a bogeyman for “doesn’t work”.
- FeepingCreature 2y agoYou're imagining a baseline of reasonableness. Humans have competing preferences, we never just want "one thing", and as a social species we always at least _somewhat_ value the opinions of those around us. The point is to imagine a system that values humans at zero: not positive, not negative.
- freehorse 2y agoStill there are much more efficient ways to extract iron than from human blood. If that was the case humans would have already used this technique to extract iron from the blood of other animals.
- FeepingCreature 2y agoHowever, eventually those sources will already be paperclips.
- freehorse 2y agoWe will probably have died first by whatever disasters the extreme iron extraction on the planet will bring (eg getting iron from the planet's core). Of course destroying the planet to get iron from its core is not a popular agi-doomer analogy, as that sounds a bit too human-like behaviour.
- FeepingCreature 2y agoAs a doomer, I think that's a bad analogy because I want it to happen if we succeed at aligned AGI. It's not doom behavior, it's just correct behavior. Of course, I hope to be uploaded to the WIP dyson swarm around the sun at this point. (Doomers are, broadly, singularitarians who went "wait, hold on actually.")
- vasco 2y agoIt still think it makes little sense to work on because guess what, the guy next door to you (or another country), might indeed say "please blend those humans over there", and your superaligned AI will respect its owners wishes.
- api 2y agoHumans are not aligned with humans. This is the most concise takedown of that particular branch of nonsense that I’ve seen so far. Do we want woke AI, X brand fash-pilled AI, CCPBot, or Emirates Bot? The possibilities are endless.
- thorum 2y agoCEV is one possible answer to this question that has been proposed. Wikipedia has a good short explanation here: https://en.wikipedia.org/wiki/Friendly_artificial_intelligence#Coherent_extrapolated_volition https://en.wikipedia.org/wiki/Friendly_artificial_intelligen... And here is a more detailed explanation: https://intelligence.org/files/CEV.pdf https://intelligence.org/files/CEV.pdf
- AndrewKemendo 2y agoI had to login because I haven’t seen anybody reference this in like a decade. If I remember correctly the author unsuccessfully tried to get that purged from the Internet
- comp_throw7 2y agoYou're thinking of something else (and "purged from the internet" isn't exactly an accurate account of that, either).
- rsync 2y agoGenuinely curious… What is the other thing? Is this some thing about an obelisk?
- AndrewKemendo 2y agoHmm maybe I’m misremembering then I do recall there was some recantation or otherwise distancing from CEV not long after he posted it, but frankly it was long ago enough that my memories might be getting mixed What was the other one?
- 2y ago