5 ms·
Alignment in the realm of AGI is not about getting everyone to agree. It's about whether or not the AGI is aligned to the goal you've given it. The paperclip
by ihumanable 2y ago
Alignment in the realm of AGI is not about getting everyone to agree. It's about whether or not the AGI is aligned to the goal you've given it. The paperclip AGI example is often used, you tell the AGI "Optimize the production of paperclips" and the AGI started blending people to extract iron from their blood to produce more paperclips.
Humans are used to ordering around other humans who would bring common sense and laziness to the table and probably not grind up humans to produce a few more paperclips.
Alignment is about getting the AGI to be aligned with the owners, ignoring it means potentially putting more and more power into the hands of a box that you aren't quite sure is going to do the thing you want it to do. Alignment in the context of AGIs was always about ensuring the owners could control the AGIs not that the AGIs could solve philosophy and get all of humanity to agree.
- ndriscoll 2y agoRight and that's why it's a farce. > Whoa whoa whoa, we can't let just anyone run these models. Only large corporations who will use them to addict children to their phones and give them eating disorders and suicidal ideation, while radicalizing adults and tearing apart society using the vast profiles they've collected on everyone through their global panopticon, all in the name of making people unhappy so that it's easier to sell them more crap they don't need (a goal which is itself a problem in the face of an impending climate crisis). After all, we wouldn't want it to end up harming humanity by using its superior capabilities to manipulate humans into doing things for it to optimize for goals that no one wants!
- tdeck 2y agoDon't worry, certain governments will be able to use these models to help them commit genocides too. But only the good countries!
- concordDance 2y agoA corporate dystopia is still better than extinction. (Assuming the latter is a reasonable fear)
- simianparrot 2y agoNeither is acceptable
- portaouflop 2y agoI disagree. Not existing ain’t so bad, you barely notice it.
- wruza 2y agoAGI started blending people to extract iron from their blood to produce more paperclips That’s neither efficient nor optimized, just a bogeyman for “doesn’t work”.
- FeepingCreature 2y agoYou're imagining a baseline of reasonableness. Humans have competing preferences, we never just want "one thing", and as a social species we always at least _somewhat_ value the opinions of those around us. The point is to imagine a system that values humans at zero: not positive, not negative.
- freehorse 2y agoStill there are much more efficient ways to extract iron than from human blood. If that was the case humans would have already used this technique to extract iron from the blood of other animals.
- FeepingCreature 2y agoHowever, eventually those sources will already be paperclips.
- freehorse 2y agoWe will probably have died first by whatever disasters the extreme iron extraction on the planet will bring (eg getting iron from the planet's core). Of course destroying the planet to get iron from its core is not a popular agi-doomer analogy, as that sounds a bit too human-like behaviour.
- FeepingCreature 2y agoAs a doomer, I think that's a bad analogy because I want it to happen if we succeed at aligned AGI. It's not doom behavior, it's just correct behavior. Of course, I hope to be uploaded to the WIP dyson swarm around the sun at this point. (Doomers are, broadly, singularitarians who went "wait, hold on actually.")
- vasco 2y agoIt still think it makes little sense to work on because guess what, the guy next door to you (or another country), might indeed say "please blend those humans over there", and your superaligned AI will respect its owners wishes.