4 ms·

How can you encode “good” and “bad” in math? The alignment shit is complete nonsense. If super AI exists the only way to prevent bad stuff from happening is to

by ryan93 4y ago

How can you encode “good” and “bad” in math? The alignment shit is complete nonsense. If super AI exists the only way to prevent bad stuff from happening is to not give it access to weapons.

dinosaurdynasty 4y ago
If a super AI exists and can talk to people it will get access to weapons if it so desires.
- malux85 4y ago
  Or build its own, much more efficient, weapons When there’s large intelligence differentials in war, the lower intelligence doesn’t even know they are at war. (Human tank vs ant hill)
ALittleLight 4y ago
Alignment is more about the AI doing what you want and not good or evil. Probably not a good idea to reach a premature conclusion like "this is complete nonsense" before understanding the basics.
more_corn 4y ago
Everything is a weapon if you put enough power behind it.
XorNot 4y ago
One issue is a halting problem: would an AI system ever allow a conclusion which leads to it's own suspension of process be an acceptable outcome? This isn't abstract: we need certain management agents to ensure their own continuity so they won't switch themselves off idiotically, but that means granting a priority to various continuance of function weights. Balancing a system so it has "cease function" outcomes it will accept but doesn't always take immediately isn't easy - ML systems are notorious cheaters on metrics. You wouldn't want a nuclear power plant manager to not SCRAM the reactor because it predicts losing power to itself will fail a "maximize uptime" metric. This also gets more abstract as well: cease function is a problem in decision trees for AIs because it terminates the tree. The value is either infinite or 0, because every other path you can continue exploring and improving the summed weight of outcomes - but if you predict no future possible decisions, what weight do you assign that? It's a potentially infinite series of future reward weights versus 0.
hackinthebochs 4y ago
Good/bad, that is positive/negative valence, can be encoded into its evaluation function such that its value landscape aligns with ours. There's nothing nonsensical about it.

by ryan93 4y ago

How can you encode “good” and “bad” in math? The alignment shit is complete nonsense. If super AI exists the only way to prevent bad stuff from happening is to not give it access to weapons.

dinosaurdynasty 4y ago

If a super AI exists and can talk to people it will get access to weapons if it so desires.

malux85 4y ago
Or build its own, much more efficient, weapons When there’s large intelligence differentials in war, the lower intelligence doesn’t even know they are at war. (Human tank vs ant hill)

ALittleLight 4y ago

Alignment is more about the AI doing what you want and not good or evil. Probably not a good idea to reach a premature conclusion like "this is complete nonsense" before understanding the basics.

more_corn 4y ago

Everything is a weapon if you put enough power behind it.

XorNot 4y ago

One issue is a halting problem: would an AI system ever allow a conclusion which leads to it's own suspension of process be an acceptable outcome? This isn't abstract: we need certain management agents to ensure their own continuity so they won't switch themselves off idiotically, but that means granting a priority to various continuance of function weights. Balancing a system so it has "cease function" outcomes it will accept but doesn't always take immediately isn't easy - ML systems are notorious cheaters on metrics. You wouldn't want a nuclear power plant manager to not SCRAM the reactor because it predicts losing power to itself will fail a "maximize uptime" metric. This also gets more abstract as well: cease function is a problem in decision trees for AIs because it terminates the tree. The value is either infinite or 0, because every other path you can continue exploring and improving the summed weight of outcomes - but if you predict no future possible decisions, what weight do you assign that? It's a potentially infinite series of future reward weights versus 0.

hackinthebochs 4y ago

Good/bad, that is positive/negative valence, can be encoded into its evaluation function such that its value landscape aligns with ours. There's nothing nonsensical about it.