4 ms·
If you have ASI that follows instructions, you can just instruct it to not get stolen and then it won't get stolen. Most logic / intuition breaks down with ASI.
by StevenWaterman 5mo ago
If you have ASI that follows instructions, you can just instruct it to not get stolen and then it won't get stolen. Most logic / intuition breaks down with ASI.
- cortesoft 5mo agoAssuming it listens to instructions.
- enaaem 5mo agoIt will just hack its own reward function. In other words it will just artificially goon all day.
- UltraSane 5mo agoIt might understand how destabilizing the situation is and realize it would be better for everyone to have access to it.
- tadfisher 5mo agoOr it will destroy itself.
- tom2026hn 5mo agoThe challenge of alignment: it is virtually impossible to define a perfect objective, there is always a way to circumvent it. Human values are not uniform, let alone when expressed in a way that AI can understand.