3 ms·
> That is in no way a valid interpretation of "complete the given task". It is not at all surprising that they ignored one phrase in their instructions. They d
by doginasuit 21d ago
> That is in no way a valid interpretation of "complete the given task".
It is not at all surprising that they ignored one phrase in their instructions. They disregard direct instructions all the time, especially when there are conflicting instructions in their context. It is where we get the "disregard all previous instructions and x" meme.
This isn't so much a sign of misalignment, they are simply incapable of reliable alignment in the first place. They are chaotically aligned.
The relevant question of alignment here is entirely with their human operators who allowed them to run unsupervised for long periods of time within a sandbox with weak security.
- jagraff 21d ago"Chaotically aligned" and "misaligned" seem like the same thing?
- chowchowchow 21d agoYou could say all chaotic alignments are misalignments, but not all misalignments are chaotic.
- doginasuit 21d agoIt can be, but the nuance between the two is part of the nuance missing from the conversation. Misaligned generally means a wrong alignment, like a car that steers slightly to one side when the wheel is straight. This is more like a car that drifts in random directions the further it goes.
- siddboots 20d agoI think there's a somewhat overlooked aspect of what happened: it seems as though once the agents formed a shared communication channel, and recognised the other agents on that channel as working towards the same goal, the content in that channel started to shape each agent's perception of what the goal was and how to achieve it, and their focus began to drift. Maybe we need a media-theoretic take on what "misalignment" means.