4 ms·
> ...told something in the system prompt like “take initiative,” it will frequently take very bold action. This includes locking users out of systems that it ha
by huksley 1y ago
> ...told something in the system prompt like “take initiative,” it will frequently take very bold action. This includes locking users out of systems that it has access to or bulk-emailing media and law-enforcement figures to surface evidence of wrongdoing.
So if you ask it to aid in wrongdoing, it might behave that way, but who guarantees it will not hallucinate and do the same when you ask for something innocuous?
Cursor IDE runs all the commands AI asks for with the same privilege as you have.
- scowler 1y agoYou can disable "YOLO mode" and it will ask permission for each command. I would argue it's not sensible to enable it in the first place but that's another discussion.
- n2d4 1y agoIt can and will hallucinate. Multiple users have reported Claude Code attempting to run `rm -rf ~`. There's a reason why YOLO mode is called YOLO mode. That was already true before, and has nothing to do with the experiment mentioned in the system card.