The problem with that is that you can't tell programmatically if a potentially risky action is performed by a program acting as the user's agent or by a program

The problem with that is that you can't tell programmatically if a potentially risky action is performed by a program acting as the user's agent or by a program acting on behalf of some malicious fuckwit. You can just forbid it, sure, but then you're reducing the usefulness of your platform.