3 ms·
I don’t call it a failure of the AI. I agree it’s doing exactly what it was trained to do. The failure is architectural: once AI is allowed to draft at scale,
by bhaviav100 9mo ago
I don’t call it a failure of the AI. I agree it’s doing exactly what it was trained to do.
The failure is architectural: once AI is allowed to draft at scale, “don’t feed it commitments” stops being a reliable control. Those patterns exist everywhere in historical data and live context.
At that point the question isn’t training, it’s where you draw the enforcement boundary for irreversible outcomes.
That’s the layer I’m testing.
- chrisjj 9mo agoI don't see that scale of drafting makes any difference. Reliability is entirely down to the training. Also I think confining irreversible outcomes to the results of commitments is unsafe. Consider the irreversible outcome of advice that leads to customer quitting. There isn't a separate "layer" here.
- bhaviav100 9mo agoThese are two different control problems. Training governs what a model tends to say. Authority governs what is allowed to be acted on. You can’t pre-block bad advice, but you can pre-block unapproved financial or contractual actions. That’s the scope.
- chrisjj 9mo agoThey are the same problem, since the actions are saying. "AI-generated messages making commitments no one explicitly approved. Refunds implied. Discounts promised. Renewals renegotiated." And if you can't block bad advice, you have a bigger problem since you cannot block resultant contractual action by the recipient e.g. termination.