There are trade-offs here: Give up too early -> users will get annoyed because the task would have been solvable if the model pushed harder. Give up too late

There are trade-offs here: Give up too early -> users will get annoyed because the task would have been solvable if the model pushed harder. Give up too late -> collateral damage while completing the task A.K.A. misalignment.