I read about a system like that, and the thing is that it not only caused different mistakes, but the mistakes are scary because they are potentially unbounded

I read about a system like that, and the thing is that it not only caused different mistakes, but the mistakes are scary because they are potentially unbounded as well as novel and extreme. When the machine/program seems perfect, then people get conditioned not to question it. There was a case where a UI issue caused the selection of an adjacent dosage to what was intended, and a nurse gave a patient something like 100 pills. There were several checks in theory, but everybody was like "gee this seems weird, if the computer says so, it must be right". Automation inherently multiplies actions, so the fear is that bugs don't just cause a single error, but many, until someone notices and has a chance to react.