3 ms·
Very close analogues to the trolley problem exist. I remember reading about a hospital which replaced the staff that measured out drug prescriptions with a comp
by mcphage 6y ago
Very close analogues to the trolley problem exist. I remember reading about a hospital which replaced the staff that measured out drug prescriptions with a computerized system. It made far fewer mistakes but, critically, it did allow for some mistakes which human operators would have caught and corrected. When I read the article I remember being annoyed that they didn’t mention the trolley problem at all because it was an almost perfect analogue.
- perl4ever 6y agoI read about a system like that, and the thing is that it not only caused different mistakes, but the mistakes are scary because they are potentially unbounded as well as novel and extreme. When the machine/program seems perfect, then people get conditioned not to question it. There was a case where a UI issue caused the selection of an adjacent dosage to what was intended, and a nurse gave a patient something like 100 pills. There were several checks in theory, but everybody was like "gee this seems weird, if the computer says so, it must be right". Automation inherently multiplies actions, so the fear is that bugs don't just cause a single error, but many, until someone notices and has a chance to react.