3 ms·

Perhaps the solution is a different approach. Instead of human monitors, we add computer monitors. Which then initiate a safe system shutdown. The goal being t

Perhaps the solution is a different approach. Instead of human monitors, we add computer monitors. Which then initiate a safe system shutdown. The goal being to "remove energy" from the system as soon as safely possible. A car for example might come to a stop as soon as there is a safe place to do that. Some systems, think aeroplanes, are fully automatable but still have human oversight. These are ones where "safe stop" is highly variable and difficult to achieve. We have pilots for that, and very sophisticated simulators for training. For industrial processes, factories and do on, safe-stop can be designed in from day one. Humans would be required for maintainence, not operation. I suspect this is already true in lots of factories.

nonrandomstring 4y ago
> Instead of human monitors, we add computer monitors. Which then initiate a safe system shutdown. The author seems to argue against this "solutionism". The main theme is atrophy, mainly of attention and internal working model (responsive competence) that decays without regular human involvement. > increasing automation, increasing system complexity, faster processing, more inter-connectivity, and an even greater human and societal dependence on technology. What could possibly go wrong? The question is what could possibly go right? And indeed the author offers good, positive suggestions; - breaking the loop (no invisible self-corrections) - making complex systems shallow (not nested) - emphasis on action/doing with hands rather than passively monitoring with eyes and ears. Unfortunately I don't think rational engineering and design will allow correction before real disasters come from over-dependency (especially in the digital realm), because the course we are on is not rational. It looks rational. It seems the very essence of rationality. But taken to the limit it is a totally irrational human death-wish to grovel, prostrate at the feet of machines, qua ideology. And it's getting harder to hide despite fervent rationalisation around "efficiency", "necessity", "inevitability"... ad nauseum. We just want to surrender into the arms of "convenience".

nonrandomstring 4y ago

> Instead of human monitors, we add computer monitors. Which then initiate a safe system shutdown. The author seems to argue against this "solutionism". The main theme is atrophy, mainly of attention and internal working model (responsive competence) that decays without regular human involvement. > increasing automation, increasing system complexity, faster processing, more inter-connectivity, and an even greater human and societal dependence on technology. What could possibly go wrong? The question is what could possibly go right? And indeed the author offers good, positive suggestions; - breaking the loop (no invisible self-corrections) - making complex systems shallow (not nested) - emphasis on action/doing with hands rather than passively monitoring with eyes and ears. Unfortunately I don't think rational engineering and design will allow correction before real disasters come from over-dependency (especially in the digital realm), because the course we are on is not rational. It looks rational. It seems the very essence of rationality. But taken to the limit it is a totally irrational human death-wish to grovel, prostrate at the feet of machines, qua ideology. And it's getting harder to hide despite fervent rationalisation around "efficiency", "necessity", "inevitability"... ad nauseum. We just want to surrender into the arms of "convenience".