This was premature automation caused by not fully understanding the context. Results in less friction at the cost of enabling a black swan. Bad trade off. The V

This was premature automation caused by not fully understanding the context. Results in less friction at the cost of enabling a black swan. Bad trade off. The Viking Sky cruise ship that was 1 minute away from releasing its damage potential of about 1300 people. 4 engines stoped simultaneously to protect them selves. Risking the entire ship in one of Norway’s most dangerous waters during harsh weather. There are so many similar examples. Tank turrets self protecting and killing soldier during peace time. Automatic gearbox on military vehicle self protecting against overheating although vehicle is under enemy fire, but the sensor can’t know that.. we need to rethink how “security automation” should work. How do you know if an override is relevant? How to train the operator?