4 ms·

>When multi-million dollar warships are crashed because of poor UX, I have zero faith we're safely managing complexity. My understanding of the incident is tha

>When multi-million dollar warships are crashed because of poor UX, I have zero faith we're safely managing complexity. My understanding of the incident is that the auto-detection functions failed and when human functions were not done per SOP due to exhaustion and senior command elsewhere the vessel came into jeopardy. Then when inexperienced and untrained crew members tried too late to evade. An apparently too complex UX was sending ship controls to more than one console. This culminated into one of the multi-million dollar warships crashing.

agoodthrowaway 7y ago
I wonder what would happen in an actual battle with the fog of war when people are pushed beyond exhaustion. What happens when senior command is dead or missing?
- metaphor 7y ago
  There exists a very distinct line between routine/exercise and war. You may have missed the opportunity to experience this distinction first hand, but I can assure you with the utmost confidence, the meta does indeed change.
dvdbloc 7y ago
What event was this?
- garrettgrimsley 7y ago
  Sounds like the US Navy destroyer that collided [0] with a cargo ship. [0] https://en.wikipedia.org/wiki/USS_Fitzgerald_and_MV_ACX_Crystal_collision https://en.wikipedia.org/wiki/USS_Fitzgerald_and_MV_ACX_Crys...
- bin0 7y ago
  Based on google results, it appears to be an incident where the USS McCain collided with the Alnic MC near Singapore. She was getting off course, and poor UI made it so hard to steer her back that she crashed. Injuries and deaths resulted. https://medium.muz.li/ui-confusion-almost-sunk-a-us-destroyer-6232e8c4e59e https://medium.muz.li/ui-confusion-almost-sunk-a-us-destroye...
Johnny555 7y ago
Though one could say that this was due to too little automation rather than too much -- the default behavior could be for the ship to steer itself away from another vessel or shallow water unless overridden (setting off alarms that this action is being taken) My $30K car does a fair job of nudging the steering wheel if it thinks I'm running off the road, surely a $2B battleship could have some basic collision avoidance capability too.
- qserasera 7y ago
  Unfortunately military ships are expensive and have strict schedules. You may be able to drive your car to a shop any day to diagnose a sensor error but military vessels dont have that luxury. To my knowledge the detection systems (for detecting ships ect) were either not calibrated/failed making a ship impossible to detect. There may have been a serious lack of training spotting sensor failure too. The ship was due for repairs and had been put off. Also, according to SOP there should have been someone making rounds and looking out and identifying ships. Because honestly how do you miss a huge cargo ship lit up like a Christmas tree There were multiple critical failures leading to the crash. Bad training, poor maintenance, and lax procedural compliance.
- joejerryronnie 7y ago
  The problem with automation is that it works perfectly until it runs into that edge case which nobody thought about. The problem with the real world is that there are infinity edge cases.
  - pferde 7y ago
    I'd like to use this in my .signature.
  - RomanBob 7y ago
    > there are infinity edge cases. this is simply not true. there are finite particles in discrete space so there's no infinity here.
  - Nasrudith 7y ago
yason 7y ago
A lot of problems with automation vs manual controls is that the line between those tends to be blurry. It's not that automation wouldn't work in the absolute majority of cases, and that we couldn't define the cases where automation will have to know it cannot make it. It's not that humans would always be better than automation: humans make mundane mistakes and casual accidents happens, and humans make bad judgements also in extreme situations. Looking at the catastrophes in aviation and shipping from the recent years the problem seems to lie where automation and human controls interact. In the typical case the automation is not completely switched off while the human still tries to take control of the system and some override functionality still triggers the automation without the human realising that. Maybe there's a sensor failure that causes the automation to misbehave. Then the human controller turns off the autopilot and begins to apply the proper fix but the electronic system, through its faulty sensors, sees it differently and concludes that the human is doing something way too dangerous and doesn't let him. Then a tug of war begins with the human controller where the automation is thinking that the situation is dire and the ship must be saved while the human realises that something is not working properly but in the heat of the moment doesn't have time to diagnose or narrow down to shutting off automation and just keeps applying more. If only the automation knew it can't "see" properly and the human is already trying to fix things, or if only the human knew part of the automation is still active and secretly, with no indication, countering the proper corrections by the human controller.
- homonculus1 7y ago
  Sounds like these systems need a little red killswitch under a plexiglass cover. It's baffling to me that anyone who's ever designed an automated system thinks there won't be situations where a human has more complete awareness of what's happening and needs to unambiguously wrest all control.
  - ethbro 7y ago
    It's usually not that simple. The majority of complex systems have substantial portions of their operation effectively invisible to us. Electricity, radiation, remote or sealed mechanical systems, deeply layered software. In all of these, you need systems to communicate actual state to a human to effect even a manual resolution.

agoodthrowaway 7y ago

I wonder what would happen in an actual battle with the fog of war when people are pushed beyond exhaustion. What happens when senior command is dead or missing?

metaphor 7y ago
There exists a very distinct line between routine/exercise and war. You may have missed the opportunity to experience this distinction first hand, but I can assure you with the utmost confidence, the meta does indeed change.

dvdbloc 7y ago

What event was this?

garrettgrimsley 7y ago
Sounds like the US Navy destroyer that collided [0] with a cargo ship. [0] https://en.wikipedia.org/wiki/USS_Fitzgerald_and_MV_ACX_Crystal_collision https://en.wikipedia.org/wiki/USS_Fitzgerald_and_MV_ACX_Crys...
bin0 7y ago
Based on google results, it appears to be an incident where the USS McCain collided with the Alnic MC near Singapore. She was getting off course, and poor UI made it so hard to steer her back that she crashed. Injuries and deaths resulted. https://medium.muz.li/ui-confusion-almost-sunk-a-us-destroyer-6232e8c4e59e https://medium.muz.li/ui-confusion-almost-sunk-a-us-destroye...

Johnny555 7y ago

Though one could say that this was due to too little automation rather than too much -- the default behavior could be for the ship to steer itself away from another vessel or shallow water unless overridden (setting off alarms that this action is being taken) My $30K car does a fair job of nudging the steering wheel if it thinks I'm running off the road, surely a $2B battleship could have some basic collision avoidance capability too.

qserasera 7y ago
Unfortunately military ships are expensive and have strict schedules. You may be able to drive your car to a shop any day to diagnose a sensor error but military vessels dont have that luxury. To my knowledge the detection systems (for detecting ships ect) were either not calibrated/failed making a ship impossible to detect. There may have been a serious lack of training spotting sensor failure too. The ship was due for repairs and had been put off. Also, according to SOP there should have been someone making rounds and looking out and identifying ships. Because honestly how do you miss a huge cargo ship lit up like a Christmas tree There were multiple critical failures leading to the crash. Bad training, poor maintenance, and lax procedural compliance.
joejerryronnie 7y ago
The problem with automation is that it works perfectly until it runs into that edge case which nobody thought about. The problem with the real world is that there are infinity edge cases.
- pferde 7y ago
  I'd like to use this in my .signature.
- RomanBob 7y ago
  > there are infinity edge cases. this is simply not true. there are finite particles in discrete space so there's no infinity here.
- Nasrudith 7y ago

yason 7y ago

A lot of problems with automation vs manual controls is that the line between those tends to be blurry. It's not that automation wouldn't work in the absolute majority of cases, and that we couldn't define the cases where automation will have to know it cannot make it. It's not that humans would always be better than automation: humans make mundane mistakes and casual accidents happens, and humans make bad judgements also in extreme situations. Looking at the catastrophes in aviation and shipping from the recent years the problem seems to lie where automation and human controls interact. In the typical case the automation is not completely switched off while the human still tries to take control of the system and some override functionality still triggers the automation without the human realising that. Maybe there's a sensor failure that causes the automation to misbehave. Then the human controller turns off the autopilot and begins to apply the proper fix but the electronic system, through its faulty sensors, sees it differently and concludes that the human is doing something way too dangerous and doesn't let him. Then a tug of war begins with the human controller where the automation is thinking that the situation is dire and the ship must be saved while the human realises that something is not working properly but in the heat of the moment doesn't have time to diagnose or narrow down to shutting off automation and just keeps applying more. If only the automation knew it can't "see" properly and the human is already trying to fix things, or if only the human knew part of the automation is still active and secretly, with no indication, countering the proper corrections by the human controller.

homonculus1 7y ago
Sounds like these systems need a little red killswitch under a plexiglass cover. It's baffling to me that anyone who's ever designed an automated system thinks there won't be situations where a human has more complete awareness of what's happening and needs to unambiguously wrest all control.
- ethbro 7y ago
  It's usually not that simple. The majority of complex systems have substantial portions of their operation effectively invisible to us. Electricity, radiation, remote or sealed mechanical systems, deeply layered software. In all of these, you need systems to communicate actual state to a human to effect even a manual resolution.