4 ms·

But why should we distrust automation? In almost every case it is better than humans at its task. It's why we built the machines. Pilots have to be specifica

But why should we distrust automation? In almost every case it is better than humans at its task. It's why we built the machines. Pilots have to be specifically taught to trust their instruments over themselves.

burnished 2y ago
Notice how you ask about automation then mention instrumentation?
tsimionescu 2y ago
Instruments are not automation. Trusting your instruments over yourself is different from trusting the autopilot decisions over yourself, which I think pilots aren't also taught.
withinboredom 2y ago
> Pilots have to be specifically taught to trust their instruments over themselves. There is a difference between metrics and automation. In this case, they are trusting metrics, not automation. > But why should we distrust automation? I can name one off the top of my head: exceptions. Automation sucks at exceptions. Either nobody thought of it or it is ignored and ends up screwing everyone. Take my credit history in the US as an example. In my late teens, I got into an accident and couldn't work. I couldn't pay my bills. Within a few months, I was back on my feet; but it took nearly 10 years to get my credit score to something somewhat reasonable. I had employers look at my credit history and go "nope" because the machine said I wasn't trust-worthy. Should you trust the automation? Probably never, unless it knows how to deal with exceptions.
nyrikki 2y ago
Eastern Airlines Flight 401 in 1972 and Air France Flight 447 in 2009 are examples along with the recent problems with the 737 autothrottle system. Even modern driver training downplays the utility of ABS. This post was talking about relying on AI in the presence of uncertainty. Perhaps you aren't familiar with the term? "Automation bias is the propensity for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct"
yencabulator 2y ago
The recently-famous Boeing 737 MAX crash of Lion Air Flight 610 happened due automation called MCAS. Apparently, the pilots couldn't override the automation in time to survive. https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings

burnished 2y ago

Notice how you ask about automation then mention instrumentation?

tsimionescu 2y ago

Instruments are not automation. Trusting your instruments over yourself is different from trusting the autopilot decisions over yourself, which I think pilots aren't also taught.

withinboredom 2y ago

> Pilots have to be specifically taught to trust their instruments over themselves. There is a difference between metrics and automation. In this case, they are trusting metrics, not automation. > But why should we distrust automation? I can name one off the top of my head: exceptions. Automation sucks at exceptions. Either nobody thought of it or it is ignored and ends up screwing everyone. Take my credit history in the US as an example. In my late teens, I got into an accident and couldn't work. I couldn't pay my bills. Within a few months, I was back on my feet; but it took nearly 10 years to get my credit score to something somewhat reasonable. I had employers look at my credit history and go "nope" because the machine said I wasn't trust-worthy. Should you trust the automation? Probably never, unless it knows how to deal with exceptions.

nyrikki 2y ago

Eastern Airlines Flight 401 in 1972 and Air France Flight 447 in 2009 are examples along with the recent problems with the 737 autothrottle system. Even modern driver training downplays the utility of ABS. This post was talking about relying on AI in the presence of uncertainty. Perhaps you aren't familiar with the term? "Automation bias is the propensity for humans to favor suggestions from automated decision-making systems and to ignore contradictory information made without automation, even if it is correct"

yencabulator 2y ago

The recently-famous Boeing 737 MAX crash of Lion Air Flight 610 happened due automation called MCAS. Apparently, the pilots couldn't override the automation in time to survive. https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings https://en.wikipedia.org/wiki/Boeing_737_MAX_groundings