5 ms·

Yes, the blameless culture is a critical safety feature and it could be ruined very easily if the failed leadership in the industry starts looking for scapegoat

Yes, the blameless culture is a critical safety feature and it could be ruined very easily if the failed leadership in the industry starts looking for scapegoats. It's sad too because analyzing every cockpit conversation with AI to highlight things that may cause common confusion could be invaluable. Instead, the short-sighted leadership in today's business world will use it for (job) performance analysis and to penalize workers for failing to act like robots :-(

mulmen 3y ago
Possibly. But as a technologist I don’t trust AI to actually deliver such a result in our generation. The system will inevitably be created by modern technologists without intimate domain knowledge. It will be motivated by short-term deadline chasing and MVP culture. The cockpit of an airliner is no place for bleeding edge technological innovation. I don’t need an Airbus with Siri. Pilots already spend hours in training every year. Is reviewing the output of this AI system really a better use of their time? What problem is it even solving?
- kelseyfrog 3y ago
  What would it take to make you trust AI?
  - mulmen 3y ago
    Decades of development and successful deployment first in non-safety-critical settings then further demonstration in progressively higher stakes environments. In addition demonstration that the technology has any benefit to this domain at all. My concerns have more to do with the technology industry than AI itself. AI is a buzzword and the technology as it exists is of dubious benefit here. We have an unhealthy tendency to deploy technology too early and for the wrong reasons. These are undesirable impulses in aviation.
    - kelseyfrog 3y ago
      Thank you for your honesty. Are there any other industries that you hold at a similar standard?
      - rlpb 3y ago
        I can't speak for GP, but "safety critical" is the term relevant to this thread I think. Any situation where people die if you screw up, where that includes screwing up at a higher level, such as introducing ill-thought-out process changes.
        kelseyfrog 3y ago
        Thank you. That's a good grounding that helps orient toward what I feel is being conveyed. I sense there's a lot of overlap with SCADA-enabled infrastructure: transportation, nuclear, industrial control, &c. And we should be hesitant to deploy AI in those environments. Am I getting that right?
    - Aeolun 3y ago
      Analysing cockpit voice recordings for confusing terms (using AI or not) does not strike me as something that would affect the operation of the aircraft in any shape or form?
      - mulmen 3y ago
        You’re assuming the AI works and does what you expect. Neither of those are reasonable assumptions. Everything has an opportunity cost. To be beneficial AI has to be better than the status quo to be. In aviation that’s a much higher bar than “not nothing”. It has to be better than the time saturation flight crews already experience.
        Aeolun 3y ago
        I’m not assuming the AI works at all. Maybe we are talking about different things? If the flight is over, and some analysis spits out a bunch of garbage, how does that hurt anyone? Nobody is going to tell pilots that they should stop using the word ‘and’ even if the AI spits that out as a possible point of improvement.

mulmen 3y ago

Possibly. But as a technologist I don’t trust AI to actually deliver such a result in our generation. The system will inevitably be created by modern technologists without intimate domain knowledge. It will be motivated by short-term deadline chasing and MVP culture. The cockpit of an airliner is no place for bleeding edge technological innovation. I don’t need an Airbus with Siri. Pilots already spend hours in training every year. Is reviewing the output of this AI system really a better use of their time? What problem is it even solving?

kelseyfrog 3y ago
What would it take to make you trust AI?
- mulmen 3y ago
  Decades of development and successful deployment first in non-safety-critical settings then further demonstration in progressively higher stakes environments. In addition demonstration that the technology has any benefit to this domain at all. My concerns have more to do with the technology industry than AI itself. AI is a buzzword and the technology as it exists is of dubious benefit here. We have an unhealthy tendency to deploy technology too early and for the wrong reasons. These are undesirable impulses in aviation.
  - kelseyfrog 3y ago
    Thank you for your honesty. Are there any other industries that you hold at a similar standard?
    - rlpb 3y ago
      I can't speak for GP, but "safety critical" is the term relevant to this thread I think. Any situation where people die if you screw up, where that includes screwing up at a higher level, such as introducing ill-thought-out process changes.
      - kelseyfrog 3y ago
        Thank you. That's a good grounding that helps orient toward what I feel is being conveyed. I sense there's a lot of overlap with SCADA-enabled infrastructure: transportation, nuclear, industrial control, &c. And we should be hesitant to deploy AI in those environments. Am I getting that right?
  - Aeolun 3y ago
    Analysing cockpit voice recordings for confusing terms (using AI or not) does not strike me as something that would affect the operation of the aircraft in any shape or form?
    - mulmen 3y ago
      You’re assuming the AI works and does what you expect. Neither of those are reasonable assumptions. Everything has an opportunity cost. To be beneficial AI has to be better than the status quo to be. In aviation that’s a much higher bar than “not nothing”. It has to be better than the time saturation flight crews already experience.
      - Aeolun 3y ago
        I’m not assuming the AI works at all. Maybe we are talking about different things? If the flight is over, and some analysis spits out a bunch of garbage, how does that hurt anyone? Nobody is going to tell pilots that they should stop using the word ‘and’ even if the AI spits that out as a possible point of improvement.