3 ms·

The way you see AI being integrated into avionics SWE process - having to certify anything - I don't think that's how it's going to develop. What do you base t

• p1esk 4y ago
  Maybe I didn't express myself clearly. Let me try again. The way I see it develop in the next few years: 1. Avionics SWEs start using GPT models to help them do all the tasks you mentioned. This does not change any certification processes, because the resulting code will still be reviewed by humans (before being submitted for any certifications), and it will be certified the same as any human generated code. Managers can try to prohibit the use of AI models, but if models do a good job, people will find a way to bypass those restrictions. 2. Eventually, SWEs realize the models consistently do a better job than they possibly could, and start trusting the models. This will lead to less and less reviewing, and eventually SWEs will do little more than passing high level instructions from their managers to the models, and then passing the reports (also AI generated) about completed work back to the managers. 3. People who perform certifications of the code will follow the same path. AI models will become much better than humans in reviewing, testing, and whatever else is involved in the certification process. Eventually those people will too have very little to do other than write prompts. 4. Eventually management will convince themselves (and higherups) that things are working great, and as a cost saving measure, decide to get rid of the SWEs. 5. Same thing happens to the managers. If GPT-5/6/7 follow the same rate of improvement as what we saw with GPT-3/3.5/4 the above scenario will most likely happen within the next 5 years.
  • tjr 4y ago
    Maybe I am also not being clear. I agree with every single point you made, from the perspective of being, either now or in the future, technologically possible. However, unless the FAA accepts the output of AI tools in lieu of humans performing those same tasks, then it really doesn't matter. Without FAA approval, your code does not (legally) fly. [Speaking in the United States here; other regions have their own, likely similar, regulations.] Right now, there are software tools that replace humans in one function or another, in avionics software development. But the difference is these tools consistently produce expected output, and can be verified as such. The way the GPT-style tools currently behave, I think it would be very difficult to see the FAA approve them as a human-replacing code-generation tool. The path toward FAA-accepted usage of neural-network-based AI tools in avionics software is a point of research right now. Maybe we will get there. Maybe we will get partially there. I think that AI tools will make their way into avionics a lot more slowly than many other industries, not because of technical inadequacy, but because of procedural oversight.
    • p1esk 4y ago
      OK, I see your point. A couple of questions: 1. Do humans consistently produce expected output? Obviously not, so why would we trust them to write complicated software? 2. How does FAA know whether the code has been produced by a human, or by a human who asked GPT to write it? In both cases a human will have signed off on it.
      • tjr 4y ago
        Couldn't humans make mistakes? Sure. And that's why there are so many checks and balances in avionics software, compared to most other software industries. The more safety critical it is, the more it is scrutinized. Why would we not trust a software tool to write code just as well as a human, when humans themselves are fallible? I have not discussed this with any FAA representative, and have not yet found any published reference material that specifically lays out this case, but just to offer my personal, unofficial opinion based on past experience: I believe that an AI tool (like ChatGPT or what have you) would be considered, per RTCA document DO-330, an AutoCode Generator. If the code generated by a tool (such as an AI tool) is subsequently treated just as if it was written by a human -- subject to the same code reviews, the same verification tests, the same structural code coverage analysis tests, etc. -- then, in my personal opinion, that ought to be no different than if a human wrote the code. ==> Thus, at least for the code-writing itself, your enumerated Step 1 should be reasonably attainable. It's only after that where things start getting tricky. No matter how good the code quality seems to be, no matter how flawless it seems to be, no matter how much you intuitively come to trust the AI tool ... it's work still has to be checked. If you could formally qualify the tool that is generating the code, then you can get away with a lot less checking. If you could formally qualify tools to do the code reviews and the verification testing and so forth, then yes, in theory you could automate the entire process. How would the FAA know what tool(s) you used to generate code? Unlike in most industries, part of formally releasing avionics software includes declaring the software development tools used to create it. You might not need to cite a tool that generated code which was subsequently human-reviewed and tested, but I would not be confident about that. If you qualified that tool to generate code which was not human-reviewed and tested, then you would certainly need to declare it.