7 ms·
F-35 Program Office Signs Off on Air Force 3i Software
- nateberkopec 11y ago> "The best analogy is you are starting up your computer and you want to use Word, Excel, PowerPoint and Outlook, and you are trying to get your work done for the day, and PowerPoint and Outlook came up but you are having an issue with Excel,” Brendan Rhatigan, director of engineering and test operations for the F-35 ITF, told Defense News last week at Edwards. “So you say, I don’t know what’s going on, so let me x out of that, let me restart it again.” So what did they do with this software update? Just install "CTRL+ALT+DEL"?
- digler999 11y agothey cite a design defect in MS products - designed to forward memes and cat pictures and add up a few columns of numbers - to justify design defects in aerospace (aka your-life-depends-on-it) code..
- chris_overseas 11y ago> Pilots found that jets’ systems would shut down midflight about once every three or four hours and have to be rebooted. > But the new, improved version of 3i is a significant improvement, according to the JPO. The new version showed approximately twice the level of stability as the previous load, Block 2B, and three times better stability than the original 3i software Does that mean it "only" needs to reboot once every 6-8 hours? If that's considered stable enough to sign off for production, does that mean they will simply reboot before each flight and just hope for the best?
- flyinglizard 11y agoThat's continuous delivery for you.
- protomyth 11y ago> Does that mean it "only" needs to reboot once every 6-8 hours? If that's considered stable enough to sign off for production, does that mean they will simply reboot before each flight and just hope for the best? Ferry flights must be an experience.
- engi_nerd 11y agoF-35s have ferried across the Atlantic without difficulty, and across the continental US on numerous occasions.
- hackuser 11y agoAccording to earlier articles about this issue, planes currently in wide use need to be 'rebooted' at a similar frequency, it's just specific subsystems that get rebooted (I think the F-35's former problem was a sensor), the planes keep flying, and missions don't often last that long anyway (IIRC all the details). Here, I found the link: https://news.ycombinator.com/item?id=11260927 https://news.ycombinator.com/item?id=11260927 EDIT: Heck, I'll just copy the meat of it here: http://breakingdefense.com/2016/02/bogdan-predicts-f-35s-for-less-than-80m-engines-included/ http://breakingdefense.com/2016/02/bogdan-predicts-f-35s-for... The toughest problem the program is having is matching the timing of the aircraft’s fusion software with its sensors’ software. “As we add different radar modes and as we add different and capabilities to the DAS system and to the EOTS system, the timing is misaligned,” and then you have to reboot it. Bogdan said he’s aiming for eight to nine hours between such software failures when a radar or DAS or EOTS needs to be rebooted, which is what legacy aircraft boast. Right now they are at four to five hours between such events. “That’s not a good metric.”
- Gravityloss 11y agoI guess if you demand a lot of interaction between almost all systems, it's harder to ensure high levels of independence.
- arca_vorago 11y agoWelcome to the perils of globalism...
- mlvljr 11y agoComing to those less-developed on the wings of F-35s! :)
- ArkyBeagle 11y agoIt's not hard at all - you buffer it and make the communications paths asynchronous. You allow dropped "packets" when necessary. I don't think you can enforce hard realtime constraints across subsystems and expect anything but tears.
- AJRF 11y agoWould a jet be flying missions for 6-8 hours anyway?
- engi_nerd 11y agoYes, with aerial refueling, missions of 12 hours or more are not unheard of. The pilot's endurance is the limiting factor. Or, as I've heard it said, "You worry about running out of ass, not running out of gas."
- billforsternz 11y agoMy intuition is that occasional software crashes like this probably follow a Poisson distribution statistically speaking. Like occasional cars appearing on a lonely street. One car appearing doesn't affect the expected arrival of the next car. So rebooting before the start of a mission is not an effective strategy. Just idle speculation based on a stats and probability class 36 years ago. Happy to be proven wrong.
- albinofrenchy 11y agoI somewhat doubt that model is valid for these issues. A large number of software issues relate to run time of a system. In particular, this one seems to be clock drift from other comments. That's more of a random walk; which crashes a system at some given distance.
- woodman 11y agoUnless the crashes are caused by things cosmic rays, they're very likely influenced by the internal state of the aircraft and therefor not independent events. If that car showing up causes the world to reset - then you won't likely see another car for a few million years.
- ArkyBeagle 11y agoIf I read the thing right, the system is in sync at reboot and effectively, jitter renders it unstable over time. So events are not independent.
- engi_nerd 11y agoWhen you start a mission you are literally starting up the plane. Along the way you turn on the avionics, authenticate, turn on the engine, ask the plane to perform a comprehensive set of built in tests, etc. You are booting up. It does not make sense to reboot before you've even booted up in the first place.
- Matthias247 11y agoIn general I would agree. But there could be indeed some SW failures that are more likely after a given time of running. E.g. memory [de]allocation problems, memory fragmentation - in case the use some kind of dynamic memory. Or other types of resource leaks.
- hackuser 11y agoDenmark's government likes it too: http://breakingdefense.com/2016/05/f-35-wins-denmark-competition-trounces-super-hornet-eurofighter/ http://breakingdefense.com/2016/05/f-35-wins-denmark-competi...
- ju-st 11y agoProbably they were "encouraged" by the US, too... (see https://en.wikipedia.org/wiki/Lockheed_Martin_F-35_Lightning_II_procurement#Norway https://en.wikipedia.org/wiki/Lockheed_Martin_F-35_Lightning... )
- kunai 11y agoWhy do government contractors refuse to use COTS hardware and software? Is it just a bureaucracy thing, tradition, or is there any significant ACTUAL benefit to not using something like RTLinux or QNX for systems instead of the proprietary mess of legacy spaghetti code they usually use instead?
- danielvf 11y agoFor the F-35, this is roughly the equivalent of asking why Google or Facebook doesn't use COTS software instead of mess of proprietary legacy code. The F-35's reason for existence is its software. Other existing aircraft are faster, can carry more, fly further, or be more stealthy. The F-35's value statement is that its pilots will know more about what's going on around it than any other combat aircraft, for the next ten or fifteen years. COTS stuff is used at the base layers of military avionics. LynxOS is used under the hood of the Army's very successful CAAS helicopter avionics system.
- mschuster91 11y agoProbably a shitload of certifications are missing...
- mavhc 11y ago"Much of the F-35's software is written in C and C++ due to programmer availability, Ada83 code also is reused from the F-22. The Integrity DO-178B real-time operating system (RTOS) from Green Hills Software runs on COTS Freescale PowerPC processors" "24 million lines of code" From https://en.wikipedia.org/wiki/Lockheed_Martin_F-35_Lightning_II https://en.wikipedia.org/wiki/Lockheed_Martin_F-35_Lightning... Integrity-178B is the DO-178B–compliant version of Integrity. It is used in several military jets such as the B-2, F-16, F-22 and F-35, as well as the commercial airframes Airbus A380. Its kernel's design guarantees bounded computation times by eliminating features such as dynamic memory allocation. The auditing and security engineering capabilities have allowed it to obtain the EAL6 rating by the NSA. Integrity-178B has a unique feature: an EAL6 rating. https://en.wikipedia.org/wiki/Integrity_%28operating_system%29 https://en.wikipedia.org/wiki/Integrity_%28operating_system%... EAL6: Semiformally Verified Design and Tested
- jfoutz 11y agoAs much as i want to hate on the F-35, mutter 1.5 trillion mutter, good for them. Shipping software is hard. By all accounts it's millions of lines of code. They're slowly picking off the critical bugs and upgrading.
- jokoon 11y agoThe more I hear about the F35 softare, the more it sounds like it is something you actually find in scifi movies. It really sounds like it is 20 years ahead, if not more. So of course it's going to fail at times. I really want to know the actual capabilities of the things, but I know I'll have to wait some years before russia or china are even able to copy those systems.
- natch 11y agoThe analogy they used to MS Word crashing once in a while doesn't quite fit. MS Word isn't loaded with jet fuel and flying over populated areas at high speed. I wouldn't be disappointed at all if they would keep this thing away from large cities for now.
- engi_nerd 11y agoThe portions of the software that need to be restarted have to do with avionics, not with the vehicle management computers that actually control the aircraft and keep it in the sky. The VMCs are triply redundant and are extremely reliable. The stuff that's making the news and causing all the problems has nothing to do with actually keeping the plane in the air.
- natch 11y agoCitation needed. And I'm not only concerned about the plane staying in the sky. I also want the jet fuel and the weapons to stay in the plane, to name a couple other items that are pluses. Nor do I have any confidence that a plane that stays in the sky today will do so tomorrow, given the record of this team. All kinds of things can go wrong[1]. Focusing on one aspect and claiming (without evidence) that this aspect is safe doesn't help a bit. And triply redundant means very little (as with a famous bomb accident example, sometimes all those redundancies are shown to still be cutting it very close)[2]. This is especially the case when the entire system is a poster child for projects going wrong[3]. 1. http://catless.ncl.ac.uk/Risks/ http://catless.ncl.ac.uk/Risks/ 2. http://www.theguardian.com/world/2013/sep/20/usaf-atomic-bomb-north-carolina-1961 http://www.theguardian.com/world/2013/sep/20/usaf-atomic-bom... 3. https://www.google.com/#q=f-35+failed+project https://www.google.com/#q=f-35+failed+project
- engi_nerd 11y agoThe split between the F-35 vehicle systems and mission systems is described in [0]. "This new design approach breaks avionics into two categories: mission systems and vehicle systems. The former includes tools that help the aircraft do its job, such as sensors, displays, and weapons. The latter are subsystems that help the aircraft function correctly, such as power generation, cooling, and flight control." For more on the Vehicle Managment Computers, see [1], particularly: ""Each F-35 will have three boxes, making it a triple-redundant system," said Tom Burbage, executive vice president and general manager of the Lockheed Martin JSF program. "Each box 'votes' and compares its decision with that of the others before executing a command -- a process that takes place in much less than the blink of an eye. If one or even two boxes were to be damaged or malfunction, the aircraft would continue to operate normally." The all-digital VMCs, which save weight and space while improving precision, are at the heart of the distributed F-35 Vehicle System." When we say "Block 3i" we are talking about the mission systems hardware[2] and software that are all about "sensors, displays, and weapons". All I intended was to point out this difference, and to point out that the vehicle systems are controlled by physically distinct components and simply saying "The F-35 software" isn't completely accurate. You'll note I made no claim about safety, nor do I intend to do so. Future performance of the system does not necessarily depend on its past performance. Allow me to point out,though, that through 50,000+ flight test hours, no F-35 has been lost through a flight control failure (again, no guarantee that there isn't some latent failure case awaiting discovery!) And, as an engineer who is struggling to tame some complexity on a rather bothersome flight test instrumentation system, I'm well aware of risks (I love Risks Digest!). Heck, check some of my post history, I'm often exhorting people to read John Gall's Systemantics and heed its lessons. Discuss technical failings of the program if you wish, but remarks about the "record of this team" and a google search to "f35 failed project" are rather offputting and are, in my humble opinion, needlessly negative. [0]: http://www.militaryaerospace.com/articles/print/volume-14/issue-5/features/special-report/f-35-jet-fighters-to-take-integrated-avionics-to-a-whole-new-level.html http://www.militaryaerospace.com/articles/print/volume-14/is... [1]:http://www.prnewswire.com/news-releases/first-lockheed-martin-f-35-joint-strike-fighter-vehicle-management-computer-delivered-55707452.html http://www.prnewswire.com/news-releases/first-lockheed-marti... [2]: http://www2.l-3com.com/displays/pdfs/redesign/ICP(2011)_LR.pdf http://www2.l-3com.com/displays/pdfs/redesign/ICP(2011)_LR.p...
- ArkyBeagle 11y agoI read these things, and I think "My disicipline - computer science - has failed." Rent seeking - now with more death.
- ilaksh 11y agoIt's not computer sciences failure it's a management issue that stems from a very primitive organizational design of the military. For example, the engineers did not sign off on buggy systems. The executive officers did because they had a contract with an archaic structure that is not compatible with software development and required the milestone to be met or there be penalities. But the reason the milestones weren't met in the first place was poor management and contract structure created by those executives in the first place. Biggest issue is too much ambition and too much budget trying to be crammed into one project. Other big issue is lack of closed feedback loops. It looks like the devs do not have a production system to do integration testing on. There are probably communication barriers between QA testers ie pilots and devs, caused again by poor structure and culture of military. Also likely very long lag between releases so few opportunities for iteration. Also likely using outdated programming languages like C++ for application-level logic. The nation-state's deadly enforcers eg military are directly descended from historical organized crime, since nations generally started as simply the most powerful families. Look at the first premise of this whole thing: we are going to fly around and missile/bomb the shit out of you if you don't go along with our global domination. So from the beginning it is poorly planned.
- ArkyBeagle 11y agoBut medicine works in the military. We just don't do "first, do no harm." This is just brought more sharply in focus by the nature of the F35. I understand completely your point. The distinctions between militaries and crime families is the chain of command and accountability - they're subject to oversight by a legitimate government. The hot job in DoD contractor is contracting itself - feed the beast. The actual work is a minor annoyance to them. It's just careerism on stilts.
- 11y ago
- zepearll 10y ago"The best analogy is you are starting up your computer and you want to use Word, Excel, PowerPoint and Outlook, and you are trying to get your work done for the day, and PowerPoint and Outlook came up but you are having an issue with Excel,” What about "Word"? Did it start?