15 ms·

F-35 radar system has bug that requires hard reboot in flight

124 points by pavornyoh 11y ago

btbuildem 11y ago
"Have you tried turning it off and on again?"
deleted 11y ago
[deleted]
Raphmedia 11y ago
> F-35 radar system has bug that requires hard reboot [of the radar] in flight
- pc86 11y ago
  To be fair, a 4-hour reboot on the radar is pretty bad to have in-flight.
  - Demiurge 11y ago
    I don't understand, why are people downvoting comments like yours without any explanation for those without the background on this :\
    - Raphmedia 11y ago
      I have no idea. I posted my comment because the title baited me into clicking it because I was under the impression that those planes were so bad they required a full reboot. His answer seems valid to me.
      - barkingcat 11y ago
        It was a reading comprehension error. You're making the same mistake in reading if you think pc86's answer is valid.
        Raphmedia 11y ago
        Is it? English is not my main language. I must not be alone that have read this the way I did because my initial comment was upvoted a lot. If the radar's bug required to reboot the entire plane, how would you write it without being redondant? "F-35 radar system has bug that requires hard reboot of the F-35 in flight"?
        barkingcat 11y ago
        The comprehension error in question is not that it requires a reboot, but that the reboot is a "4-hour reboot". The article indicated a reboot every 4 hours, not a reboot that lasts 4 hours. That's the only error in question - and that's why it was downvoted. There's no question that the radar needs a reboot, but to say it's a 4-hour reboot is factually wrong and inflammatory based on an incorrect assumption.
      - deleted 11y ago
        [deleted]
    - Sharlin 11y ago
      Because it was a simple reading comprehension fail, not specifically related to the subject at hand.
      - solipsism 11y ago
        If downvotes only affected the specific comment, that would make complete sense, but they also affect karma. I guess it's not irrational to punish people who consistently misunderstand things, but is that a purposeful decision or an unfortunate byproduct of the karma system?
        hackuser 11y ago
        The idea isn't to punish people, IMHO, but to improve content for others reading the conversation. That comment doesn't contribute to the conversation, and in fact it confuses it, so it gets downvoted to where few will read it. Voting is about the readers, not the commenters. Personally, other than for determining specific capabilities such as downvoting, I think tracking karma per user does little good. Karma per comment - a bit of feedback on your comment - is useful.
        barkingcat 11y ago
        it does punish people who consistently post opinions based on wrong information, and do not edit themselves, but that's a side effect. If the poster didn't want to be downvoted, they should read and think about the topic before posting. In general, HN is pretty good at not downvoting for dissenting opinions, but the group does downvote heavily for wrong info or posts that are inflammatory based on the wrong facts.
      - Demiurge 11y ago
        I realized that after the other comment. So, to me, at least, t's better to just explain the error, regardless of how banal it is.
    - kelvin0 11y ago
      Unfortunately this happens to create lots of frustration on the part on newcomers to HN. Karma is King, so you can be downvoted by the Elite, you should check the HN guidelines.
    - barkingcat 11y ago
      reading comprehension error on the part of the poster.
  - andrewstuart2 11y ago
    4 hours mean time between reboots means they typically reboot every 4 hours, not that a reboot takes 4 hours.
  - deleted 11y ago
    [deleted]
awinter-py 11y ago
Though to be fair, the software isn't due to reach 1.0 until 2050 (assuming the pentagon can scrape up another billion dollars to train ada programmers). No surprise that alpha software is bug-ridden.
- msbarnett 11y ago
  > assuming the pentagon can scrape up another billion dollars to train ada programmers The F-35's avionics are mostly in C++. The F-22 was the last fighter program to use Ada heavily AFAIK.
  - jpgvm 11y ago
    That seems sad. Ada seems like it would reduce costs significantly in programs that have a very long life.
- deleted 11y ago
  [deleted]
- zodPod 11y ago
  Holy CRAP. Seriously?! That's insane! How is it going to take them more than 30 years to develop it?!
  - smeyer 11y ago
    >Seriously?! No. Here's a timeline of the plane so far [0] and a page that includes some details from 2014 including a master schedule and a production status [1]. The 2050 comment was presumably an exaggeration referencing the frequent problems and delays with the development. [0] http://aviationweek.com/F-35Milestones http://aviationweek.com/F-35Milestones [1] http://intercepts.defensenews.com/2014/09/the-current-status-of-the-f-35-in-three-charts/ http://intercepts.defensenews.com/2014/09/the-current-status...
    - zodPod 11y ago
      Ah, alright, thank you!
hacker_9 11y ago
'Have you tried turning it off and on?'
- engi_nerd 11y ago
  You joke, but for so many things in aviation, this is the answer. There are also a lot of problems where the answer is "re-poll all your interrupts, that problem you (the computer) think is a problem shouldn't be anymore".
sliverstorm 11y ago
So? The SR-71 has (had?) a bug that required both engines to restart in flight. Frequently.
- alayne 11y ago
  Do you have a reference for this?
  - beamatronic 11y ago
    How about this? http://www.airspacemag.com/military-aviation/how-things-work-supersonic-inlets-35428453/?no-ist http://www.airspacemag.com/military-aviation/how-things-work... "During some Blackbird flights, however, the harmonious working of the spike and the forward and aft bypass doors broke down, and all too quickly the inlet was filled with more air than it could handle. When the air pressure inside the inlet became too great, the normal shock wave was suddenly belched out of the inlet in an unstart, accompanied by an instantaneous loss of air flow to the engine, an enormous increase in drag, and a significant yaw to the side with the affected inlet. Unstarts occurred 'when you least expected them—all relaxed and taking in the magnificent view from 75,000 feet,' wrote Graham in SR-71 Revealed. If the crew’s attempts to restart the inlet’s supersonic flow failed, they would have to slow their aircraft to subsonic speeds."
  - dogma1138 11y ago
    It wasn't really a "bug" but more that the computers could not keep up with changes in the flight conditions so if the engine spikes were not in the correct position for the speed and altitude it could cause a blow back which would extinguish the engines which will require the pilots to restart them. https://en.wikipedia.org/wiki/Lockheed_SR-71_Blackbird#Air_inlets https://en.wikipedia.org/wiki/Lockheed_SR-71_Blackbird#Air_i...
    - gherkin0 11y ago
      It also looks like they later mitigated it as technology improved: > Lockheed later installed an electronic control to detect unstart conditions and perform this reset action without pilot intervention. Beginning in 1980, the analog inlet control system was replaced by a digital system, which reduced unstart instances.
iblaine 11y ago
> Lockheed Martin has discovered the cause of the problem and has diverted developers who were working on the next increment of the F-35's code to fix it. A patch is expected by the end of March. Much ado about nothing.
- s_m_t 11y ago
  Seriously, isn't this to be expected? I know a lot of people have a massive hate boner for the f-35 because they read a lot of shitty war blogs but for a bunch of coders to act incredulous towards this information is absolutely ridiculous. How many bugs have any of you introduced and fixed in the past week?
  - solipsism 11y ago
    Most of us are working on things that are several orders of magnitude less likely to get someone killed than the F-35 radar code. The goal for safety-critical embedded flight code is 0 hard reboots mid-flight. Is that controversial, in your opinion? Edit: "standard for safety-critical embedded flight" was a bad word choice and wasn't what I meant. s/standard/goal
    - icegreentea 11y ago
      They found a bug in testing and are fixing it. That's literally why you test critical code, in case you didn't get it right.
      - solipsism 11y ago
        I realize this. But bugs are sometimes worth talking about. Many people feel that a bug requiring a hard reboot mid-flight in a 5th generation fighter, a problem apparently not able to be overcome through multiple generations of fighters, is worth talking about. If you, or the original complainer, disagree, you're free to not participate in the conversation, but complaining about the rest of us talking about this bug is annoying. Quoting from Boeing: The F-35 has the most robust communications suite of any fighter aircraft built, to date. Components include the AESA radar, EOTS targeting system, Distributed Aperture System (DAS), Helmet Mounted Display (HMD), and the Communications, Navigation and Identification (CNI) Avionics. If by robust they mean "you can go 8 hours instead of 4 hours between hard reboots", I think that's worth talking about.
hackuser 11y ago
I read elsewhere, from a reliable source that I can't find right now. that the mean time between reboots on 'mature' fighter planes is 8 hours and the F-35 is at around 4 hours so far with plans to improve it. It was the first number that shocked me the most. EDIT: I found it: http://breakingdefense.com/2016/02/bogdan-predicts-f-35s-for-less-than-80m-engines-included/ http://breakingdefense.com/2016/02/bogdan-predicts-f-35s-for... The toughest problem the program is having is matching the timing of the aircraft’s fusion software with its sensors’ software. “As we add different radar modes and as we add different and capabilities to the DAS system and to the EOTS system, the timing is misaligned,” and then you have to reboot it. Bogdan said he’s aiming for eight to nine hours between such software failures when a radar or DAS or EOTS needs to be rebooted, which is what legacy aircraft boast. Right now they are at four to five hours between such events. “That’s not a good metric.” I don't understand a lot of that, but I think the fusion softwre refers to a unified UI for the pilot that merges information from the plane's very many sensors and presents it in a comprehensible way on one screen. Apparently pilots in older planes suffer from data overload and from watching many different screens at once - all while trying to fly a plane in combat. UI is reported to be a highly consequential improvement for the new plane.
- raverbashing 11y ago
  I can only imagine such sad state of affairs comes from extremely quirky requirements and very tight and crummy process (from TPS reports to worse) Yes, I can understand the problem is complicated, but come on.
- jacquesm 11y ago
  How many fighter plane flights last longer than 8 hours? It's a mean time after all, it could simply be a reflection of the length of the sortie.
  - bluedino 11y ago
    Probably not a lot but it wouldn't be uncommon for an American plane to be in the air for 12+ hours with in-air refueling.
themodelplumber 11y ago
Anybody know how long the reboot takes? That seems like important information. Especially if e.g. your AAMs need directions from the master system (?) in order to take over tracking by themselves.
- bra-ket 11y ago
  3hr and 55 min
dboshardy 11y ago
Looks like they need to add a "Turn Radar Off and On Again" to their FENCE IN.
sean-duffy 11y ago
According to a lot of people in this thread[0], it is pretty much expected that the radar in any fighter jet will need rebooting from time to time. [0] https://www.reddit.com/r/technology/comments/49iegx/radar_glitch_requires_f35_fighter_jet_pilots_to/ https://www.reddit.com/r/technology/comments/49iegx/radar_gl...
- gherkin0 11y ago
  At one time, it was "expected" that you'd have to reboot your PC multiple times per day.
- neurotech1 11y ago
  This is correct. A radar or mission avionics system reset isn't a critical emergency. It may force a mission abort, but the jet is still flyable. A reset event would be more worrying if it was a FCS (Flight Control System) computer reset in flight. Air Asia 8501 did something similar and the jet crashed into the sea. An F-22 pilot failed to properly reset his FCS before takeoff, and destroyed a $150m+ aircraft at Nellis AFB.
bsamuels 11y ago
this doesn't surprise me one bit. I interviewed at one of the major defense contractors for a programming job last year and the technical interviewer had never heard of C# this was for a java position
- tclmeelmo 11y ago
  What use would a major defense contractor have with C#? And on what basis do you assume their code base is poor?
  - bsamuels 11y ago
    for someone to have never even heard of C# and to be working full time in a senior level programming job, you have to be completely and utterly isolated from any kind of programming community. just because I don't use node js, ruby, or perl in my work doesnt mean i have an excuse to not know that they even exist. how can you choose the best tool for a job if you only know C++ and Ada?
    - tclmeelmo 11y ago
      > for someone to have never even heard of C# and to be working full time in a senior level programming job, you have to be completely and utterly isolated from any kind of programming community. The majority of defense contractors I know don't spend time in the "programming community" in their off hours, both because it's contractually too cumbersome to do so, and because they're doing other things in their spare time. They may spend time in the "defense community". It's very different culturally from SV. > just because I don't use node js, ruby, or perl in my work doesnt mean i have an excuse to not know that they even exist. how can you choose the best tool for a job if you only know C++ and Ada? node.js, ruby, and perl are not acceptable for use on real-time systems nor safety-critical systems.
    - vonmoltke 11y ago
      > how can you choose the best tool for a job if you only know C++ and Ada? Having worked in that world for almost 10 years, I can tell you that this choice is not up to you. You are told what tools you will be using, typically driven by government customer "experts" and inter-company politics.
l3m0ndr0p 11y ago
Do they initiate the reboot with CTRL-ALT-DEL sequence?
Someone 11y ago
So, how fast does it reboot, and how much does this affect the other software components and operating the plane? Not that I expect it, but this could be a minor issue, if this 'radar system' is as good as stateless (and not, for example, a component that tracks friends and foes over time) and if it reboots in milliseconds.
godgod 11y ago
Why don't we ask the Chinese for their fix?
crb002 11y ago
How much you want to bet that simply running Valgrind would have caught it?
- officialchicken 11y ago
  Everything... valgrind can not and does not fix GIGO.
BinaryIdiot 11y ago
Had a bug. Everywhere I read said they already delivered a fix for this but I've noticed some news outlets not mentioning it. Though I gotta say this damn jet has been a nightmare. Since they'll never stop working towards newer and newer jets I'm curious if lessons learned here will apply to the next one. After doing government contracting for many years where we don't get the "lessons learned" from the previous contractors or much of anything (I've had to FIGHT to get source code for a product we were supposed to update; ended up having to rewrite the damn thing) I'm curious how it works when they contract out to have a jet built.
twoodfin 11y ago
I assume that bugs requiring reboots to resolve is part of the design of the failure model. When a software fault is detected, its consequences are unknowable. Those consequences could include killing the pilot and destroying his $100M aircraft. Therefore, the default response is to reset everything and get to a known-good state ASAP. This is not a bad general-purpose failure strategy: It's been believed for decades that most software faults encountered in production are Heisenbugs[1], and disappear rather than reproduce. [1] http://research.microsoft.com/en-us/um/people/gray/papers/TandemTR85.7_WhyDoComputersStop.pdf http://research.microsoft.com/en-us/um/people/gray/papers/Ta...
- macintux 11y ago
  Indeed. Hence Tandem's success with hardware and Erlang's with software. Crashing doesn't have to be crashing.
maxxxxx 11y ago
I am getting a little tired of the "F-35 is bad" headlines. There should be more context that tells us how common such problems are in general. I remember listening to a podcast where they said that SR-71 pilots had to restart their engine every so often. Same for the Concorde. Sounds really bad but neither of these problems hurt the airplanes' usability and were not really a problem. I firmly believe that military procurement is a mess but the discussion reminds me of the political discussion where Republicans jump on anything that goes wrong under Democrats and vice versa.
- Jtsummers 11y ago
  To be fair, the SR-71 was a highly specialized, and ultimately, highly experimental aircraft at its time. The Concorde was similar. Only a few were ever made, and they were never expected to be the ubiquitous aircraft of their time or even their type (reconnaissance and passenger transport). The F-35 is intended to replace several aircraft models which currently satisfy more specialized roles. It was intended to be the ubiquitous (in various configurations) fighter aircraft for the US military (and some allies). This problem may not be the worst of its issues, it may not even be a major issue or really (from the pilots' perspective) an issue at all since similar reboots (but less frequent) are the norm for them in other aircraft. But many other issues are demonstrating, and the reduced production numbers and increased production and development costs reveal, that there are numerous problems with this project in particular. It shows failings within the US political structure (that's essentially forced this project to continue), DoD procurement (that allowed many of these issues to occur initially), defense contractor procedures (LM, IIRC, doubling or tripling the number of engineers on the software side to "speed things up"). Really, it's going to be a classic for future engineers, computer scientists, MBAs and others to study. So I guess that's a good thing.
  - maxxxxx 11y ago
    I totally agree that the F-35 project is a mess. But I would like to see a real analysis of the problems and not just jumping on any problem that sounds bad on the surface. Maybe a hard reboot is a problem, maybe not. I don't know.
NiftyFifty 11y ago
That's like when my power goes out in my house ... I have to reboot my light switches ...
Havoc 11y ago
Judging by reddit comments this type of thing is pretty common with fighter jets which is kinda surprising to me. I would have thought that this is one area where reliability is highly prized. Along with space shuttle software & other life/death stuff etc. Instead "bash it with a hammer and maybe reset it" seems widely accepted. Weird.
- deleted 11y ago
  [deleted]
matt_wulfeck 11y ago
I bet the Chinese read about the F-35 fiasco and just laugh at us. The amount of money we're throwing down the hole here is staggering.

124 points by pavornyoh 11y ago

btbuildem 11y ago

"Have you tried turning it off and on again?"

deleted 11y ago

[deleted]

Raphmedia 11y ago

> F-35 radar system has bug that requires hard reboot [of the radar] in flight

pc86 11y ago
To be fair, a 4-hour reboot on the radar is pretty bad to have in-flight.
- Demiurge 11y ago
  I don't understand, why are people downvoting comments like yours without any explanation for those without the background on this :\
  - Raphmedia 11y ago
    I have no idea. I posted my comment because the title baited me into clicking it because I was under the impression that those planes were so bad they required a full reboot. His answer seems valid to me.
    - barkingcat 11y ago
      It was a reading comprehension error. You're making the same mistake in reading if you think pc86's answer is valid.
      - Raphmedia 11y ago
        Is it? English is not my main language. I must not be alone that have read this the way I did because my initial comment was upvoted a lot. If the radar's bug required to reboot the entire plane, how would you write it without being redondant? "F-35 radar system has bug that requires hard reboot of the F-35 in flight"?
        barkingcat 11y ago
        The comprehension error in question is not that it requires a reboot, but that the reboot is a "4-hour reboot". The article indicated a reboot every 4 hours, not a reboot that lasts 4 hours. That's the only error in question - and that's why it was downvoted. There's no question that the radar needs a reboot, but to say it's a 4-hour reboot is factually wrong and inflammatory based on an incorrect assumption.
    - deleted 11y ago
      [deleted]
  - Sharlin 11y ago
    Because it was a simple reading comprehension fail, not specifically related to the subject at hand.
    - solipsism 11y ago
      If downvotes only affected the specific comment, that would make complete sense, but they also affect karma. I guess it's not irrational to punish people who consistently misunderstand things, but is that a purposeful decision or an unfortunate byproduct of the karma system?
      - hackuser 11y ago
        The idea isn't to punish people, IMHO, but to improve content for others reading the conversation. That comment doesn't contribute to the conversation, and in fact it confuses it, so it gets downvoted to where few will read it. Voting is about the readers, not the commenters. Personally, other than for determining specific capabilities such as downvoting, I think tracking karma per user does little good. Karma per comment - a bit of feedback on your comment - is useful.
      - barkingcat 11y ago
        it does punish people who consistently post opinions based on wrong information, and do not edit themselves, but that's a side effect. If the poster didn't want to be downvoted, they should read and think about the topic before posting. In general, HN is pretty good at not downvoting for dissenting opinions, but the group does downvote heavily for wrong info or posts that are inflammatory based on the wrong facts.
    - Demiurge 11y ago
      I realized that after the other comment. So, to me, at least, t's better to just explain the error, regardless of how banal it is.
  - kelvin0 11y ago
    Unfortunately this happens to create lots of frustration on the part on newcomers to HN. Karma is King, so you can be downvoted by the Elite, you should check the HN guidelines.
  - barkingcat 11y ago
    reading comprehension error on the part of the poster.
- andrewstuart2 11y ago
  4 hours mean time between reboots means they typically reboot every 4 hours, not that a reboot takes 4 hours.
- deleted 11y ago
  [deleted]

awinter-py 11y ago

Though to be fair, the software isn't due to reach 1.0 until 2050 (assuming the pentagon can scrape up another billion dollars to train ada programmers). No surprise that alpha software is bug-ridden.

msbarnett 11y ago
> assuming the pentagon can scrape up another billion dollars to train ada programmers The F-35's avionics are mostly in C++. The F-22 was the last fighter program to use Ada heavily AFAIK.
- jpgvm 11y ago
  That seems sad. Ada seems like it would reduce costs significantly in programs that have a very long life.
deleted 11y ago
[deleted]
zodPod 11y ago
Holy CRAP. Seriously?! That's insane! How is it going to take them more than 30 years to develop it?!
- smeyer 11y ago
  >Seriously?! No. Here's a timeline of the plane so far [0] and a page that includes some details from 2014 including a master schedule and a production status [1]. The 2050 comment was presumably an exaggeration referencing the frequent problems and delays with the development. [0] http://aviationweek.com/F-35Milestones http://aviationweek.com/F-35Milestones [1] http://intercepts.defensenews.com/2014/09/the-current-status-of-the-f-35-in-three-charts/ http://intercepts.defensenews.com/2014/09/the-current-status...
  - zodPod 11y ago
    Ah, alright, thank you!

hacker_9 11y ago

'Have you tried turning it off and on?'

engi_nerd 11y ago
You joke, but for so many things in aviation, this is the answer. There are also a lot of problems where the answer is "re-poll all your interrupts, that problem you (the computer) think is a problem shouldn't be anymore".

sliverstorm 11y ago

So? The SR-71 has (had?) a bug that required both engines to restart in flight. Frequently.

alayne 11y ago
Do you have a reference for this?
- beamatronic 11y ago
  How about this? http://www.airspacemag.com/military-aviation/how-things-work-supersonic-inlets-35428453/?no-ist http://www.airspacemag.com/military-aviation/how-things-work... "During some Blackbird flights, however, the harmonious working of the spike and the forward and aft bypass doors broke down, and all too quickly the inlet was filled with more air than it could handle. When the air pressure inside the inlet became too great, the normal shock wave was suddenly belched out of the inlet in an unstart, accompanied by an instantaneous loss of air flow to the engine, an enormous increase in drag, and a significant yaw to the side with the affected inlet. Unstarts occurred 'when you least expected them—all relaxed and taking in the magnificent view from 75,000 feet,' wrote Graham in SR-71 Revealed. If the crew’s attempts to restart the inlet’s supersonic flow failed, they would have to slow their aircraft to subsonic speeds."
- dogma1138 11y ago
  It wasn't really a "bug" but more that the computers could not keep up with changes in the flight conditions so if the engine spikes were not in the correct position for the speed and altitude it could cause a blow back which would extinguish the engines which will require the pilots to restart them. https://en.wikipedia.org/wiki/Lockheed_SR-71_Blackbird#Air_inlets https://en.wikipedia.org/wiki/Lockheed_SR-71_Blackbird#Air_i...
  - gherkin0 11y ago
    It also looks like they later mitigated it as technology improved: > Lockheed later installed an electronic control to detect unstart conditions and perform this reset action without pilot intervention. Beginning in 1980, the analog inlet control system was replaced by a digital system, which reduced unstart instances.

iblaine 11y ago

> Lockheed Martin has discovered the cause of the problem and has diverted developers who were working on the next increment of the F-35's code to fix it. A patch is expected by the end of March. Much ado about nothing.

s_m_t 11y ago
Seriously, isn't this to be expected? I know a lot of people have a massive hate boner for the f-35 because they read a lot of shitty war blogs but for a bunch of coders to act incredulous towards this information is absolutely ridiculous. How many bugs have any of you introduced and fixed in the past week?
- solipsism 11y ago
  Most of us are working on things that are several orders of magnitude less likely to get someone killed than the F-35 radar code. The goal for safety-critical embedded flight code is 0 hard reboots mid-flight. Is that controversial, in your opinion? Edit: "standard for safety-critical embedded flight" was a bad word choice and wasn't what I meant. s/standard/goal
  - icegreentea 11y ago
    They found a bug in testing and are fixing it. That's literally why you test critical code, in case you didn't get it right.
    - solipsism 11y ago
      I realize this. But bugs are sometimes worth talking about. Many people feel that a bug requiring a hard reboot mid-flight in a 5th generation fighter, a problem apparently not able to be overcome through multiple generations of fighters, is worth talking about. If you, or the original complainer, disagree, you're free to not participate in the conversation, but complaining about the rest of us talking about this bug is annoying. Quoting from Boeing: The F-35 has the most robust communications suite of any fighter aircraft built, to date. Components include the AESA radar, EOTS targeting system, Distributed Aperture System (DAS), Helmet Mounted Display (HMD), and the Communications, Navigation and Identification (CNI) Avionics. If by robust they mean "you can go 8 hours instead of 4 hours between hard reboots", I think that's worth talking about.

hackuser 11y ago

I read elsewhere, from a reliable source that I can't find right now. that the mean time between reboots on 'mature' fighter planes is 8 hours and the F-35 is at around 4 hours so far with plans to improve it. It was the first number that shocked me the most. EDIT: I found it: http://breakingdefense.com/2016/02/bogdan-predicts-f-35s-for-less-than-80m-engines-included/ http://breakingdefense.com/2016/02/bogdan-predicts-f-35s-for... The toughest problem the program is having is matching the timing of the aircraft’s fusion software with its sensors’ software. “As we add different radar modes and as we add different and capabilities to the DAS system and to the EOTS system, the timing is misaligned,” and then you have to reboot it. Bogdan said he’s aiming for eight to nine hours between such software failures when a radar or DAS or EOTS needs to be rebooted, which is what legacy aircraft boast. Right now they are at four to five hours between such events. “That’s not a good metric.” I don't understand a lot of that, but I think the fusion softwre refers to a unified UI for the pilot that merges information from the plane's very many sensors and presents it in a comprehensible way on one screen. Apparently pilots in older planes suffer from data overload and from watching many different screens at once - all while trying to fly a plane in combat. UI is reported to be a highly consequential improvement for the new plane.

raverbashing 11y ago
I can only imagine such sad state of affairs comes from extremely quirky requirements and very tight and crummy process (from TPS reports to worse) Yes, I can understand the problem is complicated, but come on.
jacquesm 11y ago
How many fighter plane flights last longer than 8 hours? It's a mean time after all, it could simply be a reflection of the length of the sortie.
- bluedino 11y ago
  Probably not a lot but it wouldn't be uncommon for an American plane to be in the air for 12+ hours with in-air refueling.

themodelplumber 11y ago

Anybody know how long the reboot takes? That seems like important information. Especially if e.g. your AAMs need directions from the master system (?) in order to take over tracking by themselves.

bra-ket 11y ago
3hr and 55 min

dboshardy 11y ago

Looks like they need to add a "Turn Radar Off and On Again" to their FENCE IN.

sean-duffy 11y ago

According to a lot of people in this thread[0], it is pretty much expected that the radar in any fighter jet will need rebooting from time to time. [0] https://www.reddit.com/r/technology/comments/49iegx/radar_glitch_requires_f35_fighter_jet_pilots_to/ https://www.reddit.com/r/technology/comments/49iegx/radar_gl...

gherkin0 11y ago
At one time, it was "expected" that you'd have to reboot your PC multiple times per day.
neurotech1 11y ago
This is correct. A radar or mission avionics system reset isn't a critical emergency. It may force a mission abort, but the jet is still flyable. A reset event would be more worrying if it was a FCS (Flight Control System) computer reset in flight. Air Asia 8501 did something similar and the jet crashed into the sea. An F-22 pilot failed to properly reset his FCS before takeoff, and destroyed a $150m+ aircraft at Nellis AFB.

bsamuels 11y ago

this doesn't surprise me one bit. I interviewed at one of the major defense contractors for a programming job last year and the technical interviewer had never heard of C# this was for a java position

tclmeelmo 11y ago
What use would a major defense contractor have with C#? And on what basis do you assume their code base is poor?
- bsamuels 11y ago
  for someone to have never even heard of C# and to be working full time in a senior level programming job, you have to be completely and utterly isolated from any kind of programming community. just because I don't use node js, ruby, or perl in my work doesnt mean i have an excuse to not know that they even exist. how can you choose the best tool for a job if you only know C++ and Ada?
  - tclmeelmo 11y ago
    > for someone to have never even heard of C# and to be working full time in a senior level programming job, you have to be completely and utterly isolated from any kind of programming community. The majority of defense contractors I know don't spend time in the "programming community" in their off hours, both because it's contractually too cumbersome to do so, and because they're doing other things in their spare time. They may spend time in the "defense community". It's very different culturally from SV. > just because I don't use node js, ruby, or perl in my work doesnt mean i have an excuse to not know that they even exist. how can you choose the best tool for a job if you only know C++ and Ada? node.js, ruby, and perl are not acceptable for use on real-time systems nor safety-critical systems.
  - vonmoltke 11y ago
    > how can you choose the best tool for a job if you only know C++ and Ada? Having worked in that world for almost 10 years, I can tell you that this choice is not up to you. You are told what tools you will be using, typically driven by government customer "experts" and inter-company politics.

l3m0ndr0p 11y ago

Do they initiate the reboot with CTRL-ALT-DEL sequence?

Someone 11y ago

So, how fast does it reboot, and how much does this affect the other software components and operating the plane? Not that I expect it, but this could be a minor issue, if this 'radar system' is as good as stateless (and not, for example, a component that tracks friends and foes over time) and if it reboots in milliseconds.

godgod 11y ago

Why don't we ask the Chinese for their fix?

crb002 11y ago

How much you want to bet that simply running Valgrind would have caught it?

officialchicken 11y ago
Everything... valgrind can not and does not fix GIGO.

BinaryIdiot 11y ago

Had a bug. Everywhere I read said they already delivered a fix for this but I've noticed some news outlets not mentioning it. Though I gotta say this damn jet has been a nightmare. Since they'll never stop working towards newer and newer jets I'm curious if lessons learned here will apply to the next one. After doing government contracting for many years where we don't get the "lessons learned" from the previous contractors or much of anything (I've had to FIGHT to get source code for a product we were supposed to update; ended up having to rewrite the damn thing) I'm curious how it works when they contract out to have a jet built.

twoodfin 11y ago

I assume that bugs requiring reboots to resolve is part of the design of the failure model. When a software fault is detected, its consequences are unknowable. Those consequences could include killing the pilot and destroying his $100M aircraft. Therefore, the default response is to reset everything and get to a known-good state ASAP. This is not a bad general-purpose failure strategy: It's been believed for decades that most software faults encountered in production are Heisenbugs[1], and disappear rather than reproduce. [1] http://research.microsoft.com/en-us/um/people/gray/papers/TandemTR85.7_WhyDoComputersStop.pdf http://research.microsoft.com/en-us/um/people/gray/papers/Ta...

macintux 11y ago
Indeed. Hence Tandem's success with hardware and Erlang's with software. Crashing doesn't have to be crashing.

maxxxxx 11y ago

I am getting a little tired of the "F-35 is bad" headlines. There should be more context that tells us how common such problems are in general. I remember listening to a podcast where they said that SR-71 pilots had to restart their engine every so often. Same for the Concorde. Sounds really bad but neither of these problems hurt the airplanes' usability and were not really a problem. I firmly believe that military procurement is a mess but the discussion reminds me of the political discussion where Republicans jump on anything that goes wrong under Democrats and vice versa.

Jtsummers 11y ago
To be fair, the SR-71 was a highly specialized, and ultimately, highly experimental aircraft at its time. The Concorde was similar. Only a few were ever made, and they were never expected to be the ubiquitous aircraft of their time or even their type (reconnaissance and passenger transport). The F-35 is intended to replace several aircraft models which currently satisfy more specialized roles. It was intended to be the ubiquitous (in various configurations) fighter aircraft for the US military (and some allies). This problem may not be the worst of its issues, it may not even be a major issue or really (from the pilots' perspective) an issue at all since similar reboots (but less frequent) are the norm for them in other aircraft. But many other issues are demonstrating, and the reduced production numbers and increased production and development costs reveal, that there are numerous problems with this project in particular. It shows failings within the US political structure (that's essentially forced this project to continue), DoD procurement (that allowed many of these issues to occur initially), defense contractor procedures (LM, IIRC, doubling or tripling the number of engineers on the software side to "speed things up"). Really, it's going to be a classic for future engineers, computer scientists, MBAs and others to study. So I guess that's a good thing.
- maxxxxx 11y ago
  I totally agree that the F-35 project is a mess. But I would like to see a real analysis of the problems and not just jumping on any problem that sounds bad on the surface. Maybe a hard reboot is a problem, maybe not. I don't know.

NiftyFifty 11y ago

That's like when my power goes out in my house ... I have to reboot my light switches ...

Havoc 11y ago

Judging by reddit comments this type of thing is pretty common with fighter jets which is kinda surprising to me. I would have thought that this is one area where reliability is highly prized. Along with space shuttle software & other life/death stuff etc. Instead "bash it with a hammer and maybe reset it" seems widely accepted. Weird.

deleted 11y ago
[deleted]

matt_wulfeck 11y ago

I bet the Chinese read about the F-35 fiasco and just laugh at us. The amount of money we're throwing down the hole here is staggering.