4 ms·

I fear this sort of thing is going to become more and more common at airports.

isostatic 9y ago
Why? Sure, the skies are more and more crowded, meaning more and more people will be affected by a once-in-20 year failure, but why would it happen more and more common?
- SteveNuts 9y ago
  Well the archaic systems are getting older and older. From what I've heard it's almost impossible to get replacement parts for some of the computer systems that run airports/airlines.
  - tetha 9y ago
    And on the other hand, there are massive requirements at all levels for this kind of software. I'm not sure what the right answer is at that point. From dev-experience, I wouldn't want half-assed software directing flights. From ops-experience, I don't want software directing flights requiring specific hardware so you can just replace the hardware with easily available hardware.
    - drinchev 9y ago
      From dev-experience I would be surprised if SpaceX rockets run with decades old assembler code-base. I still believe we can create bug-free solution with modern toolset. It's a matter of zeros behind the project funding.
      - tetha 9y ago
        That's a good point. I'm just a bit disillusioned with the quality of some of the legacy stuff at my current place. Especially because robustness and probability of failures is mostly determined by requirements.
  - icebraining 9y ago
    Not sure about aviation, but in other industries they virtualize the old hardware on a new architecture, and keep the software going.
    - deleted 9y ago
      [deleted]
  - agumonkey 9y ago
    perfect replacement part maybe not, but you may be able to emulate some chips with cheap cortex-m/a or esp class
  - the_mitsuhiko 9y ago
    Eurocontrol does not run archaic systems.
  - tedunangst 9y ago
    Was this failure caused by a hardware fault?
  - tyingq 9y ago
    Has not been my experience. 2 decades in the airline space. It's mostly uninformed people assuming that TPF is the issue or that it runs on old hardware. Neither is true. In this case, for example, it wasn't the airline. It was Eurocontrol (https://www.eurocontrol.int https://www.eurocontrol.int) Airline tech outages are just more visible than outages in other industries. Like manufacturing, etc. It doesn't make the news when all your manufacturing plants halt for 6 hours due to tech issues. Airlines could improve, sure, but pundit observations are usually off base. They are also dependent on systems they don't control, like ATC, TSA, Airport owned kiosks, and so on.
  - dx034 9y ago
    That's more a problem with core banking software (and the hardware it runs on) than with airline software.
- newnewpdro 9y ago
  If the reliability stays constant expressed as a rate (arguably optimistic), like ".1% failure per X flights", and the # of flights increases, then failures become more frequent. Failure rates must decrease proportionally to the utilization/capacity increase for the # of failures to stay constant. Otherwise, the # of failures will naturally increase, it's simple multiplication. For example: If crime rates stay constant, but your population increases, you have more crime - not the same amount of crime as before. you have more terrorists, you have more school shootings, because your population doubled in size and you didn't cut the rates of abhorrent behaviors in half. Air travel has _exploded_ in popularity, this is why you see all sorts of things more frequently than you did before. From reports of airline passengers being dragged off planes in totally asinine scenarios to deaths of pets in the cabin.
  - isostatic 9y ago
    You're assuming the failure rate is related to the number of flights. With 2 failures in 20 years there's not really enough data to extrapolate. And remember this wasn't a complete system outage - it affected 50% of flights, a degraded service. In comparison the vast majority of startups that HN people work in will be lucky to last 2 years. In the last 10 years Gmail has been down, well not sure. It was down 6 times in one year. Uptime is apparently 99.8%, or down for 175 hours a decade, so 6 hours seems quite reliable. Level 3 went down last month. And in 2016. And in 2015. Having a system having to work for decades without downtime is unheard of in the web world, even in simple systems like gmail and AWS.

Why? Sure, the skies are more and more crowded, meaning more and more people will be affected by a once-in-20 year failure, but why would it happen more and more common?

SteveNuts 9y ago
Well the archaic systems are getting older and older. From what I've heard it's almost impossible to get replacement parts for some of the computer systems that run airports/airlines.
- tetha 9y ago
  And on the other hand, there are massive requirements at all levels for this kind of software. I'm not sure what the right answer is at that point. From dev-experience, I wouldn't want half-assed software directing flights. From ops-experience, I don't want software directing flights requiring specific hardware so you can just replace the hardware with easily available hardware.
  - drinchev 9y ago
    From dev-experience I would be surprised if SpaceX rockets run with decades old assembler code-base. I still believe we can create bug-free solution with modern toolset. It's a matter of zeros behind the project funding.
    - tetha 9y ago
      That's a good point. I'm just a bit disillusioned with the quality of some of the legacy stuff at my current place. Especially because robustness and probability of failures is mostly determined by requirements.
- icebraining 9y ago
  Not sure about aviation, but in other industries they virtualize the old hardware on a new architecture, and keep the software going.
  - deleted 9y ago
    [deleted]
- agumonkey 9y ago
  perfect replacement part maybe not, but you may be able to emulate some chips with cheap cortex-m/a or esp class
- the_mitsuhiko 9y ago
  Eurocontrol does not run archaic systems.
- tedunangst 9y ago
  Was this failure caused by a hardware fault?
- tyingq 9y ago
  Has not been my experience. 2 decades in the airline space. It's mostly uninformed people assuming that TPF is the issue or that it runs on old hardware. Neither is true. In this case, for example, it wasn't the airline. It was Eurocontrol (https://www.eurocontrol.int https://www.eurocontrol.int) Airline tech outages are just more visible than outages in other industries. Like manufacturing, etc. It doesn't make the news when all your manufacturing plants halt for 6 hours due to tech issues. Airlines could improve, sure, but pundit observations are usually off base. They are also dependent on systems they don't control, like ATC, TSA, Airport owned kiosks, and so on.
- dx034 9y ago
  That's more a problem with core banking software (and the hardware it runs on) than with airline software.
newnewpdro 9y ago
If the reliability stays constant expressed as a rate (arguably optimistic), like ".1% failure per X flights", and the # of flights increases, then failures become more frequent. Failure rates must decrease proportionally to the utilization/capacity increase for the # of failures to stay constant. Otherwise, the # of failures will naturally increase, it's simple multiplication. For example: If crime rates stay constant, but your population increases, you have more crime - not the same amount of crime as before. you have more terrorists, you have more school shootings, because your population doubled in size and you didn't cut the rates of abhorrent behaviors in half. Air travel has _exploded_ in popularity, this is why you see all sorts of things more frequently than you did before. From reports of airline passengers being dragged off planes in totally asinine scenarios to deaths of pets in the cabin.
- isostatic 9y ago
  You're assuming the failure rate is related to the number of flights. With 2 failures in 20 years there's not really enough data to extrapolate. And remember this wasn't a complete system outage - it affected 50% of flights, a degraded service. In comparison the vast majority of startups that HN people work in will be lucky to last 2 years. In the last 10 years Gmail has been down, well not sure. It was down 6 times in one year. Uptime is apparently 99.8%, or down for 175 hours a decade, so 6 hours seems quite reliable. Level 3 went down last month. And in 2016. And in 2015. Having a system having to work for decades without downtime is unheard of in the web world, even in simple systems like gmail and AWS.