14 ms·
Integer Overflow Bug in Boeing 787 Dreamliner
- xnull2guest 11y agoThis reporting comes on the heels of an GAO study on hijacking airliners. It is not clear why the Congress ordered a study on hacking airliners, though there's a long list of things (MH flights, Carter's claims of a 'cyber Pearl Harbor') that some people might speculate over. Does anyone know the impetus behind the study?
- VonGuard 11y agoProbably part of a larger security initiative with money to pay for studies. I'd wager there's a Congressional committee tasked with writing policy to secure critical infrastructure like power plants and such. After 9/11, you can bet planes and FAA systems would be a part of this.
- xnull2guest 11y agoThere are definitely security initiatives like this, there have been since before 9/11 (and an uptick afterwards), but it is unusual for the Congress to be involved or to demand a study.
- excel2flow 11y agoCould have abstract interpretation (http://www.astree.ens.fr/ http://www.astree.ens.fr/) or some other formal method prevented it?
- tlb 11y agoSomeone wrote something like: int32_t ticks; // 100ths of a second which overflows in 248 days, a particularly unfortunate amount of time because it doesn't show up during testing. Although it would be a good engineering choice, a formal verifier would say that: int64_t ticks; // 100ths of a second is also incorrect, since it also overflows (after 10^9 years). In a hard real time system, mpz_t ticks; // 100ths of a second, infinite precision libgmp type is still formally incorrect, since as the the number grows in size it will eventually exceed a time limit or memory (after 10^10^9 years) The overall lesson from formal methods is that it's impossibly to write formally correct useful programs. So programmers just muddle through.
- mhogomchungu 11y ago> int64_t ticks; // 100ths of a second I would go with uint64_t as it documents "ticks" as a variable that can not hold negative values and also doubles its range of positive values.
- speakeron 11y agoI would go with uint64_t ticksOfDuration10ms; // No comment necessary
- cnvogel 11y agoThe concept of timer "ticks" is well established as a unit of time in embeded programming, it's almost universally included in your embedded (realtime-)OS and might increase at any conceivable rate, both limited by the hardware constraints (e.g. a fixed, simple, 16-bit ripple counter that is clocked by the main CPU clock of 8 MHz will clock at 122.07 Hz) or at your application requirements (you let a slightly more configurable timer only count to 40000 at half the CPU clock to get exactly 100 Hz). Hence you shouldn't explicitly inscribe the tick rate in your symbol name, as it can change when requirements change. You'll almost always have a global variable, preprocessor define... or something similar to get the frequency (or time increase per tick), which you should use whenever you have to convert "ticks" to actual physical units. If the actual effective tick rate is visible at many places in your code, both as a symbol name or as a comment, you are most certainly doing something wrong.
- speakeron 11y agoI think you kind of missed the point of my post (which was a bit tongue-in-cheek). The original code fragment had the tick duration embedded in a comment, so changing a global variable which defines it something other than 10ms is going to cause all sorts of problems in maintaining that code. (Leading possibly to the very problem Boeing had).
- cnvogel 11y ago
- rasz_pl 11y agohttp://embedded.fm/episodes/2015/4/28/99-you-can-say-a-boat http://embedded.fm/episodes/2015/4/28/99-you-can-say-a-boat but I would suspect Boeing used something better than C.
- excel2flow 11y agohttp://www.pipeline.com/~hbaker1/sigplannotices/gigo-1997-04.html http://www.pipeline.com/~hbaker1/sigplannotices/gigo-1997-04...
- anderspitman 11y agoI honestly can decide whether this is serious, satire, or conspiracy theory, but it's awesome nonetheless. My first project working on the 787 was converting an Ada codebase to C.
- azth 11y agoIsn't Ada precisely suited for this sort of application? What was the motivation for switching to C?
- anderspitman 11y agoC is incredibly portable and tons of programmers know it. Ada is great in a lot of ways but just never got the traction it needed to be #1.
- azth 11y agoSure, but shouldn't safety be the number one concern here? Programmers can always be trained to learn it, as long as they demonstrate competence. It seems like an unfortunate case of trying to cut costs at the cost of safety.
- randomsearch 11y ago> What was the motivation for switching to C? Invariably, cost. SPARK Ada is demonstrably superior to C for safety-critical development (I can't cite the sources for this, but a major company developing safety-critical software has shown this to be the case). But, SPARK Ada requires a lot of highly skilled manpower and it's slow to develop. C gets the job done, albeit with lots of bugs.
- CamperBob2 11y agoAnother good practice is to initialize the time counters to something close to the overflow point, rather than zero. This encourages overflow bugs to show up at timescales where they will be noticed during testing, rather than after 248 days of service. This is a scary-ass bug in a codebase that was supposed to be authored to strict professional standards.
- cpeterso 11y agoThe Linux kernel does that. The 'jiffies' counter's initial value rolls over 5 minutes after boot: http://lxr.free-electrons.com/source/include/linux/jiffies.h#L163 http://lxr.free-electrons.com/source/include/linux/jiffies.h...
- ghshephard 11y agoI find it, well, interesting to read that the "Fail Safe" mode is to deactivate all power systems on the plane.
- noir_lord 11y agoThey probably decided that started from a known state after a major problem was better than starting from an inconsistent state.
- sitkack 11y agoI find it troubling that the generators can't reboot w/o continuing to supply power, that reboots aren't staged to ensure that the plane continues to have power and that power from the generators it necessary to control the aircraft. Isn't there battery backup to enable the plane to continue operating normally while the generator reboots? From the article it looks like their whole failsafe/redundant system architecture is flawed. Just like buffer bloat, reboot times and corruptible system state are a chronic systemic flaw in modern technology. The only stuff that works is the stuff that is used all the time. Look towards crash only software [1] and microreboots [2,3] [1] https://www.usenix.org/legacy/events/hotos03/tech/full_papers/candea/candea.pdf https://www.usenix.org/legacy/events/hotos03/tech/full_paper... [2] https://www.usenix.org/legacy/event/osdi04/tech/full_papers/candea/candea.pdf https://www.usenix.org/legacy/event/osdi04/tech/full_papers/... [3] http://dslab.epfl.ch/pubs/perfeval.pdf http://dslab.epfl.ch/pubs/perfeval.pdf
- VonGuard 11y agoThe actual reboot takes something around a minute, and the plane is designed to keep flying while rebooting. It's during take off or landing that this would be a real problem. Think Starship Enterprise. You can't turn off the whole ship, or everyone suffocates. You run diagnostics system by system and keep the warp core online as much as possible so you don't run out of power.
- xrstf 11y agoCurrent workaround: Restart the estimated 28 U.S. planes at least every 120 days[1]. Wonder how long it could take for the update to be actually available (after testing, approving, ...). Are we talking weeks, months, years? [1] https://s3.amazonaws.com/public-inspection.federalregister.gov/2015-10066.pdf https://s3.amazonaws.com/public-inspection.federalregister.g...
- ferrix 11y agoActually I am a bit surprised that somebody would want a plane to be powered on for such a long time. There is no way one could fly for that long anyway and they are regularly taken to service between long hauls.
- 51Cards 11y agoAgreed... while this is a software design issue I too would be surprised to see a plane powered up for that long continuously. Needs to be fixed, yes... causing a lot of issues at the moment? Probably not.
- userbinator 11y agoI'm guessing these GCUs are like the ECUs in a car - even when "off" they're still powered and running in a standby mode. The fact that the AD contains messages such as "batteries do not need to be disconnected" and that this "electrical power deactivation" takes approximately an hour of work suggests something needs to be gotten at, unplugged, and replugged.
- ubernostrum 11y agoBoeing 787 aircraft have been in service for longer than 248 days (entry into commercial airline service was in 2011), with zero confirmed encounters with the bug in the wild. This suggests that your "always on even when off" theory does not hold up.
- sitkack 11y ago
- zaroth 11y agoThis failure mode in particular was deemed exceedingly unlikely by Boeing, which got them an exception to some initial airworthiness issues with the RAT, which in turn would have made a total loss of power catastrophic.
- firethief 11y agoThey can deem things unlikely? That seems broken in general. I would deem it unlikely they'd ship with any errors they didn't deem unlikely; those are precisely the failure modes we should most look for...
- ams6110 11y agoThe entire aircraft is an electro/mechanical system with many thousands of things that could go wrong, but are deemed unlikely. All engines could fail at the same time, but it's deemed unlikely. Redundant hydraulic systems could fail together, but it's deemed unlikely. There is no certainty in systems this complicated.
- AshleysBrain 11y agoWindows 98 had a similar bug where the system would hang after 49.7 days: https://support.microsoft.com/en-us/kb/216641 https://support.microsoft.com/en-us/kb/216641 Although IIRC, the impact was limited, because it was quite a feat for a Windows 98 system to stay up for 49 days :)
- alfiedotwtf 11y agoI found 98 respectible. It was 95 that didn't last a whole week!
- Rexxar 11y ago"Windows Millennium Edition" was the worst.
- windsurfer 11y ago"Malfunction Edition" as it was sometimes known
- digi_owl 11y agoWin2k was golden.
- wglb 11y agoMy last surviving win2k machine finally died. Power supply would not fire up after a power cycle. I could repair it, but maybe it is just time to leave it be.
- digi_owl 11y agoI had to abandon 2k after i tried swapping the motherboard on it, and ended up with the blue screen of "don't have driver X for critical hardware Y".
- 11y ago
- shellmayr 11y agoWow, how can something like this happen? I thought airplanes had triple redundant software systems using 3-version programming [1] in order to avoid such bugs/problems. Can anyone familiar with flight technology shed some light on this? [1]http://en.wikipedia.org/wiki/N-version_programming http://en.wikipedia.org/wiki/N-version_programming
- CHY872 11y agoNot an airplane programmer, but I seem to remember that the literature says that it's not generally a cost-effective way of finding bugs. In particular, you multiply the cost of development by (say) 3x (which is fine, on its own) but also the places where bugs are inserted are typically the hard parts; so you don't reduce the number of bugs as much as you'd like; it can easily be more cost effective to invest the few million in static analysis etc. As much as we'd like plane manufacturers to test things to death, it'd become too expensive too quickly. For all we know, this software could be written by a contractor, or the firmware for a third party part. As far as I know, N-version programming was effective when software systems were small (shuttle ran on 50k lines of code) and where poring over every single line was possible, because the hard part was coming up with the spec. Nowadays a big plane like the A380 might be expected to have 100M lines of code in its subsystems, and it's simply too expensive.
- hello_there 11y ago> Nowadays a big plane like the A380 might be expected to have 100M lines of code in its subsystems, Why does an airplane require 100M lines of code?
- mschuster91 11y agoLinux kernel alone comes in at 15m SLOC. Now add an userland subsystem and you're at 20-30M just for one device. Multiply by all the little and big subsystems, the embedded chips, in-flight entertainment, network gear... 100m SLOC is too low, I think.
- dschiptsov 11y agoHow come it is not Java?
- falis 11y agoIn its current incarnations, it is considered uncertifiable for high criticality levels under DO-178C. May be used in entertainment systems and such though.
- dschiptsov 11y agoYou mean it is considered to be crap?)
- Mojah 11y agoThey're actually not alone, Dell's EqualLogic (a big & expensive storage array) had the same problem, after 248 days. They would initiate a controller failover and reboot: https://ma.ttias.be/248-days/ https://ma.ttias.be/248-days/
- baruch 11y agoThere was a similar issue in the Linux kernel in version 2.6.32 where the kernel would crash after 208 days: http://www.novell.com/support/kb/doc.php?id=7009834 http://www.novell.com/support/kb/doc.php?id=7009834 This was a serious problem in some storage systems too: https://www.ibm.com/developerworks/community/blogs/anthonyv/entry/208_day_reboot_bug3?lang=en https://www.ibm.com/developerworks/community/blogs/anthonyv/...
- lotsofmangos 11y agoReminds me a little of the floating point precision bug in the patriot missile targeting systems, where the longer it was left on, the less accurate it got. http://fas.org/spp/starwars/gao/im92026.htm http://fas.org/spp/starwars/gao/im92026.htm
- userbinator 11y agoPresumably all of Boeing's other planes have such counters in their systems, and don't have this bug (or if they did, it was corrected already), so why only the 787? That's what I find most surprising. Edit: one theory that seems plausible is that they were "overly paranoid" and put in overflow checks, on a time counter whose overflowing would not have had negative effects otherwise since the other code was designed to handle a wraparound correctly.
- Jtsummers 11y agoSoftware tends not to be reused between planes unless you go back to the same vendor and there are no major hardware changes with the component as well. Aircraft software is kind of a broken world.
- TeMPOraL 11y agoMaybe it's for the best. Ariane 5 comes to mind.
- sitkack 11y agoMost expensive software reuse ever.
- agumonkey 11y agoBroken Aircraft software makes me wanna rethink the notion of correctness, or broaden the scope of failure and function.
- jMyles 11y agoRight? If aircraft software is broken, but my linux desktop is supposed to be the picture of success, I'm not sure the definitions are meaningful. :-)
- agumonkey 11y agoAnd I was serious. We should study this and see why what could be described as a fault, a bug, etc ... is actually not that meaningful.
- dwightgunning 11y agoI found it curious that the journalist refers to the bug as a "vulnerability". This is could be misinterpreted given that's a term is more commonly used in a security context.
- GigabyteCoin 11y ago>journalist There's your answer right there. They write stories for a living, not programs.
- smsm42 11y agoSome journalists still care to acquire correct terminology. If somebody is reporting from the court and call somebody accused of car theft "murderer" because you don't know the difference between the two, they'd probably get laughed at.
- digi_owl 11y agoI guess somehow they got the idea that someone could breach the firewall between the entertainment system and the flight controls, and insert a bogus entry in the right spot to trigger this. Frankly with all the scaremongering going on with computer security these days i kinda agree with Torvalds policy of not highlighting security fixes in Linux patch notes.
- serf 11y agowell, at least it's not a pacemaker. It's a terrible oversight, and makes me wonder about the rest of the code, but are there very many airliners that are online for that long at once? I don't know much about how commercial air travel works behind the scenes.
- __Joker 11y agoI remember a legacy web service system we used to support which will crash after couple of days, after hogging all the resources. The first thing we did was to set up a monitor with a daily restart scripts. That was much cheaper and quicker fix than the fixing the memory leaks which took 3 months to reach to production.
- dvirsky 11y agoI had a memory leak in a Python program (that's a really rare thing), that would trigger OOM kills in about 3-4 days. After a few days of investigation that yielded nothing, I put a restart job every day or so, and returned to it only after a few months when I had some time. Eventually it came down to someone replacing dict.get() with dict.setdefault() in a lookup dictionary in some utility library, causing each miss to leak a small, non GC-ed empty entry in an otherwise small lookup table.
- perlgeek 11y agoWe've also had a web service that grew slower and slower, until the web server was restarted. But I tracked it down to this gem: http://bugs.otrs.org/show_bug.cgi?id=9686 http://bugs.otrs.org/show_bug.cgi?id=9686 (P.S. the link to the pull request in the bug tracker is wrong, because the repo was once deleted and re-created, losing all the old pull requests).
- kazinator 11y agoIs that it really an overflow bug? Or a counter wraparound bug? For example, incorrectly using a X > Y comparison on values that are congruential (and do not overflow) isn't an "overflow bug". You can only locally compare values that are close together on the wheel, using subtraction. The simple thing to do with tick counts is to start them at some high value that is only minutes away from rolling around. Then the situation reproduces soon after startup, rather than days or months later, and its effects are more likely to get caught in testing.
- kazinator 11y agoAnother thing you can do is reduce the range of tick counts. Say you have a 32 bit tick count which increments a hundred times a second, but the longest period (biggest delta between any two live time values) you care about in your module (driver or whatever) is well within 30 seconds. That's only 3000 ticks. Then, whenever you sample the counter, you can mask it down into, say, the 13 bit range [0,8192): effectively a tick counter that rolls over every 81.9 seconds (which you treat correctly as a 13 bit value in your calculations like is_before(t0, t1) or add_time(t0, delta)).
- ambrop7 11y agoThere's no need to reduce the range, you can just treat correctly the full counter range (see my other comment).
- kazinator 11y agoWell, it's a tautology that if you treat correctly anything, you don't need any defensive tricks. Treat all the ones and zeros correctly and everything else takes care of itself.
- ambrop7 11y agoYou mentioned correct treatment first :) I'm just saying that masking the clock is unnecessary and doesn't make correct treatment any easier.
- limaoscarjuliet 11y agoI file it under "funny" but certainly it is nothing unusual or surprising. Software, like anything else, has faults and breaks. Even on an airplane. It mostly is funny to us, developers, because we have all been trying to convince our bosses that "it will never happen". That object_id being an int4 sequence? You would need one object a second for 70 years to overflow. And yet, somehow, it does, e.g. because someone loaded data with object_id set to 1.9B and the sequence followed from there. P.S. My favorite pastime? Watching Aircraft Disasters series in an airport. Not brave enough to watch it during the flight yet. Karma might be a bitch and I do not want to test it 10 km up there ;-)
- istvan__ 11y agoLike everything else? What was the last time the Golden Gate bridge collapsed? :) Everything else does not include the most of the engineering output. In software faults are more common because of the tools we are using and because there is no life in danger if Twitter is down. On the other hand, we cannot allow a bridge to collapse or an airplane to fall down from the sky because it has a fault. There are several techniques to build reliable systems out of non-reliable parts.
- foota 11y agoI don't think this is a one to one comparison, bridges etc get maintenance work done all the time to fix small issues, and they do fail.
- istvan__ 11y agoSee above, I am talking about the frequency of catastrophic failures. I agree it is not a good comparison.
- ams6110 11y agoBridge collapses are actually not that uncommon. While often times overloading or damage is the cause, sometimes it's due to design flaws. https://en.wikipedia.org/wiki/List_of_bridge_failures https://en.wikipedia.org/wiki/List_of_bridge_failures
- mrmondo 11y agoFYI - Engadget has very intrusive advertising that you can't close on a mobile device: http://i.imgur.com/nqgc2p7.png http://i.imgur.com/nqgc2p7.png
- digi_owl 11y ago"tech for ladies", aka a power bank with a led flash and a "designer" case...
- stox 11y agoComing soon: OTA updates for Boeing aircraft. What could possibly go wrong?
- anderspitman 11y agoMight be time to remove working on the 787 from my resume. I feel like the poor thing has been one disaster after another in the news. I can't speak to the quality of the A and B level (most critical) code, but the development process for the C level software I was working on definitely could have used a lot of improvement. Messy code, tests and documentation were an afterthought/checkbox item, etc. The incentives were just wrong. I think there's a ton of room for process innovation in avionics software development. One thing I wanted to build for a long time was a tool for tracing. In theory, every industry requirement (DO-178B, etc) was supposed to trace to a hight level Software Design Document (SDD) requirement, which was supposed to trace to a Software Requirements Document (SRD) requirement, which would trace to a code function. We maintained all of that BY HAND. It was a huge mess. Perfect example of something that could have been an extremely valuable development tool, but ended up just being a hassle to try and maintain. Then of course there's language choice. C is king, which isn't necessarily a bad thing, but it's certainly not the safest, even in the restricted forms used in avionics. Sadly, my very first ever project as a programmer was porting an Ada codebase to C for the 787 (off by one errors for days...). It's almost cliche to say nowadays, but I would be really excited to see Rust gain some traction in avionics over the next 20 years or so. Because that's how far behind avionics is. We were using Visual Studio 6 in 2011!
- icegreentea 11y agoI work in the medical device field, and we have a similar process requirement (traceability from Design Input Requirements -> Software Specification -> Software Verification Procedure (and implicitly, the actual code function) -> Software Verification Report). We're currently wrangling with a giant-ass spreadsheet to keep track, and it totally sucks.
- anderspitman 11y agoAh yes I completely forgot SRD -> TESTS -> Then Code. Maybe that's because we always did it the other way around... I'm telling you, there's money to be made building tools for this stuff. I think a big part of the reason things aren't being improved is that the people in a position to recognize bad process and tooling maybe aren't the type of people to see an opportunity to make money solving the problem rather than putting up with it. I wouldn't associate most of the engineers I knew at Honeywell as the type to stay up until 2AM every night for 3 months working on a side project to pitch to their boss. I think it's really exciting what's happening in healthcare right now though. The innovative culture is exploding. Ultimately I care much more about what happens in medicine than avionics, as long as planes aren't falling out of the sky every 248 days...
- stcredzero 11y agoIIRC, the VisualWorks VM had such a bug that would mysteriously crash an automated airport people-mover after some interval, like 45 or 90 days. (Software crash, not train-hardware crash! Train would simply stop.) Also, as I recall, the train software project did not use automated tests at all! (By that time, VisualWorks VM was implementing them.) (Learn from history. Don't cling so hard to the notion that your language will make you into super-programmers. Certainly, some tools are better in certain contexts than others. However, group culture and the quality of working relationships often have an effect even greater than choice of language. Besides, people often dislike someone who projects an air of superiority.)
- varjag 11y agoThe Lord could not count grains of sand with a 32-bit word
- pslam 11y agoI recall a long time back, when Linux was configured as standard for 100Hz ticks (aka "jiffies"), the counter was initialized close to wraparound instead of 0. The result was you typically encountered "jiffy wraparound" after a few minutes of uptime. You learned whether your system was stable in this situation fairly quickly, rather than 248 (or 497) days later. Kernel developers typically don't have uptimes measured in days. Starting the counter close to wraparound increased the likelihood it was going to get code coverage.
- ekimekim 11y agoI really love this methodology. If an exceptional case exists, and it's cheap to cause the exceptional case to occur during standard usage, then do it so that the code is well-exercised.
- tzs 11y agoThis and other stories are claiming it is an integer overflow, but I've seen no source for that. It seems to be just speculation based on the observation that a 100 Hz 32-bit counter would behave similarly.