3 ms·

Could be something as simple as the first concurrency 101 example: atomic addition. The += operator behaves poorly on concurrent access.

• Jtsummers 4y ago
  We had that error in an application at work. Essentially data was coming in on multiple threads, but there was a shared value between them that was being incremented and decremented. Without a lock and without using atomics, the tally could be incorrect. But under low load this didn't appear because the odds of interrupting +=/-=/++/-- was low. Under high load, however, the odds of a thread being interrupted mid-operation increased and so it appeared in production. At least in our case it was just an annoyance (no critical decisions were based on the value, it was merely informative). If there are multiple data streams feeding in weight data per aircraft, under a low aircraft load (the scenarios they apparently tested) the odds of interruption during an operation like += can be low enough to not see the issue. Under high load, though, the odds of interruption and incorrect tallying increases substantially.
  • KennyBlanken 4y ago
    In what scenario would you anticipate data for the calculation of rotation speed and engine thrust for one particular plane to be based on a value that is not only shared and writable, but changing each time a plane's performance data is calculated? "multiple data streams feeding in weight data per aircraft"? What does that word salad even mean? Even if what any of what you said was true: these developers are writing critical safety software. If they can't manage to write code without generating race conditions and testing their system for such conditions, they are grossly unqualified to be writing this sort of software. As others have said: this is a huge fuck-up.
    • Jtsummers 4y ago
      It is a huge fuckup, I didn't say otherwise. And that's not word salad, though maybe not as clear as I could have made it. But to clarify for this hypothetical (if this is the issue): Each luggage weigh station submits data to the application or database which then has to tally the weight. Those are multiple data streams, not a hard concept. An airport has many luggage weigh stations all potentially submitting data at the same time. If those are being processed concurrently and the database or however the data is stored is not properly locked then the tally can be incorrect for the same reason non-atomic increments/decrements can become incorrect in a multithreaded application. But just like the problem with non-atomic (unlocked) increments/decrements the problem may not manifest without high enough load on the system. Low-load testing (what the article seems to describe) means that updates can happen fast enough (server or application is under loaded, less likely to interrupt any update) or submitted slow enough that the issue never appears. So you need to test with a higher load to detect the problem (if you're going to be able detect it in testing at all).
    • richwater 4y ago
      I fucking cracked up at: "multiple data streams feeding in weight data per aircraft" it's literally some integers (floats at the worst?).
      • Dylan16807 4y ago
        You can send integers from multiple sources, dude.