4 ms·
“the bug was missed because it only presented when many aircraft at the same time were using the system.” The system reports data on number of passengers, weig
by fwlr 4y ago
“the bug was missed because it only presented when many aircraft at the same time were using the system.”
The system reports data on number of passengers, weight of cargo, plane balance, etc., to the pilots. The calculation is done by the plane’s flight computer. How can it be off by 20,000 pounds, but only under heavy server load?
The explanation that comes to mind is that DynamicSource has a subservice for each source of weight and one of those subservices crashed under heavy usage. So the top-level aggregate-and-report service got an error from one subservice and said “well, guess it’s zero, lol”?
- jcutrell 4y agoMakes me think this could have been a race condition where these aircraft somehow received the load calculations from other aircraft being calculated at the same time window. Some kind of poorly managed event stream / consumer that assumed no parallel jobs would be run? Or maybe the jobs are deduped by departure / arrival instead of flight number? Seems possible
- AceyMan 4y ago>The calculation is done by the plane’s flight computer. Not exactly true. The dispatch release has a flight & performance plan based on the load manifest. It's calculated out of band (here, by the software vendor) and punched into the airplane configuration for departure. If the plan is wrong, the plane flies wrong. GIGO at its finest.
- newZWhoDis 4y agoThis is why explicit error handling is so critical. Being forced to handle the timeout error would make such a bug obvious.