5 ms·
One of my favorite software horror stories is the one of $32 _billion_ overdraft by the Bank of New York. From "Computer-Related Risks" by Peter G. Neumann, pu
by Malic 8y ago
One of my favorite software horror stories is the one of $32 _billion_ overdraft by the Bank of New York.
From "Computer-Related Risks" by Peter G. Neumann, published 1994 (REALLY recommended reading)
"One of the most dramatic examples was the $32 billion overdraft experienced by the Bank of New York (BoNY) as the result of the overflow of a 16-bit counter that went unchecked. (Most of the other counters were 32-bits wide.) BoNY was unable to process the incoming credits from security transfers, while the New York Federal Reserve automatically debited BoNY's cash account. BoNY had to borrow $24 billion to cover itself for 1 day (until the software was fixed), the interest on which was about $5 million. Many customers were also affected by the delayed transaction completions."
Additional reference: https://www.washingtonpost.com/archive/business/1985/12/13/computer-snarled-ny-bank/a707acbe-35bc-4a2e-bd80-180d131618c7/?noredirect=on&utm_term=.96220129dffd https://www.washingtonpost.com/archive/business/1985/12/13/c...
Granted, no one died because of this but ... wow ... that was bad day for some developers somewhere.
- jobigoud 8y agoImaging being called in. Ok guys we don't know what the problem is but it's costing the company $3500 per minute the bug stays unfixed just in interests. No pressure.
- joering2 8y agoAlso the follow up question should be ask: was an engineer who stopped the leak and fix the bug awarded to some reasonable point??
- dasil003 8y agoNo because it was also caused by an engineer, so this could lead to terrifying incentives.
- sidlls 8y agoHow is this different from rewarding a salesman who rescues a sale that a different salesman had botched?
- dasil003 8y agoBecause you can't purposefully botch a sale in order to later recover it. Also because you can't avoid botching sales by being more conservative or adding more process. In short, sales and engineering have basically nothing in common.
- sidlls 8y agoI don't think GP was talking about a scenario where the same engineer who created the bug fixed it and gets rewarded, rather one where a different engineer fixes it. Of course it wouldn't make sense as you describe it.
- dasil003 8y agoHow do you decide objectively who is responsible for every single bug? The whole thing is ripe for abuse from all sides. You need a blameless culture to have good engineering, not a bounty-based one.
- beokop 8y agoGetting paid a premium to go in and fix other people’s mess is just a regular consulting gig.
- jibal 8y agoWhy should they be? The difficulty and quality of the work isn't dependent on the severity of the bug. Basically you would be creating a lottery.
- deleted 8y ago[deleted]
- duxup 8y agoI was on a conference call with a bank that due to a number miscommunications and general idiocy was under the impression that they were possibly in violation of some banking laws (no comment on what laws) because they misinterpreted what exactly was happening. Their lawyers supposedly told them they were at risk for an unbelievable sum of fines and criminal stuff. In short they saw some data they didn't understand that seemed to indicate things at the bank were very much not how they were thought to be, and indicated something very specific was happening. In reality that was not the case but a lot of assumptions by morons somehow were believed and everything snowballed into lots of corroborating "evidence" that seemed to indicate bad things. This was a bank that rarely had this level of stupid information and assumptions rise to the top so nobody actually questioned it no matter how absurd it seemed. My presence on the call was purely because the bank wanted every vendor they had looking to see if they saw problems, so I was basically just poking around telling them what i saw from my end. At one point on the call there was a dude in an unmanned data center literally just flipping off power switches and cutting the cables of various equipment as he was given the locations for it. It was frantic and very unamerican bank like (at least the ones I worked with were pretty cool customers normally). I turned my speakerphone on to let my coworkers listen to the chaos. In the end, it was a stupid SQL server related virus thing that created some good old fashioned network disruption and when that was solved and nothing looked like the sky was falling, everyone came to their senses. Once the dude pulled the power to enough of them everything calmed down and I checked my bank account and it was all there... but no excess either :(
- mehrdadn 8y agoWow. Any idea what would've happened if they hadn't gotten it fixed overnight?