6 ms·

There is a really, really interesting breakdown of this document on kitchensoap.com [0] The writer breaks down why this document is not a post-mortem despite s

There is a really, really interesting breakdown of this document on kitchensoap.com [0] The writer breaks down why this document is not a post-mortem despite superficial similarities. >Again, the purpose of the doc is to point out where Knight violated rules. It is not: 1) a description of the multiple trade-offs that engineering at Knight made or considered when designing fault-tolerance in their systems, or 2) how Knight as an organization evolved over time to focus on evolving some procedures and not others, or 3) how engineers anticipated in preparation for deploying support for the new RLP effort on Aug 1, 2012. >To equate any of those things with violation of a rule is a cognitive leap that we should stay very far away from. >It’s worth mentioning here that the document only focuses on failures, and makes no mention of successes. How Knight succeeded during diagnosis and response is unknown to us, so a rich source of data isn’t available. Because of this, we cannot pretend the document to give explanation. He also makes an interesting point related to what you're saying: the SEC says the risk management controls were inappropriate, but clearly Knight thought they were appropriate or they would have fixed it. >What is deemed “appropriate”, it would seem, is dependent on the outcome. Had an accident? It was not appropriate control. Didn’t have an accident? It must be appropriate control. This would mean that Knight Capital did have appropriate controls the day before the accident. Outcome bias reigns supreme here. I'd go into it more but I would instead recommend you take a look at his breakdown as I'd be trying to do a shoddy summary of a really interesting write-up. [0] https://www.kitchensoap.com/2013/10/29/counterfactuals-knight-capital/ https://www.kitchensoap.com/2013/10/29/counterfactuals-knigh...

guitarbill 8y ago
This kitchensoap article may be more correct, but man, it's a arduous read. A bit of brevity and humour can work wonders, conversely complete correctness can be detrimental if people lose attention.
- Defenestresque 8y ago
  It is a lengthy slog for sure, but I found it a great way to kill some time. Maybe you just have to be in one of those reading moods.
  - Bartweiss 8y ago
    I'm not hugely fond of this specific article; I think the dissection of the SEC filing misunderstands what the accusation is. (Specifically, it's not having the bug, it's the lack of mitigation for large bugs in general.) To me, the "this is not a post-mortem, here's why" part would have stood better on its own. But I appreciate the writing style, and I've got his site bookmarked for more systems safety reading in the future. It's a slog, but often this sort of ruthlessly-comprehensive breakdown is the best way to understand exactly how a complex thing went awry. Reading them every so often - even for non-software topics like drug treatments - seems to be a good refresher for my own error-analysis skills.
Bartweiss 8y ago
> He also makes an interesting point related to what you're saying: the SEC says the risk management controls were inappropriate, but clearly Knight thought they were appropriate or they would have fixed it. > This would mean that Knight Capital did have appropriate controls the day before the accident. I think these claims are seriously confused. SEC fines don't require mens rea, so Knight is simply being punished for having inappropriate controls, their view on the matter be damned. Kitchensoap rightly observes that "this event was very harmful" does not imply "this event was caused by extreme negligence". But the SEC filing focuses on alerting and controls; position limits don't prevent a specific misstep, but they limit the maximum size of any error that does occur. (Knight had position limits on accounts, but didn't use them as fundamental boundaries restricting actual trade volume.) The thesis is that Knight should have prepared to mitigate "unknown unknowns", in which case the size of the error is relevant because the size was exactly what should have been controlled for. On appropriate controls, SEC fines are certainly outcome-biased, but the claim is obviously that these controls were always inappropriate, and the disaster simply revealed them. Post-disaster punishment creates an ugly system where people who don't take excess risk can be outcompeted before their competitors crumble, but the rule isn't actually conditional on failures. Kitchensoap asks whether Knight would be judged so harshly if they'd only lost $1,000. Socially, perhaps not, but legally they actually would have! The SEC isn't just punishing Knight for losing money but for disrupting the market with improper trades; it specifically notes that for some "...of those stocks, the price moved by greater than ten percent, and Knight’s executions constituted more than 50 percent of the trading volume. These share price movements affected other market participants...". A smaller loss wouldn't have defended against that charge, while a smaller trade wouldn't have violated SEC rules. I think the author is basically aware of this, since his fundamental point is that the SEC is describing the legal wrongs rather than the technical mistakes. That's a good point and I'm glad you linked this. But I think his focus on the specific deployment error neglects the fact missing position controls were the larger legal and technical failure.
- mannykannot 8y ago
  > Kitchensoap rightly observes that "this event was very harmful" does not imply "this event was caused by extreme negligence". What makes negligence extreme? Doing things you have specifically been warned against would be one thing, but there are others, including being oblivious to the magnitude of the risk when, with a little thought, it should have been clear. The inverse of the above quote is equally valid: not-very-harmful outcomes do not imply that the negligence is not extreme, and it was all the days of operating without big problems that allowed the organization to be blind to the risk it was running, day in, day out. OP wrote: >> Clearly Knight thought [its controls] appropriate or they would have fixed it. But the problem is that it did not think about it, in a meaningful way: it did not have an informed opinion. Every day in which nothing very bad happened contributed to the normalization of deviance. I am sure there were other days when things went wrong, but without the worst possible outcome, and they became just part of the way things are, instead of a wakeup call.
yellowapple 8y ago
> SEC says the risk management controls were inappropriate, but clearly Knight thought they were appropriate or they would have fixed it. By this logic, we can claim that if a hospital is storing all its employee passwords in plaintext, that's "appropriate" because if it was inappropriate they wouldn't do so. Or that if a company is neglectful about offsite backups, that's an "appropriate" data retention strategy because if it wasn't, then the company would be taking backups. In this case, if Knight thought its controls were "appropriate", then that's the problem that needs fixed.

guitarbill 8y ago

This kitchensoap article may be more correct, but man, it's a arduous read. A bit of brevity and humour can work wonders, conversely complete correctness can be detrimental if people lose attention.

Defenestresque 8y ago
It is a lengthy slog for sure, but I found it a great way to kill some time. Maybe you just have to be in one of those reading moods.
- Bartweiss 8y ago
  I'm not hugely fond of this specific article; I think the dissection of the SEC filing misunderstands what the accusation is. (Specifically, it's not having the bug, it's the lack of mitigation for large bugs in general.) To me, the "this is not a post-mortem, here's why" part would have stood better on its own. But I appreciate the writing style, and I've got his site bookmarked for more systems safety reading in the future. It's a slog, but often this sort of ruthlessly-comprehensive breakdown is the best way to understand exactly how a complex thing went awry. Reading them every so often - even for non-software topics like drug treatments - seems to be a good refresher for my own error-analysis skills.

Bartweiss 8y ago

> He also makes an interesting point related to what you're saying: the SEC says the risk management controls were inappropriate, but clearly Knight thought they were appropriate or they would have fixed it. > This would mean that Knight Capital did have appropriate controls the day before the accident. I think these claims are seriously confused. SEC fines don't require mens rea, so Knight is simply being punished for having inappropriate controls, their view on the matter be damned. Kitchensoap rightly observes that "this event was very harmful" does not imply "this event was caused by extreme negligence". But the SEC filing focuses on alerting and controls; position limits don't prevent a specific misstep, but they limit the maximum size of any error that does occur. (Knight had position limits on accounts, but didn't use them as fundamental boundaries restricting actual trade volume.) The thesis is that Knight should have prepared to mitigate "unknown unknowns", in which case the size of the error is relevant because the size was exactly what should have been controlled for. On appropriate controls, SEC fines are certainly outcome-biased, but the claim is obviously that these controls were always inappropriate, and the disaster simply revealed them. Post-disaster punishment creates an ugly system where people who don't take excess risk can be outcompeted before their competitors crumble, but the rule isn't actually conditional on failures. Kitchensoap asks whether Knight would be judged so harshly if they'd only lost $1,000. Socially, perhaps not, but legally they actually would have! The SEC isn't just punishing Knight for losing money but for disrupting the market with improper trades; it specifically notes that for some "...of those stocks, the price moved by greater than ten percent, and Knight’s executions constituted more than 50 percent of the trading volume. These share price movements affected other market participants...". A smaller loss wouldn't have defended against that charge, while a smaller trade wouldn't have violated SEC rules. I think the author is basically aware of this, since his fundamental point is that the SEC is describing the legal wrongs rather than the technical mistakes. That's a good point and I'm glad you linked this. But I think his focus on the specific deployment error neglects the fact missing position controls were the larger legal and technical failure.

mannykannot 8y ago
> Kitchensoap rightly observes that "this event was very harmful" does not imply "this event was caused by extreme negligence". What makes negligence extreme? Doing things you have specifically been warned against would be one thing, but there are others, including being oblivious to the magnitude of the risk when, with a little thought, it should have been clear. The inverse of the above quote is equally valid: not-very-harmful outcomes do not imply that the negligence is not extreme, and it was all the days of operating without big problems that allowed the organization to be blind to the risk it was running, day in, day out. OP wrote: >> Clearly Knight thought [its controls] appropriate or they would have fixed it. But the problem is that it did not think about it, in a meaningful way: it did not have an informed opinion. Every day in which nothing very bad happened contributed to the normalization of deviance. I am sure there were other days when things went wrong, but without the worst possible outcome, and they became just part of the way things are, instead of a wakeup call.

yellowapple 8y ago

> SEC says the risk management controls were inappropriate, but clearly Knight thought they were appropriate or they would have fixed it. By this logic, we can claim that if a hospital is storing all its employee passwords in plaintext, that's "appropriate" because if it was inappropriate they wouldn't do so. Or that if a company is neglectful about offsite backups, that's an "appropriate" data retention strategy because if it wasn't, then the company would be taking backups. In this case, if Knight thought its controls were "appropriate", then that's the problem that needs fixed.