3 ms·
I've mostly given up on threat modelling as a solution, even after building a tool from scratch and using it in organizations, and now only do it privately on m
by motohagiography 6y ago
I've mostly given up on threat modelling as a solution, even after building a tool from scratch and using it in organizations, and now only do it privately on my own stuff, but the reason might be a fresh perspective.
The basic problem is that the demand for security stems almost exclusively from compliance because compliance transfers risk out of the project and onto the standard/model. The idea of thinking through risks and hypotheticals and leaving a paper trail that you have acknowledged them basically converts theoretical risk into real liability within the project. In an ideal world of individual ownership and goodness, this means the incentives are aligned to create a good product, but in the real world of organizations, you've reduced the flexibility of the business to manage that risk, which means to take the risk and respond to it on the fly as it looks like it becomes realized. By problematizing the risk instead of managing it, you have destroyed potential value. Threat modelling is the exercise of problematizing risk.
This is why the threat model is often the elephant in the room. Almost nobody in business wants to have the "negative" conversation about whether to block law enforcement (foreign or domestic), surveillance on users, protect against data snooping by technical staff (which is often a platform perk and feature), or institutional privacy violations and other obvious threats that a threat modelling exercise addresses. A business PM won't problematize those things because it will demand a solution that gets in their critical path. The best they can do is manage it, which means orienting themselves to risk and being ready to respond.
When people buy security products, they almost exclusively buy ones that provide data to manage risk using surveillance and monitoring, which means generating data that drives conversations and enables the organization to adapt in time. If a security product does not either a) externalize a risk with compliance or b) provide data to flexibly manage risk, it's not a security product in the market today because nobody will have bought it.
I realize this is a 10th man view of something these very smart people have spent a lot of time on, but in the 25 years I have been in the field, I have come to the conclusion that security people are essentially environmentalists and activists trying to make a case for internalizing an exogenous problem and cost, and the only way to get traction for it is to produce either low-level discretionary developer tools, or generate data that supports the conversations and provides that flexibility in a business' attitude to risk. Threat modelling is super powerful, but as a forcing function it can destroy value, and this is why it has encountered so much resistance. All manifestos are quixotic, and that is why I think this one may also be.
I have found that the real value of a threat model is defining the business model for a security product, where the threat model is the business case for a product, but that product needs the above features, and to not be a solution to succeed. The security product provides either compliance or data for the threats you derive from your modelling exercise, and it must either transfer risk to a model or provide data to manage. If it leaves responsibility in the project, like threat modelling, it's going to fail.
- contingencies 6y agoA threat model is just a cognitive tool whose primary value is in creating an alternative perspective on a design problem to broaden thinking and improve outcomes. Like any tool, there are people who carry the hammer too far and suddenly everything is a nail. As soon as they start harping on about a "journey of understanding" and "multiple deliveries" your bubbling instinct of disgust is correct and you should run and hide. I don't trust security people to do sane things. - Linus Torvalds (2017)
- __jf__ 6y agoMost of the compliance paperwork I’ve seen does leave room for custom risk assessments, threat modeling or other wordings that invite a business team to do more. However in their rush to go live or otherwise get it over with this security work is done after all other things. It isn’t integrated, in an SDLC for example. So minimum standards become maximum standards. It’s hard work convincing teams to do better, but at least the compliance docs give permission to develop your own, often better, understanding if the data classification is high enough. It doesn’t happen often but I havent abandoned all hope yet.
- clairity 6y agowhile i appreciate the insights you bring to bear on the actual behavior of organizations and teams, your response seems to be resigned to accepting the status quo rather than working to change the incentives so that security risks and damages are internalized appropriately. this kind of excuse-making is how we slide down various slippery slopes (e.g., how google's "don't be evil" eventually failed), even with eyes wide open. instead, the better response is to advocate legislative and organizational improvements while also working within your perceived constraints. it can be both, not either-or.
- motohagiography 6y agoThis comment expresses a misunderstanding. Merely not believing hard enough is not why threat modelling has failed. There is an incentive mechanism and this description of it can help them re-orient their strategy. The counterfactual premise of threat modelling is that a business wants responsibility for mitigating or remediating risk without direct compensation, instead of a method to manage and transfer it. A technologist is just happy to solve problems, so they don't see this open loop as a source of value.