3 ms·

I've been happy to see ATT&CK, yet in practice avoided as a lot of compliance effort for limited value when that same effort can go elsewhere. The good: -- Au

I've been happy to see ATT&CK, yet in practice avoided as a lot of compliance effort for limited value when that same effort can go elsewhere. The good: -- Auditing & tuning defense in depth: From a testing/verification perspective, defense in depth should follow both your organizational structure (endpoint, network, cloud, virtualization layers, etc.) and, to economically optimize, double down around attack structure. ATT&CK compliance gives a goahead for auditing your system + vendors for holes in the latter. -- N+1 standard: Most Elastic/Splunk/etc. SIEM implementations I've seen are half-implemented dirty data messes. The world failed to agree on past efforts like CIM, CEF, CVE, STIX, etc. ATT&CK compliance is auditing your data lake for better SIEM standardization, and an N+1 effort to get folks to agree. If ATT&CK gets you signoff for cleaning your lake, great! -- Training: A surprisingly large portion of defenders are junior or overly specialized (endpoint vs network vs cloud vs malware vs...) The bad: - Expensive. It doesn't solve much more than the above. I'm much more inclined to spend the $ and effort on better data processing & automation & intelligence capabilities instead. Likewise, for auditing, spend time/money on manual + automated red team, or ATT&CK? For big banks & gov, it's more ok b/c the answer is "giant budget; do both." - Dead end / stepping stone . For the ontological aspect, manual killchain taxonomization is good, but for mapping to real-world data, my sense was, and has only increased, that a lot of that effort needs to be shifted towards (a) centralized logging efforts like Azure Sentinel / Google Chronicle's cloud SIEM's do it for you rather than admin running Splunk/ELK (b) combining w/ ML techniques (BERT?) instead of both the parsing and categorization being 100% manual. I may be especially attuned to the problem b/c teams come to Graphistry when they realize their SIEM is a mess and can't quickly see stuff like killchains, so are striving for next-level thinking. So we get to see a lot of the many ways manual / rule-based n+1 standards here go wrong. At the same time, I'm optimistic b/c of the automation+ML world we're increasingly in. (Semantic web vs. Google search.) More on the positive side, we've been working on some anti-misinformation work in covid ProjectDomino.org, and an ATT&CK-like mapping of misinformation has been helpful. Defense in that world is a lot earlier in the understanding needed for automatic detection & response, so manual taxonimization is a good early step.

badrabbit 6y ago
Wish I had time to comment on all your points. Central logging efforts with chronicle/sentinel always result in a ton of data loss. On prem is cheap and works well. Elastic/Graylog solve this well. Whether it is sentinel,chronicle or so many of the others in this market, they simply don't and never will know your environment well enough. This will always stab you in the back with the double edged sword of a) losing contextual detail that would have let you make better decisions b) Losing control over data transformation and detection/alerting logic,which means they do too good of a job in suppressing false positives that you miss bad stuff,they generate too many false alerts that waste precioud man hours,many of which could be tuned out(the bureaucracy of a 3rd party,even with admin access to their rules will always be crippling). In short, my opinion is that you need a fast failing and agile seceng(devsecops?) cycle where you have all the data (logs and enrichment) at your disposal. Your people that spend a lot of time juggling false positives could instead contribute to this, which will result in good alert fidelity and more free time for threat research and hunting! (The fun stuff!!)
- lmeyerov 6y ago
  We're talking about slightly different things: you're thinking cloud vs. on-prem, while I mean no longer giving SIEM vendors a free pass for crappy parsing & classification of most data and pushing that to users. Running the SIEM via a managed cloud happens to make this easier, but not actually 100% required (e.g., federated learning.) Historically, Splunk, Elastic, and other SIEM vendors abdicated most responsibilities here despite marketing that suggest they're good at ingest/parsing/correlation across different products ("single pane of glass"). Requiring IT/security teams to do parsing + classification across an ever-changing stack of client/server apps + security tools is a lot of work given the ever-increasing list of other responsibilities they have, and a gross duplication of effort across organizations. The result has meant a lot of incompletely configured systems, a big blocker on most data & automation efforts ("garbage in / garbage out"), and burning internal team time + professional service dollars on week 1 stuff. In contrast, not so hard for a vendor to amortize parsing across all its users. (Which Microsoft and Google are pushing towards, afaict.) And agreed, teams should be able to iterate on the interesting stuff, not reinventing the wheel on parsing winlogs/apache/netflow/zeek/palo alto/etc and normalizing them to the taxonomy of the year. To the original point about ATT&CK, teams should be training & labeling & sharing models, while ATT&CK is informal ideas (very 1970s/1980s) and manual SIGMA rules (very 1990s/2000s), yet we know that is unreliable and too much work to maintain & grow. They need to be able to quickly work w/ feature engineering teams, not work in ignorance of them nor in isolation.

badrabbit 6y ago

Wish I had time to comment on all your points. Central logging efforts with chronicle/sentinel always result in a ton of data loss. On prem is cheap and works well. Elastic/Graylog solve this well. Whether it is sentinel,chronicle or so many of the others in this market, they simply don't and never will know your environment well enough. This will always stab you in the back with the double edged sword of a) losing contextual detail that would have let you make better decisions b) Losing control over data transformation and detection/alerting logic,which means they do too good of a job in suppressing false positives that you miss bad stuff,they generate too many false alerts that waste precioud man hours,many of which could be tuned out(the bureaucracy of a 3rd party,even with admin access to their rules will always be crippling). In short, my opinion is that you need a fast failing and agile seceng(devsecops?) cycle where you have all the data (logs and enrichment) at your disposal. Your people that spend a lot of time juggling false positives could instead contribute to this, which will result in good alert fidelity and more free time for threat research and hunting! (The fun stuff!!)

lmeyerov 6y ago
We're talking about slightly different things: you're thinking cloud vs. on-prem, while I mean no longer giving SIEM vendors a free pass for crappy parsing & classification of most data and pushing that to users. Running the SIEM via a managed cloud happens to make this easier, but not actually 100% required (e.g., federated learning.) Historically, Splunk, Elastic, and other SIEM vendors abdicated most responsibilities here despite marketing that suggest they're good at ingest/parsing/correlation across different products ("single pane of glass"). Requiring IT/security teams to do parsing + classification across an ever-changing stack of client/server apps + security tools is a lot of work given the ever-increasing list of other responsibilities they have, and a gross duplication of effort across organizations. The result has meant a lot of incompletely configured systems, a big blocker on most data & automation efforts ("garbage in / garbage out"), and burning internal team time + professional service dollars on week 1 stuff. In contrast, not so hard for a vendor to amortize parsing across all its users. (Which Microsoft and Google are pushing towards, afaict.) And agreed, teams should be able to iterate on the interesting stuff, not reinventing the wheel on parsing winlogs/apache/netflow/zeek/palo alto/etc and normalizing them to the taxonomy of the year. To the original point about ATT&CK, teams should be training & labeling & sharing models, while ATT&CK is informal ideas (very 1970s/1980s) and manual SIGMA rules (very 1990s/2000s), yet we know that is unreliable and too much work to maintain & grow. They need to be able to quickly work w/ feature engineering teams, not work in ignorance of them nor in isolation.