5 ms·

let me introduce you to the much better and reliable world of: static analysis

hashtag-til 2y ago
I feel we're going to have a hard time over the next months with a stream of these "magic tools" to solve already solved problems and try to milk some money out off managers who got no clue.
- dartos 2y ago
  That strategy has been working for the past 6 or so years.
- robszumski 2y ago
  Static analysis paired with AI is the middle ground that makes sense to me (working in a similar security space). But the hard part needs to be regular computer science and the AI comes second.
  - funcDropShadow 2y ago
    > But the hard part needs to be regular computer science and the AI comes second. Yes, indeed. The AI could be used to prefilter the list of warnings generated by static analysis to reduce the amount of false positives. To achieve that an AI could use the history of the projects static analysis results to find likely false positives. Or an I could propose a patch to avoid a warning. If it is automatically compiled, passed to the test suite and the whole ci pipeline, it could reduce the manual effort to deal with finding of static analysis tools. But leaving out the static analysis tools would loose so much value.
  - asadeddin 2y ago
    We completely agree. I would redefine it a bit. We combine static analysis + LLMs to do better detection, triaging and auto-fixing because static analysis alone is broken in many ways. We've been able to reduce ~30% of tickets for customers with false positive detection, and now be able to detect classes of vulnerabilities in business and code logic that were previously undetectable.
deleted 2y ago
[deleted]
asadeddin 2y ago
I would redefine it a bit. Reliable = deterministic Accurate? Not at all. Studies show that ~30% of findings are false positive. We've also seen that with the companies we work with because we built a false positive detection feature in Corgea. There's another ~60% of issues that are false negative. https://personal.utdallas.edu/~lxz144130/publications/icst2016.pdf https://personal.utdallas.edu/~lxz144130/publications/icst20... We combine static analysis + LLMs to do better detection, triaging and auto-fixing because static analysis alone is broken in many ways.

I feel we're going to have a hard time over the next months with a stream of these "magic tools" to solve already solved problems and try to milk some money out off managers who got no clue.

dartos 2y ago
That strategy has been working for the past 6 or so years.
robszumski 2y ago
Static analysis paired with AI is the middle ground that makes sense to me (working in a similar security space). But the hard part needs to be regular computer science and the AI comes second.
- funcDropShadow 2y ago
  > But the hard part needs to be regular computer science and the AI comes second. Yes, indeed. The AI could be used to prefilter the list of warnings generated by static analysis to reduce the amount of false positives. To achieve that an AI could use the history of the projects static analysis results to find likely false positives. Or an I could propose a patch to avoid a warning. If it is automatically compiled, passed to the test suite and the whole ci pipeline, it could reduce the manual effort to deal with finding of static analysis tools. But leaving out the static analysis tools would loose so much value.
- asadeddin 2y ago
  We completely agree. I would redefine it a bit. We combine static analysis + LLMs to do better detection, triaging and auto-fixing because static analysis alone is broken in many ways. We've been able to reduce ~30% of tickets for customers with false positive detection, and now be able to detect classes of vulnerabilities in business and code logic that were previously undetectable.

deleted 2y ago

[deleted]

asadeddin 2y ago

I would redefine it a bit. Reliable = deterministic Accurate? Not at all. Studies show that ~30% of findings are false positive. We've also seen that with the companies we work with because we built a false positive detection feature in Corgea. There's another ~60% of issues that are false negative. https://personal.utdallas.edu/~lxz144130/publications/icst2016.pdf https://personal.utdallas.edu/~lxz144130/publications/icst20... We combine static analysis + LLMs to do better detection, triaging and auto-fixing because static analysis alone is broken in many ways.