5 ms·

Are there people working in cybersecurity who actually use those code querying tools in their daily work? I've heard of some showcase projects that end up in bl

Are there people working in cybersecurity who actually use those code querying tools in their daily work? I've heard of some showcase projects that end up in blog posts and such, but everyone I've spoken to who's tried CodeQL or Joern or similar says that by the time you've figured out your query with all its edge cases, you might as well have just looked at the code and found any vulns quicker that way. And probably with a better understanding of the program too.

dragonwriter 3y ago
I imagine that lots of “vulnerability detection software/SaaS” is built on that kind of tools plus queries developed for specific vulnerabilities. OTOH, my experience with these systems has been that they are popular with enterprise ISOs, but have very high inaccuracy (lots of false positives, and lots of misses on the kind of vulns they purport to detect), and while they are marginally useful, they seem do more for security checklist compliance than for security.
- pentaphobe 3y ago
  Agree on inaccuracy, but I don't think that this reduces its value as much as you posit. Ultimately other forms of static analysis won't guarantee my code is good, but I still use linters. Similarly, tests (unit, integration, etc..) won't prove that nothing can go wrong - but I'm not going to stop writing them. I'd love to be using a "perfect" language which could prove my program correct at compile time, and would presumedly also make static analysis borderline magical for these use cases - but until that's an option I think there's a place for all these tools. (Beyond simply ticking compliance boxes) With all that said, false negatives are indeed a hard problem - and one not helped by large orgs having painful bureaucracy around false _positives_. Some of this appears to be the fault of tooling (need better filtration, deferral, weighting) but much of it seems a side effect of institutional silo's rather than a lack of perfect analyses. TLDR; pobody's nerfect, but more info generally better than less
chc4 3y ago
Most of the time you shouldn't worry too much about all the edge cases. I mostly use it for a general query for variant analysis of some fixed bug, which will have false positives but also give you a list of actual code positions you can triage and review. You still are looking at the code, like in your case, you're just doing it faster or more confident you're looking at all the relevant code instead of having to grep and pray there isn't a construct that had an extra space somewhere. It's also useful IMO as basically just a search engine for constructs you're interested in when approaching new codebases: "show me all classes that inherit from X and have a pointer member of type Y" or whatever is really easy to query, and something that comes up surprisingly often.
- riku_iki 3y ago
  for someone not familiar with this topic, maybe you can give one/two sentences description of some usecase which you use this for?..

dragonwriter 3y ago

I imagine that lots of “vulnerability detection software/SaaS” is built on that kind of tools plus queries developed for specific vulnerabilities. OTOH, my experience with these systems has been that they are popular with enterprise ISOs, but have very high inaccuracy (lots of false positives, and lots of misses on the kind of vulns they purport to detect), and while they are marginally useful, they seem do more for security checklist compliance than for security.

pentaphobe 3y ago
Agree on inaccuracy, but I don't think that this reduces its value as much as you posit. Ultimately other forms of static analysis won't guarantee my code is good, but I still use linters. Similarly, tests (unit, integration, etc..) won't prove that nothing can go wrong - but I'm not going to stop writing them. I'd love to be using a "perfect" language which could prove my program correct at compile time, and would presumedly also make static analysis borderline magical for these use cases - but until that's an option I think there's a place for all these tools. (Beyond simply ticking compliance boxes) With all that said, false negatives are indeed a hard problem - and one not helped by large orgs having painful bureaucracy around false _positives_. Some of this appears to be the fault of tooling (need better filtration, deferral, weighting) but much of it seems a side effect of institutional silo's rather than a lack of perfect analyses. TLDR; pobody's nerfect, but more info generally better than less

chc4 3y ago

Most of the time you shouldn't worry too much about all the edge cases. I mostly use it for a general query for variant analysis of some fixed bug, which will have false positives but also give you a list of actual code positions you can triage and review. You still are looking at the code, like in your case, you're just doing it faster or more confident you're looking at all the relevant code instead of having to grep and pray there isn't a construct that had an extra space somewhere. It's also useful IMO as basically just a search engine for constructs you're interested in when approaching new codebases: "show me all classes that inherit from X and have a pointer member of type Y" or whatever is really easy to query, and something that comes up surprisingly often.

riku_iki 3y ago
for someone not familiar with this topic, maybe you can give one/two sentences description of some usecase which you use this for?..