4 ms·

Not just languages - Generally if the scanner isn't aware of the specific framework you're using it's unable to find entrypoints or reliably figure out sources

by zxcmx 6y ago

Not just languages - Generally if the scanner isn't aware of the specific framework you're using it's unable to find entrypoints or reliably figure out sources / sinks and filters. Many modern frameworks have one or more of: dynamic configuration, compile time annotations, reflection, IoC etc which make it very hard for a "first principles" scanner to make sense of what can actually happen at runtime.

pc86 6y ago
What exactly is a "'first principles' scanner?"
- zxcmx 6y ago
  Sorry, was perhaps poorly worded. By "first principles" I meant a static analyser working using only an understanding of the language (say, Java) and the application source code, looking over the ast/cfg etc using only built-in language rules. You get much better (read: maybe useful) results if you happen to have a "rule pack" for the specific framework you're using which provides specific hints on sources + sinks, "gotchas" and how things are wired together. As a somewhat obsolete example, I would not expect a sast working only from "first principles" to be able to do anything useful with a Spring XML configuration file. On the whole my experience is that these things work very well on certain types of codebases - e.g. naive PHP they can "go to town" because of the huge footgun surface and fairly direct control and data transfer. Stuff with lots of "magical" framework features and indirection (where even a human reviewer can often have trouble finding the implementation from the interface being invoked) they often silently fail to do anything useful.
  - bostik 6y ago
    > static analyser working using only an understanding of the language [...] looking over the ast The part I quoted is called "tainting". Or if you are more academically inclined, "data flow analysis". When it works right, it is an incredible force multiplier in security. You get detailed, actionable and above all helpful error messages directly from CI, because as a static analysis tool it's pretty fast and can be made part of the common linting pass. As you hinted, it does require a suitable config setup and/or code annotations to mark sources and sinks. And when it does work, it can eliminate an entire class of vulnerabilities - good taint analysis will prevent you from even accidentally using user-supplied data in anything that involves relaying, storing or displaying information. The downside is, when it doesn't work, it's a source of unhappiness. Debugging a taint failure because the AST analysis gives a false positive can be infuriating.
- schwag09 6y ago
  I suspect this means a scanner that can derive all necessary information without any configuration. For example, consider a scanner looking for API endpoint authorization inconsistencies. Does the scanner need you to describe your authorization scheme, or can you simply run the thing and it figures it out? This can be easy or hard depending on how bespoke your application is. If you're using something like Ruby on Rails, then there's a paved road that a scanner can preconfigure to understand your application. If you're using a homegrown authorization framework, then a scanner will likely have a hard time understanding your application and will need to be configured.
airza 6y ago
Even Fortify and other tools which have to compile beforehand don't do so well IMO. Someone is going to come up with an actually good SAST and print money...
- tru3_power 6y ago
  I’ve had really good experiences with Fortify so far. It took me a lot of hours to figure out how to write custom rules but once I was able to do so it became super valuable.
Rickasaurus 6y ago
I didn't write about this because it wasn't toward my exact point about FP, but you're totally right. This will be equally as chilling for new web frameworks.

by zxcmx 6y ago

pc86 6y ago

What exactly is a "'first principles' scanner?"

zxcmx 6y ago
Sorry, was perhaps poorly worded. By "first principles" I meant a static analyser working using only an understanding of the language (say, Java) and the application source code, looking over the ast/cfg etc using only built-in language rules. You get much better (read: maybe useful) results if you happen to have a "rule pack" for the specific framework you're using which provides specific hints on sources + sinks, "gotchas" and how things are wired together. As a somewhat obsolete example, I would not expect a sast working only from "first principles" to be able to do anything useful with a Spring XML configuration file. On the whole my experience is that these things work very well on certain types of codebases - e.g. naive PHP they can "go to town" because of the huge footgun surface and fairly direct control and data transfer. Stuff with lots of "magical" framework features and indirection (where even a human reviewer can often have trouble finding the implementation from the interface being invoked) they often silently fail to do anything useful.
- bostik 6y ago
  > static analyser working using only an understanding of the language [...] looking over the ast The part I quoted is called "tainting". Or if you are more academically inclined, "data flow analysis". When it works right, it is an incredible force multiplier in security. You get detailed, actionable and above all helpful error messages directly from CI, because as a static analysis tool it's pretty fast and can be made part of the common linting pass. As you hinted, it does require a suitable config setup and/or code annotations to mark sources and sinks. And when it does work, it can eliminate an entire class of vulnerabilities - good taint analysis will prevent you from even accidentally using user-supplied data in anything that involves relaying, storing or displaying information. The downside is, when it doesn't work, it's a source of unhappiness. Debugging a taint failure because the AST analysis gives a false positive can be infuriating.
schwag09 6y ago
I suspect this means a scanner that can derive all necessary information without any configuration. For example, consider a scanner looking for API endpoint authorization inconsistencies. Does the scanner need you to describe your authorization scheme, or can you simply run the thing and it figures it out? This can be easy or hard depending on how bespoke your application is. If you're using something like Ruby on Rails, then there's a paved road that a scanner can preconfigure to understand your application. If you're using a homegrown authorization framework, then a scanner will likely have a hard time understanding your application and will need to be configured.

airza 6y ago

Even Fortify and other tools which have to compile beforehand don't do so well IMO. Someone is going to come up with an actually good SAST and print money...

tru3_power 6y ago
I’ve had really good experiences with Fortify so far. It took me a lot of hours to figure out how to write custom rules but once I was able to do so it became super valuable.

Rickasaurus 6y ago

I didn't write about this because it wasn't toward my exact point about FP, but you're totally right. This will be equally as chilling for new web frameworks.