7 ms·

This is why it's increasingly important to not only test your code, but also develop useful vulnerability scanners. I know, it sounds impossible to write a comp

by Rygu 12y ago

This is why it's increasingly important to not only test your code, but also develop useful vulnerability scanners. I know, it sounds impossible to write a complete vulnerability scanner, but many companies would pay for such a thing. Existing solutions are either lacking or way too difficult to operate.

homakov 12y ago
That's true, checking all `rake routes` for proper CSRF handling is a trivial job but would help to spot the bug in 5 seconds.
sasas 12y ago
> Existing solutions are either lacking or way too difficult to operate Give Netsparker[1] a go.. couldn't get much easier then this! [1] https://www.netsparker.com/ https://www.netsparker.com/
lmm 12y ago
Any sufficiently advanced vulnerability scanner contains an contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of a type system. Seriously, the solution isn't better scanners or better tests, it's better languages. Make whether something is taking a potentially-destructive action part of its type, and then the compiler ensures that all such routes are appropriately protected; you can do this today in e.g. Spray (which I'm using in production, so this is not some ivory-tower theoretical solution).
- michaelmior 12y ago
  Assuming the application code is completely secure, then that's a big part of the problem solved. But there's also the application server and any other services running on the machine hosting it.
  - lmm 12y ago
    So what, we give up entirely? Most times I'm aware of a site being hacked it's been through a hole in their application code, not the stack they're running on top of.
    - michaelmior 12y ago
      I was not suggesting we give up entirely. I'm just pointing out that it isn't a complete solution of the problem. But I agree that application code is probably the bigger problem to solve.
- dsacco 12y ago
  This suggestion eliminates bugs that are introduced by unsafe typing, sure. But race conditions, logic flaws, misconfigured tools and almost every class of web vulnerability wouldn't be eliminated just by implementing this. Most vulnerabilities do not derive from a single faulty tool or framework, but the incorrect combination of several (faulty or not!) tools over a sufficiently large attack surface.
  - lmm 12y ago
    > This suggestion eliminates bugs that are introduced by unsafe typing, sure. But race conditions, logic flaws, misconfigured tools and almost every class of web vulnerability wouldn't be eliminated just by implementing this. How often are security flaws a subtle, complex thing that couldn't have been caught by a sensible type system? Almost all of the flaws we hear about, at least here, are the really simple dumb mistakes that a better language absolutely would have caught.
    - kofalt 12y ago
      Agreed. I try to explain sometimes that I'm merely tired of experiencing the same problem, over and over, for years. I would like to experience the next problem, please. If only for variety's sake.
    - rudolf0 12y ago
      Do you mean a language with a very strong type system like Haskell, where you could create a web framework in which things like user input have different types associated with them? Those can offer some real advantages by never mixing untrusted input with page output and various calls to other systems. But you don't get a lot of web application security advantages when you compare, say, Python and Java. Of course, a very weak type system like PHP's can definitely introduce additional security flaws.
      - jfoutz 12y ago
        A common pattern (at least getting there) in Scala/F#/Haskell is to have paired types, UserInput and ValidatedUserInput. All the business logic requires the Validated kind. A programmer can be certainly be lazy and just return the supplied value, but this is a conscious choice to do the wrong thing. You can't accidentally forget to validate form parameters. It won't compile. This is pretty easy to do in Java. On the weaker side, I would assume php and python have something like perl's taint mode. It's such a simple thing and it avoids so many xss problems.
- JoachimSchipper 12y ago
  Types are not that useful for security - it's as easy to create a MutatingController in Ruby as it is to concatenate some strings containing SQL and a string containing raw input in Haskell. Better typing can help, and Ruby programs are more likely to be "stringly typed" than Haskell programs in practice - but the difference is a lot more subtle than one might expect.
  - mej10 12y ago
    The issue is that Ruby can't give you the tools you need to catch such things before runtime. Haskell can.
michaelmior 12y ago
I think it is impossible to write a complete vulnerability scanner. If by complete you mean that your application should be guaranteed "secure" after running the scanner. Security has always been a moving target. However, I've heard great things about the job that Acunetix[0] does. [0]
acangiano 12y ago
Have you looked at IBM AppScan? It does a pretty good job.
dsacco 12y ago
There is a fine line between automating fully-automatable penetration testing tasks and automated scanning. Unfortunately, there is a very idealistic drive to feature-creep penetration testing tools away from "useful when used by a professional" to "half as useful when used by anybody and full of false positives." The real solution is not more or better security software, but rather more secure coding practices. People generally don't accept this idea, but the state of software security is such a moving target that no amount of automation short of strong AI will find every bug. The onus is on developer education.
- danielweber 12y ago
  > "useful when used by a professional" to "half as useful when used by anybody and full of false positives." Where I work we have a desire for both. We want developers to be able to scan their own code -- preferably automatically as part of a CI process -- for things that are clearly wrong, without needing to be appsec experts, and clear out a lot of the low-level brush. We also want complex tools that are used by appsec to be able to do their jobs faster. This is timely since I'm about to do a search for tools for this. I'd love, for example, a way to see the permissions for all lines returned by "rake routes", whether those are resolved by CanCan or where CSRF is disabled or whether it's been overruled by some skip_authorization_check.
miah_ 12y ago
Or use mutation testing: http://en.wikipedia.org/wiki/Mutation_testing http://en.wikipedia.org/wiki/Mutation_testing

by Rygu 12y ago

homakov 12y ago

That's true, checking all `rake routes` for proper CSRF handling is a trivial job but would help to spot the bug in 5 seconds.

sasas 12y ago

> Existing solutions are either lacking or way too difficult to operate Give Netsparker[1] a go.. couldn't get much easier then this! [1] https://www.netsparker.com/ https://www.netsparker.com/

lmm 12y ago

Any sufficiently advanced vulnerability scanner contains an contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of a type system. Seriously, the solution isn't better scanners or better tests, it's better languages. Make whether something is taking a potentially-destructive action part of its type, and then the compiler ensures that all such routes are appropriately protected; you can do this today in e.g. Spray (which I'm using in production, so this is not some ivory-tower theoretical solution).

michaelmior 12y ago
Assuming the application code is completely secure, then that's a big part of the problem solved. But there's also the application server and any other services running on the machine hosting it.
- lmm 12y ago
  So what, we give up entirely? Most times I'm aware of a site being hacked it's been through a hole in their application code, not the stack they're running on top of.
  - michaelmior 12y ago
    I was not suggesting we give up entirely. I'm just pointing out that it isn't a complete solution of the problem. But I agree that application code is probably the bigger problem to solve.
dsacco 12y ago
This suggestion eliminates bugs that are introduced by unsafe typing, sure. But race conditions, logic flaws, misconfigured tools and almost every class of web vulnerability wouldn't be eliminated just by implementing this. Most vulnerabilities do not derive from a single faulty tool or framework, but the incorrect combination of several (faulty or not!) tools over a sufficiently large attack surface.
- lmm 12y ago
  > This suggestion eliminates bugs that are introduced by unsafe typing, sure. But race conditions, logic flaws, misconfigured tools and almost every class of web vulnerability wouldn't be eliminated just by implementing this. How often are security flaws a subtle, complex thing that couldn't have been caught by a sensible type system? Almost all of the flaws we hear about, at least here, are the really simple dumb mistakes that a better language absolutely would have caught.
  - kofalt 12y ago
    Agreed. I try to explain sometimes that I'm merely tired of experiencing the same problem, over and over, for years. I would like to experience the next problem, please. If only for variety's sake.
  - rudolf0 12y ago
    Do you mean a language with a very strong type system like Haskell, where you could create a web framework in which things like user input have different types associated with them? Those can offer some real advantages by never mixing untrusted input with page output and various calls to other systems. But you don't get a lot of web application security advantages when you compare, say, Python and Java. Of course, a very weak type system like PHP's can definitely introduce additional security flaws.
    - jfoutz 12y ago
      A common pattern (at least getting there) in Scala/F#/Haskell is to have paired types, UserInput and ValidatedUserInput. All the business logic requires the Validated kind. A programmer can be certainly be lazy and just return the supplied value, but this is a conscious choice to do the wrong thing. You can't accidentally forget to validate form parameters. It won't compile. This is pretty easy to do in Java. On the weaker side, I would assume php and python have something like perl's taint mode. It's such a simple thing and it avoids so many xss problems.
JoachimSchipper 12y ago
Types are not that useful for security - it's as easy to create a MutatingController in Ruby as it is to concatenate some strings containing SQL and a string containing raw input in Haskell. Better typing can help, and Ruby programs are more likely to be "stringly typed" than Haskell programs in practice - but the difference is a lot more subtle than one might expect.
- mej10 12y ago
  The issue is that Ruby can't give you the tools you need to catch such things before runtime. Haskell can.

michaelmior 12y ago

I think it is impossible to write a complete vulnerability scanner. If by complete you mean that your application should be guaranteed "secure" after running the scanner. Security has always been a moving target. However, I've heard great things about the job that Acunetix[0] does. [0]

acangiano 12y ago

Have you looked at IBM AppScan? It does a pretty good job.

dsacco 12y ago

There is a fine line between automating fully-automatable penetration testing tasks and automated scanning. Unfortunately, there is a very idealistic drive to feature-creep penetration testing tools away from "useful when used by a professional" to "half as useful when used by anybody and full of false positives." The real solution is not more or better security software, but rather more secure coding practices. People generally don't accept this idea, but the state of software security is such a moving target that no amount of automation short of strong AI will find every bug. The onus is on developer education.

danielweber 12y ago
> "useful when used by a professional" to "half as useful when used by anybody and full of false positives." Where I work we have a desire for both. We want developers to be able to scan their own code -- preferably automatically as part of a CI process -- for things that are clearly wrong, without needing to be appsec experts, and clear out a lot of the low-level brush. We also want complex tools that are used by appsec to be able to do their jobs faster. This is timely since I'm about to do a search for tools for this. I'd love, for example, a way to see the permissions for all lines returned by "rake routes", whether those are resolved by CanCan or where CSRF is disabled or whether it's been overruled by some skip_authorization_check.

miah_ 12y ago

Or use mutation testing: http://en.wikipedia.org/wiki/Mutation_testing http://en.wikipedia.org/wiki/Mutation_testing