4 ms·

Sleepy Puppy: XSS detection tool

45 points by pm24601 11y ago

WA 11y ago
With delayed XSS testing, security engineers inject an XSS payload on one application that may get reflected back in a separate application with a different origin. No need to go beyond application scope. Most tools even fail to detect XSS if it is reflected not in the immediate response, but in a later response. They inject the XSS payload and check if the immediate HTTP response contains that very string. If not, there's no XSS. Maybe Sleepy Puppy is a solution. You could now define some standard click-paths through your app with Selenium IDE and inject various payloads into every parameter. I didn't take a deeper look in how Sleepy Puppy works, but automated workflow execution + payload injection + delayed XSS detection is what matters in modern complex web apps.
billyhoffman 11y ago
Sleepy Puppy doesn't really address the primary problem I've found when trying to automate the detection of non-reflected XSS issues: Mapping the input points and outputs outputs The first thing you need to do when testing for XSS is you need to find where a user-supplied input is returned back out to a user. In reflected XSS, its the page you sent the request to. With "Stored" or "Delayed" XSS, it's somewhere else in the app or even a different app entirely. Building this mapping when input and output points are in different areas, and doing that automatically has 2 challenges: - Clobbering your input - Finding the output point. Clobbering input is something automated tools are terrible at. Let's say you are auditing a "save my settings" functionality in an e-com store, but the address values you save aren't actually displayed until later in the checkout process (i.e. you are dealing with a stored XSS). Let's say the vulnerable input is the state name (devs often forget to validate input when the input is a dropdown, because they forget people can make that input be anything they want). You submit an attack string for the address, and valid input for state and zip. You don't see it reflected. You then do valid input for address and zip, but state has an attack string. Nice, but that doesn't appear until to later in the app. As an attacker, you don't know this, so you see its not a reflected XSS, and you do a 3rd attempt and submit valid input for address and state, and attack the zip. You have now clobbered your attack string that you submitted against the State with a valid, normal input for the state. When you get deeper into the app and see the state returned in the checkout process, it wouldn't be rendering your attack probe. That got clobbered by the valid input. Unless you inject an attack token, and then sweep the entire site for that token, you won't find most stored/delayed XSS. Probe address field, crawl rest of site looking for your probe. Probe State field, crawl rest of site looking for your probe. This is super time consuming to do. Automated tools suck at this. Humans are a better, because they understand context. I know that an address associated with an account is a likely stored XSS vector, and I have a gut feeling about where in the app the address will be rendered. Knowing context helps you also prune the number of inputs you should test for stored XSS. Unless I find an admin or analytics area in a site, I'm not going to bother doing this work to test if an HTTP header like User-Agent or Referer [sic] is a stored XSS vector as it is highly unlikely those inputs are reflected out somewhere else in the site. If you have an output in another app its super hard. Sleepy Puppy tries to get around this by having the XSS payload "phone home" and pull an external JS file for a site they control to register where the output point is. While this is the best attempt I've seen, it has short comings. CSP defeats this. Also, it means the vulnerable input has to allow far more more characters to do the attack. spaces, quotes, parens etc cannot get filtered or encoded for the various Sleepy Puppy payloads to work. This is a hard problem to solve. I'm glad NetFlix open sourced this.
- pm24601 11y ago
  I agree with you about Sleepy Puppy not being a universal solution - however this is true for almost every tool. The big wins with SP: * automated * narrows what kind of XSS can be present. Security seems to be like quantum mechanics: there is always an uncertainty principle that prevents perfection.
altano 11y ago
Has anyone demonstrated XSS attacks that work even when CSP is properly configured? Seems like all discussion of XSS now-a-days should be preceded with a note that you should go turn on CSP before worrying about any other form of prevention.
- seiji 11y ago
  Doesn't that depend on browser vendors supporting and properly implementing CSP? What about every embedded device browser that will never receive an update? Does your toaster, fridge, microwave, and toilet get security updates?
- david_shaw 11y ago
  From what I've seen, no -- but that depends what you mean by "properly configured." If you look at challenges like this ( https://github.com/cure53/XSSChallengeWiki/wiki/H5SC-Minichallenge-3:-%22Sh*t,-it%27s-CSP!%22 https://github.com/cure53/XSSChallengeWiki/wiki/H5SC-Minicha... ), you'll see that CSP appears to be properly configured, but there are actually ways to bypass. It's problems like this that I've seen "in the wild" most of the time. The other side of the argument is that not many people are building web applications with CSP built-in from the beginning. If you don't start development with CSP in mind, it can become very cumbersome to put CSP into place (outside of report-only mode). As a result, a lot of people are currently using CSP as just report-only analytics on potential violations.

45 points by pm24601 11y ago

WA 11y ago

With delayed XSS testing, security engineers inject an XSS payload on one application that may get reflected back in a separate application with a different origin. No need to go beyond application scope. Most tools even fail to detect XSS if it is reflected not in the immediate response, but in a later response. They inject the XSS payload and check if the immediate HTTP response contains that very string. If not, there's no XSS. Maybe Sleepy Puppy is a solution. You could now define some standard click-paths through your app with Selenium IDE and inject various payloads into every parameter. I didn't take a deeper look in how Sleepy Puppy works, but automated workflow execution + payload injection + delayed XSS detection is what matters in modern complex web apps.

billyhoffman 11y ago

Sleepy Puppy doesn't really address the primary problem I've found when trying to automate the detection of non-reflected XSS issues: Mapping the input points and outputs outputs The first thing you need to do when testing for XSS is you need to find where a user-supplied input is returned back out to a user. In reflected XSS, its the page you sent the request to. With "Stored" or "Delayed" XSS, it's somewhere else in the app or even a different app entirely. Building this mapping when input and output points are in different areas, and doing that automatically has 2 challenges: - Clobbering your input - Finding the output point. Clobbering input is something automated tools are terrible at. Let's say you are auditing a "save my settings" functionality in an e-com store, but the address values you save aren't actually displayed until later in the checkout process (i.e. you are dealing with a stored XSS). Let's say the vulnerable input is the state name (devs often forget to validate input when the input is a dropdown, because they forget people can make that input be anything they want). You submit an attack string for the address, and valid input for state and zip. You don't see it reflected. You then do valid input for address and zip, but state has an attack string. Nice, but that doesn't appear until to later in the app. As an attacker, you don't know this, so you see its not a reflected XSS, and you do a 3rd attempt and submit valid input for address and state, and attack the zip. You have now clobbered your attack string that you submitted against the State with a valid, normal input for the state. When you get deeper into the app and see the state returned in the checkout process, it wouldn't be rendering your attack probe. That got clobbered by the valid input. Unless you inject an attack token, and then sweep the entire site for that token, you won't find most stored/delayed XSS. Probe address field, crawl rest of site looking for your probe. Probe State field, crawl rest of site looking for your probe. This is super time consuming to do. Automated tools suck at this. Humans are a better, because they understand context. I know that an address associated with an account is a likely stored XSS vector, and I have a gut feeling about where in the app the address will be rendered. Knowing context helps you also prune the number of inputs you should test for stored XSS. Unless I find an admin or analytics area in a site, I'm not going to bother doing this work to test if an HTTP header like User-Agent or Referer [sic] is a stored XSS vector as it is highly unlikely those inputs are reflected out somewhere else in the site. If you have an output in another app its super hard. Sleepy Puppy tries to get around this by having the XSS payload "phone home" and pull an external JS file for a site they control to register where the output point is. While this is the best attempt I've seen, it has short comings. CSP defeats this. Also, it means the vulnerable input has to allow far more more characters to do the attack. spaces, quotes, parens etc cannot get filtered or encoded for the various Sleepy Puppy payloads to work. This is a hard problem to solve. I'm glad NetFlix open sourced this.

pm24601 11y ago
I agree with you about Sleepy Puppy not being a universal solution - however this is true for almost every tool. The big wins with SP: * automated * narrows what kind of XSS can be present. Security seems to be like quantum mechanics: there is always an uncertainty principle that prevents perfection.

altano 11y ago

Has anyone demonstrated XSS attacks that work even when CSP is properly configured? Seems like all discussion of XSS now-a-days should be preceded with a note that you should go turn on CSP before worrying about any other form of prevention.

seiji 11y ago
Doesn't that depend on browser vendors supporting and properly implementing CSP? What about every embedded device browser that will never receive an update? Does your toaster, fridge, microwave, and toilet get security updates?
david_shaw 11y ago
From what I've seen, no -- but that depends what you mean by "properly configured." If you look at challenges like this ( https://github.com/cure53/XSSChallengeWiki/wiki/H5SC-Minichallenge-3:-%22Sh*t,-it%27s-CSP!%22 https://github.com/cure53/XSSChallengeWiki/wiki/H5SC-Minicha... ), you'll see that CSP appears to be properly configured, but there are actually ways to bypass. It's problems like this that I've seen "in the wild" most of the time. The other side of the argument is that not many people are building web applications with CSP built-in from the beginning. If you don't start development with CSP in mind, it can become very cumbersome to put CSP into place (outside of report-only mode). As a result, a lot of people are currently using CSP as just report-only analytics on potential violations.