6 ms·

DDoS Attack Used 'Headless' Browsers In 150-Hour Siege

48 points by Igalze 13y ago

eulertotient 13y ago
What usually happens in these instances? A warrant for one of those computers at any of those 185,000 IPs needs to be examined, monitored, and then the proxy-snail-turtle-chain needs to be traced, which could be long, onioned (like Tor) and well -- obfuscated with lots of extra superfluous traffic. Are botnets just considered collateral in a highly vulnerable computersphere? When does anything get done, I really have disdain for the kind of people who run botnets. Every single one I've met, years ago, when I was more in the scene was a power-hungry tool out for petty cash through bribery and destruction.
roadster72 13y ago
>Incapsula was able to mitigate the attack. I wonder how considering the packets would not necessarily be similar.
- eksith 13y ago
  Maybe they're using another form of pattern recognition. Time of day for the originating traffic vs. area of site being visited or particular type of stock etc...
- alexvay 13y ago
  I'd assume the HEADERs would be the same, e.g. basic stuff like UAS could be the same, or more low-level like Accept-* headers. Note that this doesn't seem like a DDoS focused on high-bandwidth but rather focused on causing application load by triggering some server action and abusing the CPU/RAM, etc. In this case, Level7 analysis, is very effective and usually simple.
  - Igalze 13y ago
    Very true. As far as headers concerned, we actually dig very deep. For instance, we will look at little encoding-related nuances, which can help identify spoofed headers (ua and IPs are fakeable, after all) :) Also, we look for abnormalities in header order while being aware of variants that can derive from using various devices, proxies, etc. Hence the 10M signature pool, which grows as new variants are spotted across our network.
- jgreen10 13y ago
  Blocking traffic from Chinese IPs goes a long way.
  - deleted 13y ago
    [deleted]
- Igalze 13y ago
  Hi, I actually work for Incapsula. For Layer 7 mitigation we use a multi-vector approach which' among other things, consists of: Client Classification - comparing visitor's user-agent, IP, header parameters and etc to our pool of 10M signatures. Suspects will get CAPTCHA. (~0.01% false positives) Visitor Reputation - we use crowd-sourcing to compile a list of suspected IPs. The list is updated in real time. Combined with other signals, this data allows us a better understanding of the incoming traffic. Progressive Challenges - We check visitor's ability to retain cookies, execute JS and so on. In this case, the browser-based bots were able to evade those defenses. (These are also the most commonly used Layer 7 mitigation methods.) Behavior Monitoring - We look at abnormal access rates, visiting patterns, etc. Here we also look for correctional of signals, to help us pinpoint suspicious behavior. And so, by collecting and cross-referencing different types of data, the system is designed to distinguish between humans and bots. The process is mostly automated and is always seamless.
  - STRML 13y ago
    This is a great & informative answer. PhantomJS allows an attacker to get past any progressive challenge but it is nice that there is something else to go on. A properly-executed PhantomJS DDoS is a scary thing, it's great that you have some methods of mitigation.
larsmak 13y ago
These stories about mitigating DDoS-attacks often comes up, but there is seldom any actual details on how it was done. I guess it's because there often is a manual process, analyzing logs and banning IPs and/or IP-ranges. But for these more automated mitigating services, what technics are used? I guess some of the techniques are based on complex machine-learning algorithms and considered company secrets. But there must be some general, more well known, automated mitigation techniques / indicators? For application-level attacks is it enough to ban the offending IPs in the firewall? If so, how do one best identify these IPs? Assuming one have access to the access-log, I can come up with a few ideas: - Like mentioned above; banning IPs / IP-ranges originating from countries normally not associated with traffic to the site (i.e. build a statistical knowledge beforehand) - Simple request counting, per ip, and IP-ban users who are overly active (again based on the sites statistical profile) - More advanced request counting, trying to create a fingerprint of the users behaviour (time between requests, etc) - and comparing this to the sites average fingerprint. - Keep a global request-per-time-count and use this to measure the current load of the site - the mitigation techniques will only be activated if the site is under heavy-load / attack. Any other ideas?
jqueryin 13y ago
I was entirely wrong when I thought that I understood DDoS attacks fairly well. There's actually at least three different types of attacks: "Layer 3 and Layer 4 DDoS attacks are types of volumetric DDoS attacks on a network infrastructure. Layer 3 (network layer) and 4 (transport layer) DDoS attacks rely on extremely high volumes (floods) of data to slow down web server performance, consume bandwidth and eventually degrade access for legitimate users. These attack types typically include ICMP, SYN, and UDP floods." "A Layer 7 DDoS attack is an attack structured to overload specific elements of an application server infrastructure. Layer 7 attacks are especially complex, stealthy, and difficult to detect because they resemble legitimate website traffic. Even simple Layer 7 attacks – for example those targeting login pages with random user IDs and passwords, or repetitive random searches on dynamic websites – can critically overload CPUs and databases. Also, DDoS attackers can randomize or repeatedly change the signatures of a Layer 7 attack, making it more difficult to detect and mitigate." This glossary proved to be quite useful: http://www.prolexic.com/knowledge-center-dos-and-ddos-glossary.html http://www.prolexic.com/knowledge-center-dos-and-ddos-glossa...
- justkez 13y ago
  It raises interesting questions about how companies like Incapsula will integrate with their customers in the future. Fairly easy for them to sit and monitor the low level networking kit dealing with Layer 3 + 4 attacks, but that's the wrong place to focus on Layer 7, right? They need to understand the web app and infrastructure it touches. I'm sure they exist in some form or another, but it's likely we'll see more design patterns emerging to deal with mitigating application attacks. Combined with the network/protocol control you'd have application logic feeding down suspected attack data to block
  - Igalze 13y ago
    You raise an excellent point. We have a lot of application awareness configurations and our signature pool (which is one of the tools we use for traffic profiling) holds over 10M variant, sometime as much as ~20,000 per user-agent type, to cover all scenarios.
level09 13y ago
How would you differentiate the traffic coming form headless browsers from normal users ? with a proper user-agent and javascript enabled, isn't it almost impossible ?
- zzzcpan 13y ago
  No, it's not impossible. Basically, you detect anomalies, i.e. gather statistical information about your users and their behavior in advance and apply that as a filter during attacks. For example, if you have like 10 new refererless users per hour, i.e. without a cookie and a referrer, but suddenly there are 1000 of them and backend is already slow - you should probably drop their requests. Same with specific referrers and any other information you have about your users. But this is kind of preventative only approach, it won't work if your site is already under attack.
  - STRML 13y ago
    The Referer header is just one more thing that a sophisticated attacker will spoof. I think the GP is correct - with enough preparation, a concentrated PhantomJS attack will look exactly like users accessing the website en masse, and it can be very, very difficult to tell who is real and who is not. That is why they use a tool as heavy as PhantomJS, which (from experience) is not very quick compared to more traditional methods. It is because it spoofs a real browser quite completely that it is so good for these methods. For example, some pages might do a trick where they set a cookie, load another page, and redirect / drop based on whether or not that cookie is present in exactly the form it should be present. Maybe they set two, one expires immediately and the other contains some complex data. PhantomJS can handle that easily (as it is just Webkit + QT) but traditional tools do not.
    - zzzcpan 13y ago
      > The Referer header is just one more thing > that a sophisticated attacker will spoof. It doesn't really matter, as attacker wouldn't have your data. For example, you know that on average you have 10 users per hour from some specific referrer and that most of them are coming from a single country. And you know this for every referrer. But attacker doesn't, even if he tries to fake it, his requests are either not going to satisfy the profile (he won't get something right, like country, user-agent, etc) or he will reach the limit (by trying to send more than N requests per hour for specific referrer, as he doesn't know the limit either). Of course you could make it very precise by using a lot more data points for each user: where users usually go after which URL, how long do they stay on each URL, how many requests do they usually make, do they have your cookie, how long, etc. And I'm talking about cookies as a sign of a user, who already visited your website in the past.
al1x 13y ago
I started a thread where I was hoping to get a discussion of DDOS mitigation techniques, tools, and providers. Something that should be useful and highly relevant to web-based small business owners. It's here if anyone's interested -- https://news.ycombinator.com/item?id=6587640 https://news.ycombinator.com/item?id=6587640
k3n 13y ago
Fair warning - I just got pinged from corporate security that this site triggered an alert on our IDS: "the IDS reported the machine accessed a site containing a JavaScript known to contain hidden iFrames and malware redirects". He said it was named "tongji.js". Can't investigate further as I'm still on said network and don't want to re-ping it...

48 points by Igalze 13y ago

What usually happens in these instances? A warrant for one of those computers at any of those 185,000 IPs needs to be examined, monitored, and then the proxy-snail-turtle-chain needs to be traced, which could be long, onioned (like Tor) and well -- obfuscated with lots of extra superfluous traffic. Are botnets just considered collateral in a highly vulnerable computersphere? When does anything get done, I really have disdain for the kind of people who run botnets. Every single one I've met, years ago, when I was more in the scene was a power-hungry tool out for petty cash through bribery and destruction.

roadster72 13y ago

>Incapsula was able to mitigate the attack. I wonder how considering the packets would not necessarily be similar.

eksith 13y ago
Maybe they're using another form of pattern recognition. Time of day for the originating traffic vs. area of site being visited or particular type of stock etc...
alexvay 13y ago
I'd assume the HEADERs would be the same, e.g. basic stuff like UAS could be the same, or more low-level like Accept-* headers. Note that this doesn't seem like a DDoS focused on high-bandwidth but rather focused on causing application load by triggering some server action and abusing the CPU/RAM, etc. In this case, Level7 analysis, is very effective and usually simple.
- Igalze 13y ago
  Very true. As far as headers concerned, we actually dig very deep. For instance, we will look at little encoding-related nuances, which can help identify spoofed headers (ua and IPs are fakeable, after all) :) Also, we look for abnormalities in header order while being aware of variants that can derive from using various devices, proxies, etc. Hence the 10M signature pool, which grows as new variants are spotted across our network.
jgreen10 13y ago
Blocking traffic from Chinese IPs goes a long way.
- deleted 13y ago
  [deleted]
Igalze 13y ago
Hi, I actually work for Incapsula. For Layer 7 mitigation we use a multi-vector approach which' among other things, consists of: Client Classification - comparing visitor's user-agent, IP, header parameters and etc to our pool of 10M signatures. Suspects will get CAPTCHA. (~0.01% false positives) Visitor Reputation - we use crowd-sourcing to compile a list of suspected IPs. The list is updated in real time. Combined with other signals, this data allows us a better understanding of the incoming traffic. Progressive Challenges - We check visitor's ability to retain cookies, execute JS and so on. In this case, the browser-based bots were able to evade those defenses. (These are also the most commonly used Layer 7 mitigation methods.) Behavior Monitoring - We look at abnormal access rates, visiting patterns, etc. Here we also look for correctional of signals, to help us pinpoint suspicious behavior. And so, by collecting and cross-referencing different types of data, the system is designed to distinguish between humans and bots. The process is mostly automated and is always seamless.
- STRML 13y ago
  This is a great & informative answer. PhantomJS allows an attacker to get past any progressive challenge but it is nice that there is something else to go on. A properly-executed PhantomJS DDoS is a scary thing, it's great that you have some methods of mitigation.

larsmak 13y ago

These stories about mitigating DDoS-attacks often comes up, but there is seldom any actual details on how it was done. I guess it's because there often is a manual process, analyzing logs and banning IPs and/or IP-ranges. But for these more automated mitigating services, what technics are used? I guess some of the techniques are based on complex machine-learning algorithms and considered company secrets. But there must be some general, more well known, automated mitigation techniques / indicators? For application-level attacks is it enough to ban the offending IPs in the firewall? If so, how do one best identify these IPs? Assuming one have access to the access-log, I can come up with a few ideas: - Like mentioned above; banning IPs / IP-ranges originating from countries normally not associated with traffic to the site (i.e. build a statistical knowledge beforehand) - Simple request counting, per ip, and IP-ban users who are overly active (again based on the sites statistical profile) - More advanced request counting, trying to create a fingerprint of the users behaviour (time between requests, etc) - and comparing this to the sites average fingerprint. - Keep a global request-per-time-count and use this to measure the current load of the site - the mitigation techniques will only be activated if the site is under heavy-load / attack. Any other ideas?

jqueryin 13y ago

I was entirely wrong when I thought that I understood DDoS attacks fairly well. There's actually at least three different types of attacks: "Layer 3 and Layer 4 DDoS attacks are types of volumetric DDoS attacks on a network infrastructure. Layer 3 (network layer) and 4 (transport layer) DDoS attacks rely on extremely high volumes (floods) of data to slow down web server performance, consume bandwidth and eventually degrade access for legitimate users. These attack types typically include ICMP, SYN, and UDP floods." "A Layer 7 DDoS attack is an attack structured to overload specific elements of an application server infrastructure. Layer 7 attacks are especially complex, stealthy, and difficult to detect because they resemble legitimate website traffic. Even simple Layer 7 attacks – for example those targeting login pages with random user IDs and passwords, or repetitive random searches on dynamic websites – can critically overload CPUs and databases. Also, DDoS attackers can randomize or repeatedly change the signatures of a Layer 7 attack, making it more difficult to detect and mitigate." This glossary proved to be quite useful: http://www.prolexic.com/knowledge-center-dos-and-ddos-glossary.html http://www.prolexic.com/knowledge-center-dos-and-ddos-glossa...

justkez 13y ago
It raises interesting questions about how companies like Incapsula will integrate with their customers in the future. Fairly easy for them to sit and monitor the low level networking kit dealing with Layer 3 + 4 attacks, but that's the wrong place to focus on Layer 7, right? They need to understand the web app and infrastructure it touches. I'm sure they exist in some form or another, but it's likely we'll see more design patterns emerging to deal with mitigating application attacks. Combined with the network/protocol control you'd have application logic feeding down suspected attack data to block
- Igalze 13y ago
  You raise an excellent point. We have a lot of application awareness configurations and our signature pool (which is one of the tools we use for traffic profiling) holds over 10M variant, sometime as much as ~20,000 per user-agent type, to cover all scenarios.

level09 13y ago

How would you differentiate the traffic coming form headless browsers from normal users ? with a proper user-agent and javascript enabled, isn't it almost impossible ?

zzzcpan 13y ago
No, it's not impossible. Basically, you detect anomalies, i.e. gather statistical information about your users and their behavior in advance and apply that as a filter during attacks. For example, if you have like 10 new refererless users per hour, i.e. without a cookie and a referrer, but suddenly there are 1000 of them and backend is already slow - you should probably drop their requests. Same with specific referrers and any other information you have about your users. But this is kind of preventative only approach, it won't work if your site is already under attack.
- STRML 13y ago
  The Referer header is just one more thing that a sophisticated attacker will spoof. I think the GP is correct - with enough preparation, a concentrated PhantomJS attack will look exactly like users accessing the website en masse, and it can be very, very difficult to tell who is real and who is not. That is why they use a tool as heavy as PhantomJS, which (from experience) is not very quick compared to more traditional methods. It is because it spoofs a real browser quite completely that it is so good for these methods. For example, some pages might do a trick where they set a cookie, load another page, and redirect / drop based on whether or not that cookie is present in exactly the form it should be present. Maybe they set two, one expires immediately and the other contains some complex data. PhantomJS can handle that easily (as it is just Webkit + QT) but traditional tools do not.
  - zzzcpan 13y ago
    > The Referer header is just one more thing > that a sophisticated attacker will spoof. It doesn't really matter, as attacker wouldn't have your data. For example, you know that on average you have 10 users per hour from some specific referrer and that most of them are coming from a single country. And you know this for every referrer. But attacker doesn't, even if he tries to fake it, his requests are either not going to satisfy the profile (he won't get something right, like country, user-agent, etc) or he will reach the limit (by trying to send more than N requests per hour for specific referrer, as he doesn't know the limit either). Of course you could make it very precise by using a lot more data points for each user: where users usually go after which URL, how long do they stay on each URL, how many requests do they usually make, do they have your cookie, how long, etc. And I'm talking about cookies as a sign of a user, who already visited your website in the past.

al1x 13y ago

I started a thread where I was hoping to get a discussion of DDOS mitigation techniques, tools, and providers. Something that should be useful and highly relevant to web-based small business owners. It's here if anyone's interested -- https://news.ycombinator.com/item?id=6587640 https://news.ycombinator.com/item?id=6587640

k3n 13y ago

Fair warning - I just got pinged from corporate security that this site triggered an alert on our IDS: "the IDS reported the machine accessed a site containing a JavaScript known to contain hidden iFrames and malware redirects". He said it was named "tongji.js". Can't investigate further as I'm still on said network and don't want to re-ping it...