6 ms·

A DDoS in Asia Pacific

89 points by boutcher 11y ago

asdfaoeu 11y ago
200Gbps (if true) seems very high for a non reflection attack.
- nullrouted 11y ago
  A lot of people have been claiming these types of numbers but can't really show the hard proof needed.
  - placeybordeaux 11y ago
    How would hard evidence be provided?
    - toomuchtodo 11y ago
      NetFlow logs would be one way.
- cft 11y ago
  This tsunami TCP SYN attack uses 1000 byte SYN packets apparently. A good countermeasure for these would be rejection of all large SYN packets. Verisign DDoS protection services claim that they can withstand 2Tbps attacks of most types.
  - r1ch 11y ago
    Unfortunately this would break TCP Fast Open, which transmits data with the initial SYN.
    - georgerobinson 11y ago
      Would a client that supports TCP Fast Open then fallback to the standard 3-way handshake once it's SYNs timed out?
    - scurvy 11y ago
      I can tell you that almost no one uses TCP Fast Open. It's a draft RFC that violates other RFCs. Google has given up on it in favor of QUIC. You should give up on it, too. It's not going to happen. It's a bad idea cooked up by ivory tower researchers who have never run a network.
- scurvy 11y ago
  I'd normally say that doesn't seem that high for a botnet or collection of botnets. To put it in perspective, that's only twenty 10gig attached servers. Not that much when you think about it. Sure, you need transit to match the server but that's not uncommon at all these days. The most unusual aspect of this attack was that it was an easily blocked, rudimentary attack using spoofed, big SYNs. Volumetric attacks have subsided and fallen out of favor over the past year. Everything now is layer 7 floods at high rates or low-and-slow to avoid detection. Either way it's mostly layer 7 these days. People I've talked with at Cloudflare and Prolexic have seen the same thing. Also, we saw these big SYN floods about 3 years ago (before Radware coined the term). They are easy to block, the attackers went away, and we haven't really seen any since. I think this is a 3+ year old botnet run by an attacker who hasn't kept up with the times. tl;dr this botnet is a bit long in the tooth
mahranch 11y ago
I knew it would be S.Korea. The company I used to work for, at the time I left, was dealing with some particularly spiteful individuals from S.Korea who have been DDoSing their gaming platform and their separate video host. This was happening off and on for about 12 months. Interestingly enough, each attack was committed by completely different individual and were unrelated. In one attack where the guy was caught (I think they caught all but 2 of the attackers), he claimed he ran the DDoS because he didn't like the fact that there was a Japanese pop music video being hosted on the video site. This wasn't a young kid either, the guy was 33 and had a full time job at some advertising company.
- dylz 11y ago
  This more or less reflects my stance on SK too. I ran a reasonably large APAC/SEA esports-related site for a while, competing sites in SK often attempted to attack it (and outright told me as such, to get out of korea). It's strange and I really have no idea why.
  - cpncrunch 11y ago
    We had a 9Gbps attack from SK earlier in the year. I have no idea why we were targeted, but my best guess is that a user in SK got upset at some user-generated content and decided to DDoS the site rather than report it to us. Weird. Anyway, we decided to move to OVH, and haven't had any problem since. We did get an email about an attack being mitigated a few months ago (which didn't cause any outage at all), and since then the trolls have realised that we can't be DDoSed any more :)
- kijin 11y ago
  There are assholes in every country. It's easier for assholes in some countries to launch DDoS attacks than it is for assholes in other countries. With the one of the world's largest userbases of outdated IE with tons of ActiveX plugins, South Korea sure is a nice place to run a botnet. On the other hand, most of the ISPs mentioned in the article are not Korean, so maybe it's a bit more complicated.
- brobinson 11y ago
kijin 11y ago
According to the founder [1], Telegram was even removed from Play Store for a few hours at the request of a South Korean competitor. For whatever reason, somebody in South Korea is seriously pissed off with Telegram. [1] https://twitter.com/durov/status/619486763032182784 https://twitter.com/durov/status/619486763032182784
- zodiakzz 11y ago
  LINE is a Japanese company IIRC.
  - kijin 11y ago
    It is a subsidiary of Naver, the largest (and politically well-connected) portal in Korea.
- tuananh 11y ago
  he didn't back anything he said. as much as i like Telegram, at least show some proof when you put such strong statement.
  - brobinson 11y ago
    He just replied with more info on his claim: https://twitter.com/durov/status/620538556134653952 https://twitter.com/durov/status/620538556134653952
- cpncrunch 11y ago
  They've been having DDoS attacks since September last year, so it seems unlikely that it's caused by a recent event. I'm surprised they haven't done anything about it before now.
  - kijin 11y ago
    The attack in September occurred just as a large number of Koreans suddenly moved to Telegram. The mass exodus was triggered by a surveillance scandal where a major Korean competitor was found to be handing over a large amount of user data to the government. In recent days, there's been another exodus of Koreans from domestic IM services due to the revelation that the Korean army has been a customer of Hacking Team, the Italian spyware vendor who got hacked last week. The two incidents might be related.
irvinfly 11y ago
China is doing a mass arrestment*1 of 100+ human right lawyers last weekend, in the same times as DDoS start and end, and there's a news from China's official news agent indicate that Telegram is the main secret contacting tool that human right lawyers used. Some people think it's China who attack Telegram, to avoid the lawyers to warning each other for the arrestment. 1) https://www.facebook.com/chrlcg/photos/a.1571958406350448.1073741828.1571955643017391/1634700570076231/?type=1 https://www.facebook.com/chrlcg/photos/a.1571958406350448.10... 2) http://news.xinhuanet.com/politics/2015-07/11/c_128010249.htm http://news.xinhuanet.com/politics/2015-07/11/c_128010249.ht...
- kijin 11y ago
  Interesting. We know from this year's GitHub attack that (a) China is willing to DDoS foreign internet services, and (b) China is able to enlist foreign traffic to carry out the actual attack. So it wouldn't be surprising if China DDoSed Telegram and made it look like the attack came from South Korea.
dbrannan 11y ago
I got hit by two of these (1/27 to Feb 4th and 6/4 to 6/22), and they were relentless. It was difficult to know where the attack originated because many proxies were involved - most inside the USA). We only managed a 62% uptime during the whole affair, many customers were upset, and it really hurt business. We ended up refunding everyone for the month and sending out a huge apology, for which many customers were understanding. Still, it hurt our business dramatically.
linhat 11y ago
This is most interesting... The garbage traffic came from about a hundred thousand infected servers, most noticeably, in LeaseWeb B.V., Hetzner Online AG, PlusServer AG, NFOrce Entertainment BV, Amazon and Comcast networks. That said, the attack was distributed evenly across thousands of hosts and none contributed more than 5% of the total volume. I used to host a lot with Hetzner, and while quite expensive, they mostly responded to these kinds of things very quickly and with a certain level of technical competence (which definitely cannot be said of every hoster). Also, I'm quite surprised to not see OVH in there, as their network has a kind of "reputation" for these things... Fighting back would‘ve been a little easier, if the abuse departments in most of the mentioned companies didn’t process requests 9-5, Mon-Fri only. (Hours more befitting a scuba-diving shop in Vatican.) Business as usual I would say...although I don't scuba-dive... Edit: formatting
- latch 11y ago
  Did you mean to say that they are quite UNexpensive? (because they are)
cpncrunch 11y ago
Simple solution: move to OVH. Although they don't have servers in SE Asia, perhaps 100% uptime is more important than shaving 100ms off the ping time. (As far as I can tell they don't have real-time audio or video anyway).
- nly 11y ago
  What makes you think OVH could cope with a 200 Gbps DoS attack of this nature? A quick look at their services indicates they don't mention what kind of attacks they defend against, and SYN floods are some of the hardest to defend against.
  - cpncrunch 11y ago
    They say they have facilities to clean 480Gbps of data, and 5Tbps of mostly spare inbound bandwidth, so their DDoS mitigation capacity is somewhere in that range (http://www.ovh.com/ca/en/a1171.protection-anti-ddos-service-standard http://www.ovh.com/ca/en/a1171.protection-anti-ddos-service-...).
    - gruez 11y ago
      480Gbps across all their datacenters. Each datacenter only has 160Gbps, and I doubt that they'll devote all of that to one client.
      - cpncrunch 11y ago
        They do activate all 3 datacenters. https://www.ovh.com/ca/en/anti-ddos/hoovering-up.xml https://www.ovh.com/ca/en/anti-ddos/hoovering-up.xml
    - scurvy 11y ago
      Customer testimony on places like Webhosting Talk have cast all of those numbers into serious doubt. OVH is more likely to nullroute your IPs than it is to fight off a 300gig attack.
ised 11y ago
Question: Is this possible because they are using Linux servers? The Linux kernel adopted TCP Fast Open? https://www.ietf.org/mail-archive/web/tcpm/current/msg08204.html https://www.ietf.org/mail-archive/web/tcpm/current/msg08204....
- scurvy 11y ago
  TCP Fast Open is a terrible idea, and RFC 7413 should be marked as historic so that no one actually tries to implement it. Google has given up on it in favor of QUIC and so should everyone else. QUIC allows for "zero RTT" requests that are also signed (preventing spoofing). We started blocking these large requests over 3 years ago when we started seeing them. Interestingly enough, that was a full 6-9 months before Radware wrote an article and coined the term Tsunami SYN. We just called it "big SYN". The attack is trivially easy to stop, and anyone running a client that tries a TCP Fast Open should expect failure frequently.

89 points by boutcher 11y ago

asdfaoeu 11y ago

200Gbps (if true) seems very high for a non reflection attack.

nullrouted 11y ago
A lot of people have been claiming these types of numbers but can't really show the hard proof needed.
- placeybordeaux 11y ago
  How would hard evidence be provided?
  - toomuchtodo 11y ago
    NetFlow logs would be one way.
cft 11y ago
This tsunami TCP SYN attack uses 1000 byte SYN packets apparently. A good countermeasure for these would be rejection of all large SYN packets. Verisign DDoS protection services claim that they can withstand 2Tbps attacks of most types.
- r1ch 11y ago
  Unfortunately this would break TCP Fast Open, which transmits data with the initial SYN.
  - georgerobinson 11y ago
    Would a client that supports TCP Fast Open then fallback to the standard 3-way handshake once it's SYNs timed out?
  - scurvy 11y ago
    I can tell you that almost no one uses TCP Fast Open. It's a draft RFC that violates other RFCs. Google has given up on it in favor of QUIC. You should give up on it, too. It's not going to happen. It's a bad idea cooked up by ivory tower researchers who have never run a network.
scurvy 11y ago
I'd normally say that doesn't seem that high for a botnet or collection of botnets. To put it in perspective, that's only twenty 10gig attached servers. Not that much when you think about it. Sure, you need transit to match the server but that's not uncommon at all these days. The most unusual aspect of this attack was that it was an easily blocked, rudimentary attack using spoofed, big SYNs. Volumetric attacks have subsided and fallen out of favor over the past year. Everything now is layer 7 floods at high rates or low-and-slow to avoid detection. Either way it's mostly layer 7 these days. People I've talked with at Cloudflare and Prolexic have seen the same thing. Also, we saw these big SYN floods about 3 years ago (before Radware coined the term). They are easy to block, the attackers went away, and we haven't really seen any since. I think this is a 3+ year old botnet run by an attacker who hasn't kept up with the times. tl;dr this botnet is a bit long in the tooth

mahranch 11y ago

I knew it would be S.Korea. The company I used to work for, at the time I left, was dealing with some particularly spiteful individuals from S.Korea who have been DDoSing their gaming platform and their separate video host. This was happening off and on for about 12 months. Interestingly enough, each attack was committed by completely different individual and were unrelated. In one attack where the guy was caught (I think they caught all but 2 of the attackers), he claimed he ran the DDoS because he didn't like the fact that there was a Japanese pop music video being hosted on the video site. This wasn't a young kid either, the guy was 33 and had a full time job at some advertising company.

dylz 11y ago
This more or less reflects my stance on SK too. I ran a reasonably large APAC/SEA esports-related site for a while, competing sites in SK often attempted to attack it (and outright told me as such, to get out of korea). It's strange and I really have no idea why.
- cpncrunch 11y ago
  We had a 9Gbps attack from SK earlier in the year. I have no idea why we were targeted, but my best guess is that a user in SK got upset at some user-generated content and decided to DDoS the site rather than report it to us. Weird. Anyway, we decided to move to OVH, and haven't had any problem since. We did get an email about an attack being mitigated a few months ago (which didn't cause any outage at all), and since then the trolls have realised that we can't be DDoSed any more :)
kijin 11y ago
There are assholes in every country. It's easier for assholes in some countries to launch DDoS attacks than it is for assholes in other countries. With the one of the world's largest userbases of outdated IE with tons of ActiveX plugins, South Korea sure is a nice place to run a botnet. On the other hand, most of the ISPs mentioned in the article are not Korean, so maybe it's a bit more complicated.
brobinson 11y ago

kijin 11y ago

According to the founder [1], Telegram was even removed from Play Store for a few hours at the request of a South Korean competitor. For whatever reason, somebody in South Korea is seriously pissed off with Telegram. [1] https://twitter.com/durov/status/619486763032182784 https://twitter.com/durov/status/619486763032182784

zodiakzz 11y ago
LINE is a Japanese company IIRC.
- kijin 11y ago
  It is a subsidiary of Naver, the largest (and politically well-connected) portal in Korea.
tuananh 11y ago
he didn't back anything he said. as much as i like Telegram, at least show some proof when you put such strong statement.
- brobinson 11y ago
  He just replied with more info on his claim: https://twitter.com/durov/status/620538556134653952 https://twitter.com/durov/status/620538556134653952
cpncrunch 11y ago
They've been having DDoS attacks since September last year, so it seems unlikely that it's caused by a recent event. I'm surprised they haven't done anything about it before now.
- kijin 11y ago
  The attack in September occurred just as a large number of Koreans suddenly moved to Telegram. The mass exodus was triggered by a surveillance scandal where a major Korean competitor was found to be handing over a large amount of user data to the government. In recent days, there's been another exodus of Koreans from domestic IM services due to the revelation that the Korean army has been a customer of Hacking Team, the Italian spyware vendor who got hacked last week. The two incidents might be related.

irvinfly 11y ago

China is doing a mass arrestment*1 of 100+ human right lawyers last weekend, in the same times as DDoS start and end, and there's a news from China's official news agent indicate that Telegram is the main secret contacting tool that human right lawyers used. Some people think it's China who attack Telegram, to avoid the lawyers to warning each other for the arrestment. 1) https://www.facebook.com/chrlcg/photos/a.1571958406350448.1073741828.1571955643017391/1634700570076231/?type=1 https://www.facebook.com/chrlcg/photos/a.1571958406350448.10... 2) http://news.xinhuanet.com/politics/2015-07/11/c_128010249.htm http://news.xinhuanet.com/politics/2015-07/11/c_128010249.ht...

kijin 11y ago
Interesting. We know from this year's GitHub attack that (a) China is willing to DDoS foreign internet services, and (b) China is able to enlist foreign traffic to carry out the actual attack. So it wouldn't be surprising if China DDoSed Telegram and made it look like the attack came from South Korea.

dbrannan 11y ago

I got hit by two of these (1/27 to Feb 4th and 6/4 to 6/22), and they were relentless. It was difficult to know where the attack originated because many proxies were involved - most inside the USA). We only managed a 62% uptime during the whole affair, many customers were upset, and it really hurt business. We ended up refunding everyone for the month and sending out a huge apology, for which many customers were understanding. Still, it hurt our business dramatically.

linhat 11y ago

This is most interesting... The garbage traffic came from about a hundred thousand infected servers, most noticeably, in LeaseWeb B.V., Hetzner Online AG, PlusServer AG, NFOrce Entertainment BV, Amazon and Comcast networks. That said, the attack was distributed evenly across thousands of hosts and none contributed more than 5% of the total volume. I used to host a lot with Hetzner, and while quite expensive, they mostly responded to these kinds of things very quickly and with a certain level of technical competence (which definitely cannot be said of every hoster). Also, I'm quite surprised to not see OVH in there, as their network has a kind of "reputation" for these things... Fighting back would‘ve been a little easier, if the abuse departments in most of the mentioned companies didn’t process requests 9-5, Mon-Fri only. (Hours more befitting a scuba-diving shop in Vatican.) Business as usual I would say...although I don't scuba-dive... Edit: formatting

latch 11y ago
Did you mean to say that they are quite UNexpensive? (because they are)

cpncrunch 11y ago

Simple solution: move to OVH. Although they don't have servers in SE Asia, perhaps 100% uptime is more important than shaving 100ms off the ping time. (As far as I can tell they don't have real-time audio or video anyway).

nly 11y ago
What makes you think OVH could cope with a 200 Gbps DoS attack of this nature? A quick look at their services indicates they don't mention what kind of attacks they defend against, and SYN floods are some of the hardest to defend against.
- cpncrunch 11y ago
  They say they have facilities to clean 480Gbps of data, and 5Tbps of mostly spare inbound bandwidth, so their DDoS mitigation capacity is somewhere in that range (http://www.ovh.com/ca/en/a1171.protection-anti-ddos-service-standard http://www.ovh.com/ca/en/a1171.protection-anti-ddos-service-...).
  - gruez 11y ago
    480Gbps across all their datacenters. Each datacenter only has 160Gbps, and I doubt that they'll devote all of that to one client.
    - cpncrunch 11y ago
      They do activate all 3 datacenters. https://www.ovh.com/ca/en/anti-ddos/hoovering-up.xml https://www.ovh.com/ca/en/anti-ddos/hoovering-up.xml
  - scurvy 11y ago
    Customer testimony on places like Webhosting Talk have cast all of those numbers into serious doubt. OVH is more likely to nullroute your IPs than it is to fight off a 300gig attack.

ised 11y ago

Question: Is this possible because they are using Linux servers? The Linux kernel adopted TCP Fast Open? https://www.ietf.org/mail-archive/web/tcpm/current/msg08204.html https://www.ietf.org/mail-archive/web/tcpm/current/msg08204....

scurvy 11y ago
TCP Fast Open is a terrible idea, and RFC 7413 should be marked as historic so that no one actually tries to implement it. Google has given up on it in favor of QUIC and so should everyone else. QUIC allows for "zero RTT" requests that are also signed (preventing spoofing). We started blocking these large requests over 3 years ago when we started seeing them. Interestingly enough, that was a full 6-9 months before Radware wrote an article and coined the term Tsunami SYN. We just called it "big SYN". The attack is trivially easy to stop, and anyone running a client that tries a TCP Fast Open should expect failure frequently.