6 ms·
Disclaimer: VideoLAN president and lead VLC developer here. The attack started on our new mirroring system (powered by mirrorbrain) 2.5 days ago, during the ni
by jbk 14y ago
Disclaimer: VideoLAN president and lead VLC developer here.
The attack started on our new mirroring system (powered by mirrorbrain) 2.5 days ago, during the night (after 2am, so we were sleeping).
We were woken up (OP and I) in the morning by many mirrors complaining of high bandwidth use. The actual number of requests was not that high (400 req/s), but the botnet was downloading the whole vlc.exe, aka 22MB. So, we were at around 70Gbps during the night, in average.
Afterwards, North America got up, and things got worse. We had up to 1660 req/s, so around 292 Gbps...
This is very weird for a DDoS, to be honest.
Our front machine that splits the down mirrors was taking most of the load, and we were able to find the patterns to drop the botnet connexions, in order to not kill too much our mirrors. I won't discuss too much of the patterns, as you can imagine, but as usual, I'll be happy to discuss it IRL or by mail.
Tweaking the front server was also important to reduce the number of open connexions, to not kill our server.
2.5 days after, the attack is still going on, with an average of 500req/s.
The video was done using logstalgia, using scripts of OP, on my machine (<troll>he was running eclipse, he couldn't do both at the same time :)</troll>).
- subsystem 14y ago"This is very weird for a DDoS, to be honest." Maybe it was aimed at bandwidth costs?
- jbk 14y agoMaybe. Seriously, no idea.
- drivebyacct2 14y agoThat's precisely what I don't get in this case, especially. The source is out there, there are other people mirroring vlc. So if you're trying to censor it, you're going to fail pretty hard. In the absence of that, what can it be? Of course! It must be the mplayer folks!!
- kin3tic 14y agoOr in this case, who you'd literally least suspect: wmplayer. All hail wmv!
- vabmit 14y agoI don't know what request throttling solutions are available for nginx. But, for apache there are modules and scripts/code that can rate limit traffic and requests from specific IPs. Depending on the size of the botnet attacking, rate limiting requests by IP or network may have been a way to mitigate the DDoS effects. Again, depending on the specifics, you should be able to use iptables (if you're running Linux) to limit requests per source IP or IP range. The type of actions you can take with iptables are really quite robust. I've had to deploy request rate limiting solutions in the past because of well meaning federated search mash-ups that included content from servers that I run. Filtering by useragent seems to be a temporary solution. I think that's especially true since you've disclosed publicly that's what you're doing.
- daemon13 14y agoWith nginx you can easily rate-limit connections, requests and downloads (which you can throttle also). Can also do this by GeoIP - country, etc. No need for extra steps - this module shall be compiled and part of nginx-full package from nginx official PPA, if OP is using Ubuntu.
- to3m 14y ago"Disclaimer"?
- newman314 14y agoWhat did you tweak in nginx?
- huhtenberg 14y agoWho toes have you stepped on recently, I wonder? Also, pattern-wise - most of the traffic in the video seems to be hitting a single .asc file.
- jbk 14y agoNo idea, seriously. VLC scammers, mainly. The .asc is a logstalgia visualisation bug, because in fact, it merges the few .exe.asc requests with the massive .exe requests
- rb12345 14y agoI had problems with the Windows updater apparently downloading version 2.0.6 and then not installing it. I ended up downloading and installing it manually instead. Could that be related somehow?
- daemon13 14y agoIs this originating from specific country or all over?
- jbk 14y agoAll over.
- derefr 14y agoIs there a reason VLC doesn't offer a Bittorrent download option, by the way? As far as I know VLC's audience is pretty technologically-inclined, so they would understand what a magnet link is, and you'd be able to just switch off the archive-download option for a while in cases like this. (As a bonus, if you embed a magnet link in your page, you're implicitly embedding a SHA-verification of the resulting torrent file.)
- merlincorey 14y agoWhy do you return 403 (according to the article) instead of 444? 444 will drop and close the connection, so I am guessing it has something to do with keep alives vs connection overhead?