6 ms·
Second large Hetzner outage in a week caused by DDoS attack
- walrus01 4y ago[spiderman-pointing-at-spiderman.gif] seriously, aren't they commonly the SOURCE of many DoS attacks... any hosting provider where some random person on the internet and $5 of credit on a prepaid visa card will have this problem.
- MrStonedOne 4y agoThere is also the annoying confusion that some attacks involve spoofing the victim's ip to other hosts so the reply goes to the victim while masking the attacker's ip(s).
- missedthecue 4y agoHetzner requires government ID to open an account
- Dma54rhs 4y agoSince when? I've never provided them an ID and neither have they asked later.
- missedthecue 4y agoAt least 3-4 years. That's when I signed up.
- xmpir 4y agoLast time it took about 9 hours: https://status.hetzner.com/incident/129728ce-ba25-49b6-96cc-aafcd39ab0b7 https://status.hetzner.com/incident/129728ce-ba25-49b6-96cc-...
- _-david-_ 4y ago>This concerns UDP traffic on port 9000-65535. Does anybody know what usually runs on those ports?
- rozenmd 4y agoOnline games (MMOs, shooters, etc) come to mind
- baisq 4y agoMMOs over UDP?
- Retr0id 4y agoAbsolutely
- sodality2 4y agoThat's the preferred protocol for ultra-real-time games because a few ms ago is not helpful information to spend time recovering. A sufficiently fast-moving MMO could apply
- baisq 4y agoWhat MMOs use UDP? Asking sincerely because I have never seen one.
- koolba 4y agoAnything with real time communications like an FPS would use UDP as stale action data is mostly useless. The latest state of is all that matters. Most such games will either layer their own streaming channel atop UDP for guaranteed ordered delivery of important messages or use a separate TCP socket as well.
- xnyanta 4y agoYou must not be looking very hard, pretty much every game engine uses UDP as the network transport. There are some notable exceptions like Java Minecraft.
- ricardobeat 4y agoAt their size, don’t they have some kind of hardware-level packet filtering ability like cloudflare to protect against these attacks?
- xmpir 4y agoThey state using hardware DDoS protection but it seems not to be sufficient: https://www.hetzner.com/unternehmen/ddos-schutz https://www.hetzner.com/unternehmen/ddos-schutz
- rstupek 4y agoNot if the level of incoming bandwidth exceeds the available bandwidth of the circuits involved.. you can't filter it when the link is saturated. Cloudflare uses other techniques like global distribution so aggregate bandwidth is higher than the attack bandwidth
- pigtailgirl 4y agodoes anyone else have a network that can do what Cloudflare can do? seems like magic sometimes.
- seunosewa 4y agoPerhaps OVH?
- viraptor 4y agoYes, there's a few distributed DDoS protection services. For example Fastly, Akamai, GcoreLabs, and a few smaller ones. They're mostly less evil too as a bonus.
- smartbit 4y agoThe Netherlands has NaWas non-profit service that filters out DDOS attacks, in Q1’22 7,4 times per days with DDOS traffic up to 300Gbps. It’s a few man shop, costs of membership are low. From their FAQ https://www.nbip.nl/en/nawas/faq/ https://www.nbip.nl/en/nawas/faq/ : The NaWas infrastructure is designed as an on-demand service. After detecting an attack, the traffic is routed via BGP to the NaWas hardware and then the mitigation process starts. All traffic is then rerouted and the own connections can thus manage with less capacity and thus remain cheaper. To connect to the NaWas, a port must be available from one of the following parties: AMS-IX, NL-IX, LINX, NET-IX, Top-IX, M-IX, V-IX or one of these cloud interconnects DCSPine, Epsilon, Megaport.
- xmpir 4y agoI am wondering what the attacker's intent is
- belter 4y agoRetribution :-)
- unnouinceput 4y agoMaybe, just maybe, rely less on embedded framework on embedded framework that spit JavaScript that gets 95% unused. If for a simple outage apology page the output was 1.7MB, I can only imagine for their normal pages how much it is. At this size I feel only like 10k legit users would unwillingly do the outage anyway. But hey, Kubernetes and Node.js is all the rage nowadays.
- danuker 4y ago"I have only made this letter longer because I have not had the time to make it shorter." - Blaise Pascal
- tempnow987 4y agoI thought OVH and Hetzner were the source of a ton of these DDoS attacks. Their IP ranges always seem to be in abuse logs. Cloudflare write in a recent attack: The top networks included the German provider Hetzner Online GmbH (Autonomous System Number 24940), Azteca Comunicaciones Colombia (ASN 262186), OVH in France (ASN 16276), as well as other cloud providers. https://blog.cloudflare.com/15m-rps-ddos-attack/ https://blog.cloudflare.com/15m-rps-ddos-attack/
- CircleSpokes 4y agoI mean that makes sense no? Attacks like that rely on compromised servers so it shouldn't be a big surprise large hosting provides are among the biggest attackers. Other large ISPs like digital ocean and Alibaba are among the top attackers in that attack also. I assume this attack is UDP based unlike the one you linked too.
- onphonenow 4y agoWhere are the AWS and GCP ranges then? They aren't even in the top 10 here. Its the claim hetzner is larger than AWS? I find that highly unlikely.
- manishsharan 4y agoWe probably don't see AWS or Azure as source if these DDOS attacks because of egress costs.
- viraptor 4y agoPeople aren't paying with their own money for DDoS machines normally. Well, maybe there are some small operations like that. But often the traffic comes from a hacked service that's a part of a bigger botnet. It may lead to a larger detection ratio on AWS/GCP, but the attackers are not paying the costs.
- vladvasiliu 4y ago
- ffhhj 4y agoExcuse the ignorance, but couldn't ISPs block the attacks?
- vardagsnyttt 4y agoThat would make sense, but its hard: - You need to identify the traffic to be filtered and the post states: "Due to always different destinations (IPs, ports, packet size) (..)" - You need to maintain some agreement with a large number of ISPs - You need to maintain some gossiping infrastructure to these ISPs - ISPs may not care about your DDoS attack
- mike_d 4y agoYes, network operators (should) participate in centralized black hole services like UTRS[1]. If you can identify the specific IPs that are under attack you make a BGP announcement to other participating networks asking them to drop traffic to that IP within their networks. As a participant you can avoid paying to send outbound attack traffic, and also identify attack sources within your own network. 1. https://team-cymru.com/community-services/utrs/ https://team-cymru.com/community-services/utrs/
- davidtinker 4y agoAnyone know if it is possible to mitigate the impact of Hetzner blocking UDP traffic on port 9000+? These outages whacked our Kubernetes clusters (Calico + vxlan + Wireguard). https://serverfault.com/questions/1100482/how-to-limit-udp-port-range-with-k8s-calico-wireguard https://serverfault.com/questions/1100482/how-to-limit-udp-p...