8 ms·

The Windows servers that have been fueling DDoSes for months

112 points by kungfudoi 4y ago

userbinator 4y ago
I bet they'd notice and shut them off pretty quickly if bandwidth wasn't nearly free.
- bombcar 4y ago
  I’m wondering about these religious organizations that have 10-20Gb/s pipes.
  - pixl97 4y ago
    Some of the first high bandwidth customers that did video streaming I had to deal with were churches that were casting their sermons. While most people think of non profit organizations as poor that can be a mistake as many are very effective at collecting donations.
  - nijave 4y ago
    Sometimes non profits can get special deals. They can apply for tax exemptions (in the U.S.)
snvzz 4y ago
It'd be sensible to the ISP to just disconnect these from the network, particularly if they can't get their shit together after a warning.
- lucb1e 4y ago
  I keep thinking the same. It's the best option short of just targeting it against themselves and causing just enough nuisance that they either firewall, upgrade, or correctly configure these ancient protocols. For example for the protocol that this article is about: "Windows 2000 Server requires UDP" https://wiki.wireshark.org/MS-CLDAP https://wiki.wireshark.org/MS-CLDAP. Anything newer than Win2k means you can already turn it off.
- iforgotpassword 4y ago
  To me it's crazy that a religious organization apparently has a 17gbits+ connection to the internet. Admittedly I'm not an expert and only know of one that I'd consider large enough here, but yeah... Second question is why their ISP never had any alarm bells going off with such a traffic pattern.
gmuslera 4y ago
Am I the only disturbed by the amount of times the article used the word torrent for huge amount of traffic?
- lucb1e 4y ago
  Trips me up as well, perhaps because I'm a non-native speaker. The mention of torrent in a networking-related context will immediately make me think of the bittorrent protocol.
  - pixl97 4y ago
    In the days before we dealt with bufferbloat well using torrents was a great way to DOS one's self, especially if you had async internet service with a much slower upload or a router with terrible NAT that overflowed when a lot of IPs were in use. A torrent torrent would flood your equipment.
codedokode 4y ago
That article is completely wrong. Misconfigured servers that amplify traffic are not a problem at all. Blacklisting all of them would take no morethan several hours. The problem is that there is no protocol for rejecting traffic. That's why DDOS is so easy and successful. If you are a website you cannot send a message to uplink saying "I don't want traffic from this network". So small and large ISPs do nothing to help protect victims and make it easy for criminals to extort money.
- lucb1e 4y ago
  This sort of thing is being done as far as I know, but I'm not aware how big a customer you have to be for your ISP to care about such notices. Perhaps so large that you've nearly got enough capacity to handle the traffic yourself anyway. It also creates a nice DoS vector: if I want to not take an exam today, I'll just do a little ping flood from the school network and get banned for a while. Similar to account locking on 3 invalid attempts, except now you can block everyone at once as soon as you're in the network or on the same ISP.
  - somehnguy 4y ago
    As a small customer I’ve experienced them doing the complete opposite. I’ve had a 1u collocated at one center or another for the last ~15 years and occasionally we would be the target of a DDoS. The data center would just blackhole us until things calmed down. It’s better than a surprise bandwidth bill but certainly not great.
    - toast0 4y ago
      FWIW, that's what happens for large customers too, from my past experience. Someone aims the DDoS cannon at one of our webservers [1], and if the traffic is above the threshold, that server's IP gets blackholed for a while. As a large customer, you do get to have some input into the thresholds, and how long the blackholes are, and to definitely have traffic blackholed rather than put through some crazy filtering apparatus that adds so much latency it'd be better to drop everything. OTOH, we always had at least 4 webservers, so if just one got blackholed, no big deal, update DNS so most of the real users make it to the available servers, most of the abusers will keep hitting the blackholed server and not move on. If your abusers aren't run of the mill script kiddies using a rent-a-ddos service, it's more difficult. [1] it's almost always the webservers, not the servers that do real work
lucb1e 4y ago
TL;DR of Ars: Connectionless LDAP Actual article that Ars writes about: https://blog.lumen.com/cldap-reflectors-on-the-rise-despite-best-practice/ https://blog.lumen.com/cldap-reflectors-on-the-rise-despite-... Actual news (less vague title than Ars'): the number of vulnerable systems has increased ("from the 7K range to over 12K" "over the last 12 months"). Only 15% stays online longer than a year, though, so it seems that most admins catch on after a while. "[T]he Connectionless Lightweight Directory Access Protocol (CLDAP) [has a] Bandwidth Amplification Factor (BAF) of 56 to 70x". Actionable advice for Windows Server owners is provided at the bottom of both articles. --- > [heading] A never-ending arms race Are there still protocols newly put into production that do not take reflection attacks into account? In my experience, it's all legacy. And don't more and more ISPs enforce BCP38 (http://bcp38.info http://bcp38.info)? Not sure this headline of Ars' is true.
jiggawatts 4y ago
Wait... so there are thousands of Windows Domain Controllers just... "on" the Internet!? What are these people thinking?
- hulitu 4y ago
  Everybody is on "internet" now. These people are not thinking. Just like it is normal for your average phone user to have wifi, cellular data, bluetooth and location always turned on, so it is normal for these admins to have their DC on the internet.
  - jiggawatts 4y ago
    I've managed Windows servers since the NT4 era, and it would never occur to me to make a public-routable domain controller. If the requirement came up, I would go to the n-th degree to make it "Internet safe", but how could you guarantee something like that!? It has dozens of protocols enabled, most of which were designed for private networks in the 1990s! Microsoft has just recently made SMB safe for the Internet (Azure File Shares), but that's just one of many protocols on a Domain Controller...
    - hansel_der 4y ago
      i kind of share your sentiment about putting windows machines on the internet, but otoh ther are a lot of companies doing it and azure ad is almost dominant. > Microsoft has just recently made SMB safe for the Internet cifs anyone?
      - jiggawatts 4y ago
        You're mixing up your products. Azure Active Directory has almost nothing in common with Active Directory, other than the name. Amongst other things, it uses HTTPS almost exclusively. CIFS is an older protocol that Linux admins use incorrectly to refer to SMB, which is the file sharing protocol used by Windows.
jgaa 4y ago
Why are people running Windows in the first place? On a server it's just crazy. Microsoft has thoroughly proven that they have no clue how to make a mail server, remote desktop server or network services that can be safely connected to a public network. On a PC it's also a privacy nightmare.
- prmoustache 4y ago
  Active Directory, Exchange and to a lower level Citrix are the answer. Very few businesses think they can do without those. IT history of these 30 last years have kind of segregated sysadmin/architects types. The one focused on internal IT / desktop services solutions are typically more knowledgeable in Microsoft technologies while unix/linux sysadmins usually focus on the non desktop/office related stuff. Despite decent solutions existing without Microsoft, very few people know enough about them to recommend them.
  - jve 4y ago
    What is your AD replacement in Linux env?
    - hansel_der 4y ago
      samba?
    - prmoustache 4y ago
      Only talking about on-premises tools: Samba (if you need GPO supports for windows desktops), 389 Server, redhat directory server, openldap can work with Keycloak to provide saml2/openid authentication for individual applications. Most emails clients support ldap for contacts with additional carddav server to sync personnal contacts accross devices, Sogo groupware works well for that , add a fairly decent webmail interface and has an outlook connector for easy configuration of microsoft desktops but if you need office 365 functionnalities, Nextcloud would be the way to go imho. It is available as a service too and is cheaper than Microsoft 365. I am not sure how an on premise Nextcloud would scale to huge companies (but enteprise offering say it scales to hundreds of millions of users) but for small businesses it is easy to set up.

112 points by kungfudoi 4y ago

userbinator 4y ago

I bet they'd notice and shut them off pretty quickly if bandwidth wasn't nearly free.

bombcar 4y ago
I’m wondering about these religious organizations that have 10-20Gb/s pipes.
- pixl97 4y ago
  Some of the first high bandwidth customers that did video streaming I had to deal with were churches that were casting their sermons. While most people think of non profit organizations as poor that can be a mistake as many are very effective at collecting donations.
- nijave 4y ago
  Sometimes non profits can get special deals. They can apply for tax exemptions (in the U.S.)

snvzz 4y ago

It'd be sensible to the ISP to just disconnect these from the network, particularly if they can't get their shit together after a warning.

lucb1e 4y ago
I keep thinking the same. It's the best option short of just targeting it against themselves and causing just enough nuisance that they either firewall, upgrade, or correctly configure these ancient protocols. For example for the protocol that this article is about: "Windows 2000 Server requires UDP" https://wiki.wireshark.org/MS-CLDAP https://wiki.wireshark.org/MS-CLDAP. Anything newer than Win2k means you can already turn it off.
iforgotpassword 4y ago
To me it's crazy that a religious organization apparently has a 17gbits+ connection to the internet. Admittedly I'm not an expert and only know of one that I'd consider large enough here, but yeah... Second question is why their ISP never had any alarm bells going off with such a traffic pattern.

gmuslera 4y ago

Am I the only disturbed by the amount of times the article used the word torrent for huge amount of traffic?

lucb1e 4y ago
Trips me up as well, perhaps because I'm a non-native speaker. The mention of torrent in a networking-related context will immediately make me think of the bittorrent protocol.
- pixl97 4y ago
  In the days before we dealt with bufferbloat well using torrents was a great way to DOS one's self, especially if you had async internet service with a much slower upload or a router with terrible NAT that overflowed when a lot of IPs were in use. A torrent torrent would flood your equipment.

codedokode 4y ago

That article is completely wrong. Misconfigured servers that amplify traffic are not a problem at all. Blacklisting all of them would take no morethan several hours. The problem is that there is no protocol for rejecting traffic. That's why DDOS is so easy and successful. If you are a website you cannot send a message to uplink saying "I don't want traffic from this network". So small and large ISPs do nothing to help protect victims and make it easy for criminals to extort money.

lucb1e 4y ago
This sort of thing is being done as far as I know, but I'm not aware how big a customer you have to be for your ISP to care about such notices. Perhaps so large that you've nearly got enough capacity to handle the traffic yourself anyway. It also creates a nice DoS vector: if I want to not take an exam today, I'll just do a little ping flood from the school network and get banned for a while. Similar to account locking on 3 invalid attempts, except now you can block everyone at once as soon as you're in the network or on the same ISP.
- somehnguy 4y ago
  As a small customer I’ve experienced them doing the complete opposite. I’ve had a 1u collocated at one center or another for the last ~15 years and occasionally we would be the target of a DDoS. The data center would just blackhole us until things calmed down. It’s better than a surprise bandwidth bill but certainly not great.
  - toast0 4y ago
    FWIW, that's what happens for large customers too, from my past experience. Someone aims the DDoS cannon at one of our webservers [1], and if the traffic is above the threshold, that server's IP gets blackholed for a while. As a large customer, you do get to have some input into the thresholds, and how long the blackholes are, and to definitely have traffic blackholed rather than put through some crazy filtering apparatus that adds so much latency it'd be better to drop everything. OTOH, we always had at least 4 webservers, so if just one got blackholed, no big deal, update DNS so most of the real users make it to the available servers, most of the abusers will keep hitting the blackholed server and not move on. If your abusers aren't run of the mill script kiddies using a rent-a-ddos service, it's more difficult. [1] it's almost always the webservers, not the servers that do real work

lucb1e 4y ago

TL;DR of Ars: Connectionless LDAP Actual article that Ars writes about: https://blog.lumen.com/cldap-reflectors-on-the-rise-despite-best-practice/ https://blog.lumen.com/cldap-reflectors-on-the-rise-despite-... Actual news (less vague title than Ars'): the number of vulnerable systems has increased ("from the 7K range to over 12K" "over the last 12 months"). Only 15% stays online longer than a year, though, so it seems that most admins catch on after a while. "[T]he Connectionless Lightweight Directory Access Protocol (CLDAP) [has a] Bandwidth Amplification Factor (BAF) of 56 to 70x". Actionable advice for Windows Server owners is provided at the bottom of both articles. --- > [heading] A never-ending arms race Are there still protocols newly put into production that do not take reflection attacks into account? In my experience, it's all legacy. And don't more and more ISPs enforce BCP38 (http://bcp38.info http://bcp38.info)? Not sure this headline of Ars' is true.

jiggawatts 4y ago

Wait... so there are thousands of Windows Domain Controllers just... "on" the Internet!? What are these people thinking?

hulitu 4y ago
Everybody is on "internet" now. These people are not thinking. Just like it is normal for your average phone user to have wifi, cellular data, bluetooth and location always turned on, so it is normal for these admins to have their DC on the internet.
- jiggawatts 4y ago
  I've managed Windows servers since the NT4 era, and it would never occur to me to make a public-routable domain controller. If the requirement came up, I would go to the n-th degree to make it "Internet safe", but how could you guarantee something like that!? It has dozens of protocols enabled, most of which were designed for private networks in the 1990s! Microsoft has just recently made SMB safe for the Internet (Azure File Shares), but that's just one of many protocols on a Domain Controller...
  - hansel_der 4y ago
    i kind of share your sentiment about putting windows machines on the internet, but otoh ther are a lot of companies doing it and azure ad is almost dominant. > Microsoft has just recently made SMB safe for the Internet cifs anyone?
    - jiggawatts 4y ago
      You're mixing up your products. Azure Active Directory has almost nothing in common with Active Directory, other than the name. Amongst other things, it uses HTTPS almost exclusively. CIFS is an older protocol that Linux admins use incorrectly to refer to SMB, which is the file sharing protocol used by Windows.

jgaa 4y ago

Why are people running Windows in the first place? On a server it's just crazy. Microsoft has thoroughly proven that they have no clue how to make a mail server, remote desktop server or network services that can be safely connected to a public network. On a PC it's also a privacy nightmare.

prmoustache 4y ago
Active Directory, Exchange and to a lower level Citrix are the answer. Very few businesses think they can do without those. IT history of these 30 last years have kind of segregated sysadmin/architects types. The one focused on internal IT / desktop services solutions are typically more knowledgeable in Microsoft technologies while unix/linux sysadmins usually focus on the non desktop/office related stuff. Despite decent solutions existing without Microsoft, very few people know enough about them to recommend them.
- jve 4y ago
  What is your AD replacement in Linux env?
  - hansel_der 4y ago
    samba?
  - prmoustache 4y ago
    Only talking about on-premises tools: Samba (if you need GPO supports for windows desktops), 389 Server, redhat directory server, openldap can work with Keycloak to provide saml2/openid authentication for individual applications. Most emails clients support ldap for contacts with additional carddav server to sync personnal contacts accross devices, Sogo groupware works well for that , add a fairly decent webmail interface and has an outlook connector for easy configuration of microsoft desktops but if you need office 365 functionnalities, Nextcloud would be the way to go imho. It is available as a service too and is cheaper than Microsoft 365. I am not sure how an on premise Nextcloud would scale to huge companies (but enteprise offering say it scales to hundreds of millions of users) but for small businesses it is easy to set up.