4 ms·

I understand the first part -- sending requests with no host header to a spam log (or even better, don't log). What I don't understand is the second part -- bl

I understand the first part -- sending requests with no host header to a spam log (or even better, don't log). What I don't understand is the second part -- blocking those hosts. Seems pointless now that you've de-noised your logs. They're still sending packets. Saves thousands of bytes on outbound? What about all the scan-spam on sites WITH host headers? Whatevs.

yabones 4y ago
Serves a few purposes, but as you said the main objective is already done by de-noising. The other reason I do this is because it's easy to detect that kind of scanning in HTTP logs, but not as easy for other services (ssh, ftp, smtpd, etc) without something like fail2ban, and the blanket ban applies to all of them. So, if a bot scans your HTTP server enough times, they can't go after "softer" targets later. For scan-spam that does hit your "real" site, it's a bit more tricky as there absolutely will be false positives. You can grep for all 401/403's and add them to the list, but that will sooner or later hit a real user. So it's much more specific to the application you're hosting, where this works for just about any site. The other nice thing is that even when they scan your "real" site, they'll often hit the default host via IP scans at the same time, so you can still manage to ban them. It's not perfect, but it's good enough :)
- dpifke 4y ago
  Running your own mail server is a great source of data to identify botnet-compromised hosts. When I started banning IPs that send "HELO <myhostname>" for 24 hours, I cut the number of fake login/registration attempts on a bunch of my web-based projects by ~50%. It works the other way, too. Temporary bans on hosts that try to access /wp-admin (I don't run Wordpress anywhere) cut my email spam significantly. (Some day, I'll get around to implementing a real reputation tracking system, with exponential ban lengths.)
  - bombcar 4y ago
    This is an important aspect of it - you can use information on one angle of attack to protect other devices. Do note that doing this kind of thing can block people on Tor, because Tor is used for attacks quite often, also.

yabones 4y ago

Serves a few purposes, but as you said the main objective is already done by de-noising. The other reason I do this is because it's easy to detect that kind of scanning in HTTP logs, but not as easy for other services (ssh, ftp, smtpd, etc) without something like fail2ban, and the blanket ban applies to all of them. So, if a bot scans your HTTP server enough times, they can't go after "softer" targets later. For scan-spam that does hit your "real" site, it's a bit more tricky as there absolutely will be false positives. You can grep for all 401/403's and add them to the list, but that will sooner or later hit a real user. So it's much more specific to the application you're hosting, where this works for just about any site. The other nice thing is that even when they scan your "real" site, they'll often hit the default host via IP scans at the same time, so you can still manage to ban them. It's not perfect, but it's good enough :)

dpifke 4y ago
Running your own mail server is a great source of data to identify botnet-compromised hosts. When I started banning IPs that send "HELO <myhostname>" for 24 hours, I cut the number of fake login/registration attempts on a bunch of my web-based projects by ~50%. It works the other way, too. Temporary bans on hosts that try to access /wp-admin (I don't run Wordpress anywhere) cut my email spam significantly. (Some day, I'll get around to implementing a real reputation tracking system, with exponential ban lengths.)
- bombcar 4y ago
  This is an important aspect of it - you can use information on one angle of attack to protect other devices. Do note that doing this kind of thing can block people on Tor, because Tor is used for attacks quite often, also.