5 ms·

> what i'm saying is, if you're fucked, you're fucked. fail2ban will not help I see this attitude frequently in online discussions around {security thing}, but

> what i'm saying is, if you're fucked, you're fucked. fail2ban will not help I see this attitude frequently in online discussions around {security thing}, but it really doesn't match my understanding of reality. It's not about making any one part of the system perfectly secure, but instead about having all of the pieces of it contribute to a whole. Many real-life stories of caught/thwarted attacks include the threat actor successfully breaching/moving through multiple layers of a system but ultimately getting stuck on something relatively simple. Defense in depth is not just a catchphrase. In evaluating whether something like this is genuinely useful, it can make sense to look at how difficult it is to operate, and how much it might reduce the probability of a successful attack. In my experience, fail2ban has been extremely easy to install and operate, and I think it brings at least some meaningful impact (particularly if applied to other things than SSH). At the very least it can cause someone making a focused attack to have work more slowly and take alternative approaches–"noisy" behavior that may itself trigger an alarm and get a human involved.

ddtaylor 3y ago
I agree, defense by depth.
djoletina 3y ago
I agree completely. Security is not black and white, same goes for agents that are malicious, if you’re protected from script kiddies by changing your ssh port and adding fail2van then it’s a net positive any way you look at it. It’s important to be aware of course that you’ll need to go a step further if you want to protect from more competent attackers, but what percentage of us will be a target of one of those.
rstuart4133 3y ago
You're being far too polite. He's just plain wrong on this point: > do not change your standard ports. do not use fail2ban. That used to be my attitude too. But reality intruded. Example: I ran a SIP Server for a business. SIP Servers are very attractive targets because if you get past the security, you can make then call paid numbers and make real money. Knowing that, I used huge security tokens. No one got past them. But - it didn't stop them from trying. 100's of times per second at times. The CPU load created by processing the login attempts was so great the SIP server started dropping calls. As it happens, asking the kernel to drop packets from miscreant IP's takes far, far less CPU. Fail2ban to the rescue. Example: Move on a few years, and now spammers costs have dropped so much it is worth their while to target IMAP, POP and SMTP servers. The story is the same: they never got through, but despite that their attempts increased to the point the server became overloaded. I can't listen on a different port. Fail2ban to the rescue. Example: Move on to the last couple of years. I move my personal stuff to a VPS. To save costs it's very memory constrained. A problem soon arises. After a few weeks of trouble free operation it keeps slowing down, and sometimes becoming unresponsive. Turns out it's running out of memory; the OOM killer is running amok. That's odd, as there should be almost no load on this thing. Well, apart from ssh login attempts it seems. Turns out there is one of them every few seconds. They have no hope of succeeding of course as I only allow pubic key auth, but the load it creates is a problem. Fail2ban to the rescue. I don't see anything wrong with the principle, which is you block these attacks at the cheapest point possible. That happens to be the point of entry, before you spend CPU cycles processing the packet (even in TCP). Fail2ban leverages that by using the kernel's networking stack. It's forced to glean the information on who to ban from log files, which is messy, error prone and fragile. I don't like that either, but I'll take what I can get.
- thraxil 3y ago
  Yeah, they also miss (IMO) one of the biggest advantages of using something like fail2ban which is the reduction in logging noise. I've seen multiple VMs effectively DoS-ed because their disk filled up with SSH log entries. I've been paged by security monitoring tools that registered all the failed login attempts as attacks, and I've had to filter out or just wade through all those logs when investigating other issues. fail2ban also works with services other than SSH, and one of my favorite tricks is to point it at nginx logs, add a `robots.txt` with a honeypot entry and have fail2ban block any IP that hits that or any of a dozen common vulnerability endpoints that I know aren't on the server (like `wp-admin.php`). Keeps the web logs cleaner and more useful.
- tourmalinetaco 3y ago
  I too only allow pubic key auth :P More seriously, I don’t see why anyone would want to disable fail2ban (outside of purposefully being against the grain) because it seems like “free” protection. I’m sure there’s some situations where it may be beneficial to, but those seem like extreme edge cases.
- mgkimsal 3y ago
  > I can't listen on a different port. And the author insists you shouldn't change ports any way. But seems to have no trouble just saying "use the security in your applications". Well... what sort of security is there in an IMAP server beyond "validate credentials"? My other thought, reading the author's post, and having seen enough of this type of attitude, is that this type of person would be the one to say "just use... <external mail service>" or similar. "You can't ever do better than them!" Well... I have to suspect that techniques like fail2ban - blocking network access after X suspicious access attempts - get employed at the big service providers too. Possibly not fail2ban specifically, but the same ideas.
  - 22c 3y ago
    Several years ago I was part of team managing one of the busiest recursive DNS resolvers in the country. Around the time, DNS reflection attacks were becoming all the rage (I think there might have been a vulnerability in PowerDNS to blame, I don't specifically recall the details). The solution we employed was a "dumb" fail2ban like script to ban any abusive IP address. It worked wonders and practically completely neutralized the attacks. We received zero complaints from any legitimate users of the service. If I recall correctly, even after the attacks subsided (CVEs was patched etc.), we still kept a toned down version of the script in place. > at best, fail2ban: does nothing This is just plain wrong. Even if you're only using key based auth and there's no way an attacker could have your keys, fail2ban will still block the abusive traffic. I have no issues with blocking abusive traffic instead of letting the attackers beat their fists against my armored doors. > at worst, fail2ban: causes you massive inconvinience or total lockout The authors main criticism seem to be that you might lock yourself out of your own server one day. This hardly seems fair because the author is saying "just setup SSH correctly and you won't have any issues" while also saying "if you don't setup fail2ban correctly, you'll have a whole bunch of issues!" OK, so shouldn't we just setup fail2ban correctly as well? Having said all that, my recommendation for people who are going to use an indiscriminate fail2ban setup on SSH access would be to have some out-of-band access in case you do happen to ban yourself, even if it's less convenient to use than SSH.

ddtaylor 3y ago

I agree, defense by depth.

djoletina 3y ago

I agree completely. Security is not black and white, same goes for agents that are malicious, if you’re protected from script kiddies by changing your ssh port and adding fail2van then it’s a net positive any way you look at it. It’s important to be aware of course that you’ll need to go a step further if you want to protect from more competent attackers, but what percentage of us will be a target of one of those.

rstuart4133 3y ago

You're being far too polite. He's just plain wrong on this point: > do not change your standard ports. do not use fail2ban. That used to be my attitude too. But reality intruded. Example: I ran a SIP Server for a business. SIP Servers are very attractive targets because if you get past the security, you can make then call paid numbers and make real money. Knowing that, I used huge security tokens. No one got past them. But - it didn't stop them from trying. 100's of times per second at times. The CPU load created by processing the login attempts was so great the SIP server started dropping calls. As it happens, asking the kernel to drop packets from miscreant IP's takes far, far less CPU. Fail2ban to the rescue. Example: Move on a few years, and now spammers costs have dropped so much it is worth their while to target IMAP, POP and SMTP servers. The story is the same: they never got through, but despite that their attempts increased to the point the server became overloaded. I can't listen on a different port. Fail2ban to the rescue. Example: Move on to the last couple of years. I move my personal stuff to a VPS. To save costs it's very memory constrained. A problem soon arises. After a few weeks of trouble free operation it keeps slowing down, and sometimes becoming unresponsive. Turns out it's running out of memory; the OOM killer is running amok. That's odd, as there should be almost no load on this thing. Well, apart from ssh login attempts it seems. Turns out there is one of them every few seconds. They have no hope of succeeding of course as I only allow pubic key auth, but the load it creates is a problem. Fail2ban to the rescue. I don't see anything wrong with the principle, which is you block these attacks at the cheapest point possible. That happens to be the point of entry, before you spend CPU cycles processing the packet (even in TCP). Fail2ban leverages that by using the kernel's networking stack. It's forced to glean the information on who to ban from log files, which is messy, error prone and fragile. I don't like that either, but I'll take what I can get.

thraxil 3y ago
Yeah, they also miss (IMO) one of the biggest advantages of using something like fail2ban which is the reduction in logging noise. I've seen multiple VMs effectively DoS-ed because their disk filled up with SSH log entries. I've been paged by security monitoring tools that registered all the failed login attempts as attacks, and I've had to filter out or just wade through all those logs when investigating other issues. fail2ban also works with services other than SSH, and one of my favorite tricks is to point it at nginx logs, add a `robots.txt` with a honeypot entry and have fail2ban block any IP that hits that or any of a dozen common vulnerability endpoints that I know aren't on the server (like `wp-admin.php`). Keeps the web logs cleaner and more useful.
tourmalinetaco 3y ago
I too only allow pubic key auth :P More seriously, I don’t see why anyone would want to disable fail2ban (outside of purposefully being against the grain) because it seems like “free” protection. I’m sure there’s some situations where it may be beneficial to, but those seem like extreme edge cases.
mgkimsal 3y ago
> I can't listen on a different port. And the author insists you shouldn't change ports any way. But seems to have no trouble just saying "use the security in your applications". Well... what sort of security is there in an IMAP server beyond "validate credentials"? My other thought, reading the author's post, and having seen enough of this type of attitude, is that this type of person would be the one to say "just use... <external mail service>" or similar. "You can't ever do better than them!" Well... I have to suspect that techniques like fail2ban - blocking network access after X suspicious access attempts - get employed at the big service providers too. Possibly not fail2ban specifically, but the same ideas.
- 22c 3y ago
  Several years ago I was part of team managing one of the busiest recursive DNS resolvers in the country. Around the time, DNS reflection attacks were becoming all the rage (I think there might have been a vulnerability in PowerDNS to blame, I don't specifically recall the details). The solution we employed was a "dumb" fail2ban like script to ban any abusive IP address. It worked wonders and practically completely neutralized the attacks. We received zero complaints from any legitimate users of the service. If I recall correctly, even after the attacks subsided (CVEs was patched etc.), we still kept a toned down version of the script in place. > at best, fail2ban: does nothing This is just plain wrong. Even if you're only using key based auth and there's no way an attacker could have your keys, fail2ban will still block the abusive traffic. I have no issues with blocking abusive traffic instead of letting the attackers beat their fists against my armored doors. > at worst, fail2ban: causes you massive inconvinience or total lockout The authors main criticism seem to be that you might lock yourself out of your own server one day. This hardly seems fair because the author is saying "just setup SSH correctly and you won't have any issues" while also saying "if you don't setup fail2ban correctly, you'll have a whole bunch of issues!" OK, so shouldn't we just setup fail2ban correctly as well? Having said all that, my recommendation for people who are going to use an indiscriminate fail2ban setup on SSH access would be to have some out-of-band access in case you do happen to ban yourself, even if it's less convenient to use than SSH.