7 ms·

I like the idea of a Fail2Ban alternative, but I'm not sold on the idea of sending information to a third party. It may be optional to upload/download the malic

I like the idea of a Fail2Ban alternative, but I'm not sold on the idea of sending information to a third party. It may be optional to upload/download the malicious IP addresses, but the text on this project's GitHub page doesn't indicate that it is optional. My immediate thought is "Fail2ban works. I don't need or want communication with a 3rd party right now."

golem14 6y ago
I think that's fair, and I had the same initial reaction. However, people are generally fine with DMARC or other crowdsourced spam signals, so it's not clearly bad. I haven't looked at the source, but it should be easy to turn the crowdfiltering off or self-host if you have your own larger enterprise, no ? It also depends on what exactly is transmitted. Transmitting all IP addresses in the clear might be bad, sending out hashes might be better ? I think it's actually a good idea to try to detect those IP at scale. On your own individual server, you just see a million login attempts from a million different nodes on a botnet. But if you have millions of servers report the bad IPs and aggregate, you can see patterns and hopefully catch whole botnets.
- klodolph 6y ago
  > However, people are generally fine with DMARC or other crowdsourced spam signals, so it's not clearly bad. DMARC isn't a crowdsourced spam signal. And from my memories chatting with people running mail servers—the crowdsourced spam filtering is fairly controversial.
  - golem14 6y ago
    Apologies, I was thinking of the spamhaus list, which I'm also not sure now is crowdsourced in the sense most people would use. But most email providers build spamfilters using 'crowdsourced' techniques between their users (if enough users mark a sender spam it might be marked spam for all users). And while that may be controversial, it's also highly effective.
    - monadic2 6y ago
      Speaking from personal experience working with these reputation warehouses and with handling a good deal of incoming and outgoing email volume, they aren't any more effective than other IP-based spam heuristics and if everyone configured their DKIM correctly these warehouses would be largely obsolete. I don't see a problem with crowdsourcing heuristic data per se, but email is a much narrower field than internet traffic at large and I can see a lot of the ways that poorly configured fail2ban type software could accidentally ban a lot of valid traffic.
- kenniskrag 6y ago
  > Transmitting all IP addresses in the clear might be bad, sending out hashes might be better How many IP adresses are valid? Can't you hash them all and compare them with the hash you receive? 2^32 IPs is 4GB * 4 Bytes (1 IP) + 4GB * size of the hash should be the needed space I think.
- _jal 6y ago
  1) That is not what DMARC is. 2) I won't be using any "security" tool that automatically shares details on active mitigation efforts. > I think it's actually a good idea to try to detect those IP at scale. It is! But not with my machines.
egberts1 6y ago
Fail2ban is a bit shaky when it transitions to Python3. Hasn’t pass the pylint,?futurize, or 2to3 by much. But if the maintainers are willing, I can do those things.
- oefrha 6y ago
  Not sure what you mean by shaky, fail2ban has been working under Python 3 for quite a while. (See for instance fail2ban on Ubuntu 16.04 depending on python3[0].) People definitely would have noticed. I saw your issue[1] on GitHub, and that's just a misunderstanding of how fail2ban was ported to Python 3. fail2ban uses the use_2to3 feature of setuptools.setup to do automatic translation upon installation.[2] While I do wish the code runs under Python 3 as is, what users actually run is totally working code. [0] https://packages.ubuntu.com/xenial/amd64/fail2ban https://packages.ubuntu.com/xenial/amd64/fail2ban [1] https://github.com/fail2ban/fail2ban/issues/2853 https://github.com/fail2ban/fail2ban/issues/2853 [2] https://github.com/fail2ban/fail2ban/blob/960e30cfcdae7e2c81f74046efdbff1606f9774e/setup.py#L178-L182 https://github.com/fail2ban/fail2ban/blob/960e30cfcdae7e2c81...
  - egberts1 6y ago
    Yes. It may be totally working code. But it wasn’t written cleanly as many tools would attest.
    - viraptor 6y ago
      Those tools tell you if you comply with some persons style choices, and maybe catch a few potential problems. That says close to nothing about cleanliness or correctness. They're for internal use, not for making external measurements of code.
Philippe_H 6y ago
Sorry, we should have made it clear, it is totally optional. You just don't get the IP rep DB part of the soft if you don't share, but the behavior is still 100% functional.
sleepybrett 6y ago
There is some discussion in the diquis about them providing a server you can run yourself. So if you have a fleet of servers you don't have to share with crowdsec but instead can share only with yourself (my read, perhaps a hybrid)
- wongarsu 6y ago
  To me the main value proposition would be blocking anyone who was observed attacking one of my servers or even a honeypot. I'm not opposed to sharing that data as far as GDPR allows, but receiving block lists is something I would be a lot more careful about.
- Philippe_H 6y ago
  yepn indeed, we call it private sharding or private consensus. Far on the roadmap (4 months), but nevertheless, the team is thinking about it. You could also include or exclude some Geographics for ex if you don't trust a country or have a private consensus between only your own machine. If you are, say Morgan Stanley, you may be attacked only you by some machines and the crowd wouldn't know. But all your servers teaming together will see it.
pantalaimon 6y ago
This sounds terrible. I’ve locked myself out of a machine once because I remembered the password wrong. That was annoying. I might have done it without noticing when trying to log into the wrong machine accidentally more than one. Locking myself out of all machines would be way worse.
- dnautics 6y ago
  I mean you could also imagine someone somehow seeding your ip address into the crowdsourced fail2ban to maliciously deny you access to your machine for maintenance, or to trigger some sort of opsec regression to get access.
  - pas 6y ago
    Spoofing an IP is not exactly hard, so ... yeah, I'm a bit skeptical of the idea. Yet at the same time I think setting up a long running reputation weighted reputation clearing house would be amazing. (And if nothing else at least something like this could emerge from all the crypto-staking-reputation research. See Polkadot and Ethereum's plans about keeping folks honest with bounties (reverse staking?) for example.)
    - Philippe_H 6y ago
      Well actually Spoofing on a private network is trivial, but in TCP over a public network, it's another story entirely and it's not simple at all. UDP can be easily spoofed though, hence we do not treat reports in the same way so far. Beyond this, there is eventually BGP spoofing, but funny enough, CrowdSec could detect them, provided you have logs. It should be fairly easy to track in terms of behavior.
      - dnautics 6y ago
        You wouldn't have to spoof. I said seed, not spoof - you could just spin up a bunch of servers, connect it to the crowdsourced security networks, and issue false claims that IP address X has been attempting to break into your network, and suddenly block X from their own servers.
tambourine_man 6y ago
I have the opposite feeling. I don’t see why we need to replace fail2ban, but I find the idea of a global repository of bad IPs and reputation very interesting.
- SEJeff 6y ago
  I can think of a few: https://www.cvedetails.com/vulnerability-list/vendor_id-5567/Fail2ban.html https://www.cvedetails.com/vulnerability-list/vendor_id-5567...

golem14 6y ago

I think that's fair, and I had the same initial reaction. However, people are generally fine with DMARC or other crowdsourced spam signals, so it's not clearly bad. I haven't looked at the source, but it should be easy to turn the crowdfiltering off or self-host if you have your own larger enterprise, no ? It also depends on what exactly is transmitted. Transmitting all IP addresses in the clear might be bad, sending out hashes might be better ? I think it's actually a good idea to try to detect those IP at scale. On your own individual server, you just see a million login attempts from a million different nodes on a botnet. But if you have millions of servers report the bad IPs and aggregate, you can see patterns and hopefully catch whole botnets.

klodolph 6y ago
> However, people are generally fine with DMARC or other crowdsourced spam signals, so it's not clearly bad. DMARC isn't a crowdsourced spam signal. And from my memories chatting with people running mail servers—the crowdsourced spam filtering is fairly controversial.
- golem14 6y ago
  Apologies, I was thinking of the spamhaus list, which I'm also not sure now is crowdsourced in the sense most people would use. But most email providers build spamfilters using 'crowdsourced' techniques between their users (if enough users mark a sender spam it might be marked spam for all users). And while that may be controversial, it's also highly effective.
  - monadic2 6y ago
    Speaking from personal experience working with these reputation warehouses and with handling a good deal of incoming and outgoing email volume, they aren't any more effective than other IP-based spam heuristics and if everyone configured their DKIM correctly these warehouses would be largely obsolete. I don't see a problem with crowdsourcing heuristic data per se, but email is a much narrower field than internet traffic at large and I can see a lot of the ways that poorly configured fail2ban type software could accidentally ban a lot of valid traffic.
kenniskrag 6y ago
> Transmitting all IP addresses in the clear might be bad, sending out hashes might be better How many IP adresses are valid? Can't you hash them all and compare them with the hash you receive? 2^32 IPs is 4GB * 4 Bytes (1 IP) + 4GB * size of the hash should be the needed space I think.
_jal 6y ago
1) That is not what DMARC is. 2) I won't be using any "security" tool that automatically shares details on active mitigation efforts. > I think it's actually a good idea to try to detect those IP at scale. It is! But not with my machines.

egberts1 6y ago

Fail2ban is a bit shaky when it transitions to Python3. Hasn’t pass the pylint,?futurize, or 2to3 by much. But if the maintainers are willing, I can do those things.

oefrha 6y ago
Not sure what you mean by shaky, fail2ban has been working under Python 3 for quite a while. (See for instance fail2ban on Ubuntu 16.04 depending on python3[0].) People definitely would have noticed. I saw your issue[1] on GitHub, and that's just a misunderstanding of how fail2ban was ported to Python 3. fail2ban uses the use_2to3 feature of setuptools.setup to do automatic translation upon installation.[2] While I do wish the code runs under Python 3 as is, what users actually run is totally working code. [0] https://packages.ubuntu.com/xenial/amd64/fail2ban https://packages.ubuntu.com/xenial/amd64/fail2ban [1] https://github.com/fail2ban/fail2ban/issues/2853 https://github.com/fail2ban/fail2ban/issues/2853 [2] https://github.com/fail2ban/fail2ban/blob/960e30cfcdae7e2c81f74046efdbff1606f9774e/setup.py#L178-L182 https://github.com/fail2ban/fail2ban/blob/960e30cfcdae7e2c81...
- egberts1 6y ago
  Yes. It may be totally working code. But it wasn’t written cleanly as many tools would attest.
  - viraptor 6y ago
    Those tools tell you if you comply with some persons style choices, and maybe catch a few potential problems. That says close to nothing about cleanliness or correctness. They're for internal use, not for making external measurements of code.

Philippe_H 6y ago

Sorry, we should have made it clear, it is totally optional. You just don't get the IP rep DB part of the soft if you don't share, but the behavior is still 100% functional.

sleepybrett 6y ago

There is some discussion in the diquis about them providing a server you can run yourself. So if you have a fleet of servers you don't have to share with crowdsec but instead can share only with yourself (my read, perhaps a hybrid)

wongarsu 6y ago
To me the main value proposition would be blocking anyone who was observed attacking one of my servers or even a honeypot. I'm not opposed to sharing that data as far as GDPR allows, but receiving block lists is something I would be a lot more careful about.
Philippe_H 6y ago
yepn indeed, we call it private sharding or private consensus. Far on the roadmap (4 months), but nevertheless, the team is thinking about it. You could also include or exclude some Geographics for ex if you don't trust a country or have a private consensus between only your own machine. If you are, say Morgan Stanley, you may be attacked only you by some machines and the crowd wouldn't know. But all your servers teaming together will see it.

pantalaimon 6y ago

This sounds terrible. I’ve locked myself out of a machine once because I remembered the password wrong. That was annoying. I might have done it without noticing when trying to log into the wrong machine accidentally more than one. Locking myself out of all machines would be way worse.

dnautics 6y ago
I mean you could also imagine someone somehow seeding your ip address into the crowdsourced fail2ban to maliciously deny you access to your machine for maintenance, or to trigger some sort of opsec regression to get access.
- pas 6y ago
  Spoofing an IP is not exactly hard, so ... yeah, I'm a bit skeptical of the idea. Yet at the same time I think setting up a long running reputation weighted reputation clearing house would be amazing. (And if nothing else at least something like this could emerge from all the crypto-staking-reputation research. See Polkadot and Ethereum's plans about keeping folks honest with bounties (reverse staking?) for example.)
  - Philippe_H 6y ago
    Well actually Spoofing on a private network is trivial, but in TCP over a public network, it's another story entirely and it's not simple at all. UDP can be easily spoofed though, hence we do not treat reports in the same way so far. Beyond this, there is eventually BGP spoofing, but funny enough, CrowdSec could detect them, provided you have logs. It should be fairly easy to track in terms of behavior.
    - dnautics 6y ago
      You wouldn't have to spoof. I said seed, not spoof - you could just spin up a bunch of servers, connect it to the crowdsourced security networks, and issue false claims that IP address X has been attempting to break into your network, and suddenly block X from their own servers.

tambourine_man 6y ago

I have the opposite feeling. I don’t see why we need to replace fail2ban, but I find the idea of a global repository of bad IPs and reputation very interesting.

SEJeff 6y ago
I can think of a few: https://www.cvedetails.com/vulnerability-list/vendor_id-5567/Fail2ban.html https://www.cvedetails.com/vulnerability-list/vendor_id-5567...