5 ms·

For all but the most basic attack, you really want a script putting these IP's into iptables. Using the application itself to block them still requires the con

For all but the most basic attack, you really want a script putting these IP's into iptables. Using the application itself to block them still requires the connection setup/teardown resources to be used, as well as the application itself.

giancarlostoro 11y ago
I suppose that's where Lua would come in handy?
- iMerNibor 11y ago
  I personally have a access_by_lua script that counts accesses per ip and applies a (very generous) rate limit If the limit is reached the user will just be presented with a page explaining you hit a rate limit and a button that runs some javascript to verify you're not a bot which in turn whitelists the user This strategy has worked really well so far - havent been a target of too bad things yet though. Its a very good and cheap way to go for smaller sites though
  - jsmeaton 11y ago
    Can a bot dedicated to your site just ping back whatever the javascript would have done anyway to cancel the rate limit?
    - iMerNibor 11y ago
      It /could/, the next step would be a captcha or something harder for bots to solve - haven't had to go that far yet though. But I usually only have to deal with script kiddies who rent out a botnet, enter a url and click the "attack" button
hyperdunc 11y ago
I can see how doing it lower level would be more efficient. Are there any scripts anyone could recommend as a starting point?
- SwellJoe 11y ago
  fail2ban can watch the nginx logs for throttling and/or blocking messages and add iptables rules for you. I haven't read this all the way through, but on a cursory glance it looks reasonable: https://easyengine.io/tutorials/nginx/fail2ban/ https://easyengine.io/tutorials/nginx/fail2ban/
  - XorNot 11y ago
    Urgh logwatching actively pains me these days. So much waste string parsing what was originally binary data anyway. I'm starting to think that we need some agreement where instead of logs, we just get apps to emit a stream of protocol buffers and a format string for the messages and data. Which does make me wonder if you couldn't LD_PRELOAD something which replaced fprintf and the like...
    - SwellJoe 11y ago
      "So much waste string parsing what was originally binary data anyway." "So much" is pretty imprecise. How much waste do you believe string parsing incurs in this case?
    - superuser2 11y ago
      Streams of plain text are what UNIX was built on. If you want binary APIs, look outside the *nix family.
      - Ao7bei3s 11y ago
        Or modernize the applications. Throw out the ad-hoc formats and parsers, replace them with machine-readable equivalents. For example, systemd finally provides a logging system that allows structured logging with key/value fields.
    - baudehlo 11y ago
      You can go the systemd route if you want, or just know the fact that parsing strings works, is mostly reliable, and really isn't as much overhead as people make it out to be. How many system profiles have identified it as a problem?

giancarlostoro 11y ago

I suppose that's where Lua would come in handy?

iMerNibor 11y ago
I personally have a access_by_lua script that counts accesses per ip and applies a (very generous) rate limit If the limit is reached the user will just be presented with a page explaining you hit a rate limit and a button that runs some javascript to verify you're not a bot which in turn whitelists the user This strategy has worked really well so far - havent been a target of too bad things yet though. Its a very good and cheap way to go for smaller sites though
- jsmeaton 11y ago
  Can a bot dedicated to your site just ping back whatever the javascript would have done anyway to cancel the rate limit?
  - iMerNibor 11y ago
    It /could/, the next step would be a captcha or something harder for bots to solve - haven't had to go that far yet though. But I usually only have to deal with script kiddies who rent out a botnet, enter a url and click the "attack" button

hyperdunc 11y ago

I can see how doing it lower level would be more efficient. Are there any scripts anyone could recommend as a starting point?

SwellJoe 11y ago
fail2ban can watch the nginx logs for throttling and/or blocking messages and add iptables rules for you. I haven't read this all the way through, but on a cursory glance it looks reasonable: https://easyengine.io/tutorials/nginx/fail2ban/ https://easyengine.io/tutorials/nginx/fail2ban/
- XorNot 11y ago
  Urgh logwatching actively pains me these days. So much waste string parsing what was originally binary data anyway. I'm starting to think that we need some agreement where instead of logs, we just get apps to emit a stream of protocol buffers and a format string for the messages and data. Which does make me wonder if you couldn't LD_PRELOAD something which replaced fprintf and the like...
  - SwellJoe 11y ago
    "So much waste string parsing what was originally binary data anyway." "So much" is pretty imprecise. How much waste do you believe string parsing incurs in this case?
  - superuser2 11y ago
    Streams of plain text are what UNIX was built on. If you want binary APIs, look outside the *nix family.
    - Ao7bei3s 11y ago
      Or modernize the applications. Throw out the ad-hoc formats and parsers, replace them with machine-readable equivalents. For example, systemd finally provides a logging system that allows structured logging with key/value fields.
  - baudehlo 11y ago
    You can go the systemd route if you want, or just know the fact that parsing strings works, is mostly reliable, and really isn't as much overhead as people make it out to be. How many system profiles have identified it as a problem?