16 ms·

Back when I started managing self-hosted sites, I would look through access logs as well. We even had an IDS for while that would aggregate the data and flag in

Back when I started managing self-hosted sites, I would look through access logs as well. We even had an IDS for while that would aggregate the data and flag incoming attack attempts for us. Eventually I stopped proactively reviewing logs and stopped paying for the IDS. It was a waste of time and a distraction. It's not hard to find really useful content that summarizes common vulnerabilities and attacks, and just use that to guide your server management. There are a ton of best practices guides for any common web server technology. Just executing these best practices to 100% will put you way ahead of almost all attackers. And then the next best use of your time and resources is to prioritize the fastest possible patching cadence, since the vast majority of attacks target disclosed vulnerabilities. Where logs are super helpful is in diagnosing problems after they happen. We used log analysis software to store and search logs and this was helpful 2-3 times to help find (and therefore address) the root cause of attacks that succeeded. (In every case it turned out to be a known vulnerability that we had been too slow to patch.)

dandrew5 3y ago
> And then the next best use of your time and resources is to prioritize the fastest possible patching cadence, since the vast majority of attacks target disclosed vulnerabilities. Just curious, do you leverage any tools to decide when to patch or is it time-interval based? We currently attempt[0] to update our packages quarterly but it would be nice to have a tool alert us of known vulnerabilities so we can take action on them immediately. [0] "Attempt" meaning we can't always upgrade immediately if the latest version contains difficult-to-implement breaking changes or if it's a X.0.0 release that we don't yet trust
- eyegor 3y ago
  Besides pointing pentester tools like metasploit at yourself, there are some nice scanners out there. Examples in no particular order: https://github.com/quay/clair https://github.com/quay/clair https://github.com/anchore/grype/ https://github.com/anchore/grype/ https://github.com/eliasgranderubio/dagda/ https://github.com/eliasgranderubio/dagda/ https://github.com/aquasecurity/trivy https://github.com/aquasecurity/trivy So then you set up something like a cron job to scan everything for you and email the results once a week or whatever if you don't want to monitor things actively.
  - dandrew5 3y ago
    Thanks for these. I should have clarified, I'm more interested in something that will alert me when newly-discovered vulnerabilities surface. The systems I maintain are protected [enough] today but a new hack could drop that doesn't make mainstream media and I may not hear about it. We have annual security audits but it would be nice to patch things immediately. Aside from subscribing to a security forum/discord/slack, I'm wondering what other methods folks are employing to solve this.
    - bradknowles 3y ago
      Keep your pentesting tools up-to-date. Run them against yourself on every single deployment, if you can. Don't just run them quarterly because that's all that your PCI-DSS requirements say you have to do. Integrate security code scanning tools into your CI/CD process. Tools like Dry Run Security, or something comparable. There's much more, but that has to do with how to run your CI/CD systems and how to do your deployments in general, and less to do with security aspects thereof.
    - hiStartex 3y ago
      For Gentoo: glsa-check --list
  - BLKNSLVR 3y ago
    I just setup GVM / OpenVAS[0] to work out where I need to put in some maintenance work, how does that rate as worthwhile in comparison to those you've listed above? (which I will also look into). (not a fan of the effort Greenbone have gone to for hiding their community edition and promoting their commercial products) [0]: https://greenbone.github.io/docs/latest/22.4/container/index.html https://greenbone.github.io/docs/latest/22.4/container/index...
- layer8 3y ago
  The simplest is to only use packages from a distribution like Debian and run unattended-upgrades or equivalent for the security-updates repository. They usually fix vulnerabilities in less than a day.
- snowwrestler 3y ago
  I’m out of the self-hosting game now, but back in the day we just tried to keep up on security announcements.
gnyman 3y ago
> In every case it turned out to be a known vulnerability that we had been too slow to patch. Yes. This is why relying on "patching" is a bound to fail at some point. Maybe it's a 0-day, or maybe the attackers are just quicker. The solution to this is defence in depth, and it's very easy for most services, especially when self-hosting personal things. Few tips most people can do is. Put up a firewall in front or put it behind VPN/tailscale. Hide it in a subfolder. The automated attacks will go for /phpmyadmin/ , putting it in /mawer/phpmyadmin/ means 99.9% of the attackers won't find it. (This is sometimes called security by obscurity and people correctly say you should not rely on it, but as a additional layer it's very useful). Sandbox the app, and isolate the server. If the attackers get in, make it hard or impossible for them to get anywhere else. Keep logs, they allow you to check if and how you got attacked, if the attack succeeded and so on. Depending on the service, pick one or more of these. Add more as necessary. The key thing is that you should not rely on any ONE defence, be it keeping it patched or firewalled, because they will all fail at some point.
- citizenpaul 3y ago
  >security by obscurity I detest this phrase right up there with "fake it till you make it". All security is by definition obscurity. Just a meaningless platitude that rhymes. I suppose un-formalized un-proofed security practices will eventually be broken or counting on hackers not to do any investigation of your system will get you hacked, doesn't roll off the tongue though.
  - rablackburn 3y ago
    > All security is by definition obscurity. Can you expound on this? For example, if I add a 30 second lockout after failed authentication attempts I don’t see how that comes under any non-tortured definition of “obscurity”.
    - hanszarkov 3y ago
      yes, also add cryptographic complexity, don't think that qualifies either.

dandrew5 3y ago

> And then the next best use of your time and resources is to prioritize the fastest possible patching cadence, since the vast majority of attacks target disclosed vulnerabilities. Just curious, do you leverage any tools to decide when to patch or is it time-interval based? We currently attempt[0] to update our packages quarterly but it would be nice to have a tool alert us of known vulnerabilities so we can take action on them immediately. [0] "Attempt" meaning we can't always upgrade immediately if the latest version contains difficult-to-implement breaking changes or if it's a X.0.0 release that we don't yet trust

eyegor 3y ago
Besides pointing pentester tools like metasploit at yourself, there are some nice scanners out there. Examples in no particular order: https://github.com/quay/clair https://github.com/quay/clair https://github.com/anchore/grype/ https://github.com/anchore/grype/ https://github.com/eliasgranderubio/dagda/ https://github.com/eliasgranderubio/dagda/ https://github.com/aquasecurity/trivy https://github.com/aquasecurity/trivy So then you set up something like a cron job to scan everything for you and email the results once a week or whatever if you don't want to monitor things actively.
- dandrew5 3y ago
  Thanks for these. I should have clarified, I'm more interested in something that will alert me when newly-discovered vulnerabilities surface. The systems I maintain are protected [enough] today but a new hack could drop that doesn't make mainstream media and I may not hear about it. We have annual security audits but it would be nice to patch things immediately. Aside from subscribing to a security forum/discord/slack, I'm wondering what other methods folks are employing to solve this.
  - bradknowles 3y ago
    Keep your pentesting tools up-to-date. Run them against yourself on every single deployment, if you can. Don't just run them quarterly because that's all that your PCI-DSS requirements say you have to do. Integrate security code scanning tools into your CI/CD process. Tools like Dry Run Security, or something comparable. There's much more, but that has to do with how to run your CI/CD systems and how to do your deployments in general, and less to do with security aspects thereof.
  - hiStartex 3y ago
    For Gentoo: glsa-check --list
- BLKNSLVR 3y ago
  I just setup GVM / OpenVAS[0] to work out where I need to put in some maintenance work, how does that rate as worthwhile in comparison to those you've listed above? (which I will also look into). (not a fan of the effort Greenbone have gone to for hiding their community edition and promoting their commercial products) [0]: https://greenbone.github.io/docs/latest/22.4/container/index.html https://greenbone.github.io/docs/latest/22.4/container/index...
layer8 3y ago
The simplest is to only use packages from a distribution like Debian and run unattended-upgrades or equivalent for the security-updates repository. They usually fix vulnerabilities in less than a day.
snowwrestler 3y ago
I’m out of the self-hosting game now, but back in the day we just tried to keep up on security announcements.

gnyman 3y ago

> In every case it turned out to be a known vulnerability that we had been too slow to patch. Yes. This is why relying on "patching" is a bound to fail at some point. Maybe it's a 0-day, or maybe the attackers are just quicker. The solution to this is defence in depth, and it's very easy for most services, especially when self-hosting personal things. Few tips most people can do is. Put up a firewall in front or put it behind VPN/tailscale. Hide it in a subfolder. The automated attacks will go for /phpmyadmin/ , putting it in /mawer/phpmyadmin/ means 99.9% of the attackers won't find it. (This is sometimes called security by obscurity and people correctly say you should not rely on it, but as a additional layer it's very useful). Sandbox the app, and isolate the server. If the attackers get in, make it hard or impossible for them to get anywhere else. Keep logs, they allow you to check if and how you got attacked, if the attack succeeded and so on. Depending on the service, pick one or more of these. Add more as necessary. The key thing is that you should not rely on any ONE defence, be it keeping it patched or firewalled, because they will all fail at some point.

citizenpaul 3y ago
>security by obscurity I detest this phrase right up there with "fake it till you make it". All security is by definition obscurity. Just a meaningless platitude that rhymes. I suppose un-formalized un-proofed security practices will eventually be broken or counting on hackers not to do any investigation of your system will get you hacked, doesn't roll off the tongue though.
- rablackburn 3y ago
  > All security is by definition obscurity. Can you expound on this? For example, if I add a 30 second lockout after failed authentication attempts I don’t see how that comes under any non-tortured definition of “obscurity”.
  - hanszarkov 3y ago
    yes, also add cryptographic complexity, don't think that qualifies either.