5 ms·

They fill the logs with noise. That's "something." I finally set up fail2ban a while back... it works wonders.

by icedchai 6y ago

gruez 6y ago
Do people even read the logs? What value does it add compared to just monitoring successful logins?
- cedilla 6y ago
  There's a lot of value. For example, if you see failed logins against random user names like "dbadmin" or "root" it's likely just random scanning, but what if suddenly lots and lots of valid user names appear?
  - gruez 6y ago
    >but what if suddenly lots and lots of valid user names appear? then what are you going to do?
    - sokoloff 6y ago
      At a minimum, spend some of my limited time and attention on this issue rather than the 100s of other things that might be clamoring for my time.
      - wbl 6y ago
        What are you going to do to solve that issue?
    - cedilla 6y ago
      Well that would highly depend on what I'm seeing. If it's a single user there might be an attack on the way against that user. If it's multiple users, there might have been a compromise of some credentials. It's definitely something you need to investigate.
  - cthalupa 6y ago
    grep and zgrep will work wonders for checking for actual usernames in these logs even if they have significant amount of spam in them.
  - nkozyra 6y ago
    That's a great point, but I get back to the root question: who's actually looking at this? If people are examining logs it's usually for a particular trigger or a problem and filtering that signal from the noise is hard.
    - thelamest 6y ago
      It's more typical of the servers-as-pets than servers-as-cattle scenario, but sometimes one is simply curious [or extra cautious]. SSH honeypots exist at least in part for this reason.
      - boojums 6y ago
        > servers-as-pets This is a great way to put it.
    - jldugger 6y ago
      > who's actually looking at this? Well, your security team, post incident. But also automated systems like fail2ban.
      - warrenm 6y ago
        And log-collectors like Splunk (with configured alerts, etc)
    - Sohcahtoa82 6y ago
      Likely, nobody is directly looking at the logs. But they might be using software that automatically raises an alert when it sees repeated login attempts for a valid username. Isn't that one of the purposes of Splunk?
- elmo2you 6y ago
  Did you ever had the "pleasure" of a server grinding to a halt because the logs filled up all the space? To where you had to mount the disk to another system and clean it up before it wants to boot from again. Can be a bitch if it's a machine on a remote location. Not everything is cloud (yet) these days. Granted, there usually is a lot more at fault when you run into such problems, but I find people not looking at logs a rather weak argument for letting them get spammed full with garbage. Certainly terrible hygiene, at least.
  - cthalupa 6y ago
    >Did you ever had the "pleasure" of a server grinding to a halt because the logs filled up all the space? Everyone should be using logrotate, and if they actually read the things, shipping logs to ELK or Splunk or Greylog or whatever.
    - elmo2you 6y ago
      > Everyone should be using logrotate, and if they actually read the things, shipping logs to ELK or Splunk or Greylog or whatever. Certainly they should. That is, if they have that much control over the server and if it's not some legacy system build by some defunct organization or John Doe. I do not disagree with your theory, on the contrary. But then there is reality, where this theory isn't always feasible.
    - warrenm 6y ago
      >Everyone should be using logrotate Doesn't run often enough to counter belligerents dumping 100s of gigs into logs too fast for logrotate to keep up
  - gruez 6y ago
    >Did you ever had the "pleasure" of a server grinding to a halt because the logs filled up all the space? I've never seen this issue on any systems I manage, mostly because they all have log rotation. >but I find people not looking at logs a rather weak argument for letting them get spammed full with garbage. Certainly terrible hygiene, at least. Why is it weak argument? If it's something that doesn't materially impact you, why should you expend effort into remediating it? Hygiene is only important for things we interact with on a regular basis. We as a society don't care about the hygiene of the sewer system, for instance.

They fill the logs with noise. That's "something." I finally set up fail2ban a while back... it works wonders.

gruez 6y ago

Do people even read the logs? What value does it add compared to just monitoring successful logins?

cedilla 6y ago
There's a lot of value. For example, if you see failed logins against random user names like "dbadmin" or "root" it's likely just random scanning, but what if suddenly lots and lots of valid user names appear?
- gruez 6y ago
  >but what if suddenly lots and lots of valid user names appear? then what are you going to do?
  - sokoloff 6y ago
    At a minimum, spend some of my limited time and attention on this issue rather than the 100s of other things that might be clamoring for my time.
    - wbl 6y ago
      What are you going to do to solve that issue?
  - cedilla 6y ago
    Well that would highly depend on what I'm seeing. If it's a single user there might be an attack on the way against that user. If it's multiple users, there might have been a compromise of some credentials. It's definitely something you need to investigate.
- cthalupa 6y ago
  grep and zgrep will work wonders for checking for actual usernames in these logs even if they have significant amount of spam in them.
- nkozyra 6y ago
  That's a great point, but I get back to the root question: who's actually looking at this? If people are examining logs it's usually for a particular trigger or a problem and filtering that signal from the noise is hard.
  - thelamest 6y ago
    It's more typical of the servers-as-pets than servers-as-cattle scenario, but sometimes one is simply curious [or extra cautious]. SSH honeypots exist at least in part for this reason.
    - boojums 6y ago
      > servers-as-pets This is a great way to put it.
  - jldugger 6y ago
    > who's actually looking at this? Well, your security team, post incident. But also automated systems like fail2ban.
    - warrenm 6y ago
      And log-collectors like Splunk (with configured alerts, etc)
  - Sohcahtoa82 6y ago
    Likely, nobody is directly looking at the logs. But they might be using software that automatically raises an alert when it sees repeated login attempts for a valid username. Isn't that one of the purposes of Splunk?
elmo2you 6y ago
Did you ever had the "pleasure" of a server grinding to a halt because the logs filled up all the space? To where you had to mount the disk to another system and clean it up before it wants to boot from again. Can be a bitch if it's a machine on a remote location. Not everything is cloud (yet) these days. Granted, there usually is a lot more at fault when you run into such problems, but I find people not looking at logs a rather weak argument for letting them get spammed full with garbage. Certainly terrible hygiene, at least.
- cthalupa 6y ago
  >Did you ever had the "pleasure" of a server grinding to a halt because the logs filled up all the space? Everyone should be using logrotate, and if they actually read the things, shipping logs to ELK or Splunk or Greylog or whatever.
  - elmo2you 6y ago
    > Everyone should be using logrotate, and if they actually read the things, shipping logs to ELK or Splunk or Greylog or whatever. Certainly they should. That is, if they have that much control over the server and if it's not some legacy system build by some defunct organization or John Doe. I do not disagree with your theory, on the contrary. But then there is reality, where this theory isn't always feasible.
  - warrenm 6y ago
    >Everyone should be using logrotate Doesn't run often enough to counter belligerents dumping 100s of gigs into logs too fast for logrotate to keep up
- gruez 6y ago
  >Did you ever had the "pleasure" of a server grinding to a halt because the logs filled up all the space? I've never seen this issue on any systems I manage, mostly because they all have log rotation. >but I find people not looking at logs a rather weak argument for letting them get spammed full with garbage. Certainly terrible hygiene, at least. Why is it weak argument? If it's something that doesn't materially impact you, why should you expend effort into remediating it? Hygiene is only important for things we interact with on a regular basis. We as a society don't care about the hygiene of the sewer system, for instance.