4 ms·

100% agree. Disclaimer I own a data center and have dealt with customer collocated equipment breaches. In addition to the above steps: -> disable root being

100% agree. Disclaimer I own a data center and have dealt with customer collocated equipment breaches. In addition to the above steps: -> disable root being able to login inside your sshd_config file. Make sure PermitRootLogin no -> rename the root account too so if they are using an exploit based on user authentication then perhaps they won't be able to elevate to root. -> disable password based logins and go to cert based auth. This will shutdown brute force attacks. -> lower MaxAuthTries in sshd_config to something like 1 or 2 to help slow down attackers. -> change the port you listen on. A little security through obscurity while not very effective might slow future casual port scanners that are testing a single port. In practice I've seen this really eliminate a lot of reconnaissance or farming type activities. -> Make sure you're openssl and openssh are at the latest stable releases. -> If the perp is coming from the same IP, you can use an iptables rule to block the netblock. Again, not perfect but may help slow things down. -> grab shell history of all the user accounts on the box. Example ~/.bash_history. This is more reconnaissance but may be helpful if they are sloppy - you might see what they were doing on the box. -> look for any modified files or new ones. Obviously logs will show up in the list but you're looking for things that should not be there. Example: find / * -mtime -60 -print where 60 is how many days ago you want to go back. -> look at chron file to see if any timed delay bombs exist. -> look at ps -aux output for any running processes that don't make sense -> look at iptraf for any suspicious traffic to IPs you can't reconcile. Good luck!

VikingCoder 10y ago
...it's a Windows box. (Just kidding.)
- sredfern2 10y ago
  :-)
eonw 10y ago
just a note: intelligent exploiters hide their files inside of yours, so the -mtime is useless in many cases, they will set the mtime of their upload to match the rest of the folder they hide in. command history is also easy to alter if you know what you are doing.
- porker 10y ago
  So the only way to detect exploits is to scan the server regularly and log the mtime and the file size, and look for changed files that shouldn't have changed? Re the command history, is there any way around them wiping it - e.g. piping all .bash_history entries to an append-only store?
  - spdustin 10y ago
    RootKit Hunter [0] works pretty well on most distros to check hashes on files and other potential problems. You can also use a global bash configuration that would log all commands [1] entered into any bash shell to a central log, which could be shipped off-server simultaneously. [0]: http://rkhunter.sourceforge.net http://rkhunter.sourceforge.net [1]: http://askubuntu.com/questions/93566/how-to-log-all-bash-commands-by-all-users-on-a-server http://askubuntu.com/questions/93566/how-to-log-all-bash-com...
    - uxp 10y ago
      Does RKHunter scan userspace for mtimes? I haven't used it in years, so I'm honestly curious. Back when I did, it was customary to install it side-by-side with Tripwire, which essentially does only that; scan userspace and categorically log changed files based on a configurable severity depending on location (eg, /root/ is high, /var/log/messages is low)
      - spdustin 10y ago
        They really serve two different functions. Tripwire and similar tools like aide (I use aide now rather than tripwire, but that's my personal preference, I'm not an infosec domain expert) are file integrity checkers that check files for property changes (including mtime). Tuning out false alarms can result in an admin just turning off the reporting functions of the tools, that's the downside as I understand it. However, rkhunter has additional logic to specifically seek out rootkits and malware-like behavior, and is more specifically targeted to system file modifications. Combining it with unhide (to compare actual processes running with those visible from userspace) provides a reasonable assurance that nothing nefarious is going on. They're all part of a spectrum, however. I use scanners like aide alongside rkhunter as well, but I'm the sort of guy that will spend a day tuning the config of aide to avoid constant false alarms.
scoot 10y ago
> Disclaimer I own a data center I'm curious why you think owning a DC makes you less qualified to respond? Presumably because someone that senior is less in touch with day-to-day security operations?
- rahimnathwani 10y ago
  I think they meant 'disclosure'
  - scoot 10y ago
    Possibly, but that would suggest a conflict of interest which is apparent here. More of an appeal to authority perhaps?
- morninj 10y ago
  The "disclaimer" prologue is often humble bragging. It's often less about flagging a conflict of interest and more about claiming to have authority or status.
  - scoot 10y ago
    That would be a "disclosure".

VikingCoder 10y ago

...it's a Windows box. (Just kidding.)

sredfern2 10y ago
:-)

eonw 10y ago

just a note: intelligent exploiters hide their files inside of yours, so the -mtime is useless in many cases, they will set the mtime of their upload to match the rest of the folder they hide in. command history is also easy to alter if you know what you are doing.

porker 10y ago
So the only way to detect exploits is to scan the server regularly and log the mtime and the file size, and look for changed files that shouldn't have changed? Re the command history, is there any way around them wiping it - e.g. piping all .bash_history entries to an append-only store?
- spdustin 10y ago
  RootKit Hunter [0] works pretty well on most distros to check hashes on files and other potential problems. You can also use a global bash configuration that would log all commands [1] entered into any bash shell to a central log, which could be shipped off-server simultaneously. [0]: http://rkhunter.sourceforge.net http://rkhunter.sourceforge.net [1]: http://askubuntu.com/questions/93566/how-to-log-all-bash-commands-by-all-users-on-a-server http://askubuntu.com/questions/93566/how-to-log-all-bash-com...
  - uxp 10y ago
    Does RKHunter scan userspace for mtimes? I haven't used it in years, so I'm honestly curious. Back when I did, it was customary to install it side-by-side with Tripwire, which essentially does only that; scan userspace and categorically log changed files based on a configurable severity depending on location (eg, /root/ is high, /var/log/messages is low)
    - spdustin 10y ago
      They really serve two different functions. Tripwire and similar tools like aide (I use aide now rather than tripwire, but that's my personal preference, I'm not an infosec domain expert) are file integrity checkers that check files for property changes (including mtime). Tuning out false alarms can result in an admin just turning off the reporting functions of the tools, that's the downside as I understand it. However, rkhunter has additional logic to specifically seek out rootkits and malware-like behavior, and is more specifically targeted to system file modifications. Combining it with unhide (to compare actual processes running with those visible from userspace) provides a reasonable assurance that nothing nefarious is going on. They're all part of a spectrum, however. I use scanners like aide alongside rkhunter as well, but I'm the sort of guy that will spend a day tuning the config of aide to avoid constant false alarms.

scoot 10y ago

> Disclaimer I own a data center I'm curious why you think owning a DC makes you less qualified to respond? Presumably because someone that senior is less in touch with day-to-day security operations?

rahimnathwani 10y ago
I think they meant 'disclosure'
- scoot 10y ago
  Possibly, but that would suggest a conflict of interest which is apparent here. More of an appeal to authority perhaps?
morninj 10y ago
The "disclaimer" prologue is often humble bragging. It's often less about flagging a conflict of interest and more about claiming to have authority or status.
- scoot 10y ago
  That would be a "disclosure".