5 ms·

I hope you have killed and reinstalled the box because once a system has been compromised you can't ever trust it again.

michh 12y ago
In this case, if I read the article correctly, only a single account was compromised. I'm assuming the www user doesn't have sudo privileges (though I might be wrong as it shouldn't have a password either and it clearly did..). Sure there might have also been a local privilege escalation vulnerability, so rootkits are definitely something to check for. Depending on the situation, re-installing might be less work, so then it'd make sense. But especially if you're not the only one using the machine, having to reinstall the OS every time a single user fucks up and has 'password' for a password, it's going to get really tedious really quickly.
- artursapek 12y ago
  I made sure. www has no privileges. It can't read root's home directory or touch any of its processes. If it could, I'm sure I would be busy cleaning up a much larger mess instead of blogging about it.
  - kybernetyk 12y ago
    I'm not trying to imply you didn't check thoroughly because I don't know the whole story and your background. But for me personally I wouldn't take the risk - even if the compromised account was 'nobody'. I certainly don't know enough about computer forensics to make sure that a system hasn't been altered (and I dissect malware for recreational purposes). So unless it's a system that hosts something completely unimportant I'd take the weekend free to wipe + reinstall the stuff. Even recently a linux local privilege escalation has been discovered[0]. So who knows what the hacked www user really had access to. [0] http://seclists.org/oss-sec/2014/q2/467 http://seclists.org/oss-sec/2014/q2/467
- jon-wood 12y ago
  > But especially if you're not the only one using the machine, having to reinstall the OS every time a single user fucks up and has 'password' for a password, it's going to get really tedious really quickly. This is what automated installs are for. Kill the box, fire up a new one, and run the provisioning scripts. As a bonus you've got actual documentation showing how the box should be set up.
  - michh 12y ago
    If an attacker can get elevated privileges through an user account, the users themselves can do so as well. And you shouldn't trust them either, so you need to assume they have if they could have. That means definitely reinstall when you find out you've been vulnerable to e.g. a local root exploit, regardless of user accounts have been hacked.
jnbiche 12y ago
I'm always amazed at the number of people who simply re-use servers that have been owned. It explains a lot.
artursapek 12y ago
I know. I'm taking the risk with it for a few more days because I was already in the process of moving those apps to fresh boxes (the other app is http://mondrian.io http://mondrian.io). I'd rather not take them down because they get considerable traffic. At this point I'm just keeping the server alive until I can fully replace it - not using it to do anything new. I did immediately change www's password again and de-privilege the ssh key that that box had been equipped with. In fact I've recently started always ssh-ing into servers with the -A flag so I don't have to give them keys at all. So far I don't see any more mysterious activity. Thanks for the word of caution.
- ceffry 12y ago
  Another word of caution. If someone can gain root access on a machine you log into with the -A flag they can "borrow" your authentication agent and use your keys. It's only borrowing because the connection to the agent disappears once you log off. But by then they could already have added new keys for themselves on your other severs.

michh 12y ago

In this case, if I read the article correctly, only a single account was compromised. I'm assuming the www user doesn't have sudo privileges (though I might be wrong as it shouldn't have a password either and it clearly did..). Sure there might have also been a local privilege escalation vulnerability, so rootkits are definitely something to check for. Depending on the situation, re-installing might be less work, so then it'd make sense. But especially if you're not the only one using the machine, having to reinstall the OS every time a single user fucks up and has 'password' for a password, it's going to get really tedious really quickly.

artursapek 12y ago
I made sure. www has no privileges. It can't read root's home directory or touch any of its processes. If it could, I'm sure I would be busy cleaning up a much larger mess instead of blogging about it.
- kybernetyk 12y ago
  I'm not trying to imply you didn't check thoroughly because I don't know the whole story and your background. But for me personally I wouldn't take the risk - even if the compromised account was 'nobody'. I certainly don't know enough about computer forensics to make sure that a system hasn't been altered (and I dissect malware for recreational purposes). So unless it's a system that hosts something completely unimportant I'd take the weekend free to wipe + reinstall the stuff. Even recently a linux local privilege escalation has been discovered[0]. So who knows what the hacked www user really had access to. [0] http://seclists.org/oss-sec/2014/q2/467 http://seclists.org/oss-sec/2014/q2/467
jon-wood 12y ago
> But especially if you're not the only one using the machine, having to reinstall the OS every time a single user fucks up and has 'password' for a password, it's going to get really tedious really quickly. This is what automated installs are for. Kill the box, fire up a new one, and run the provisioning scripts. As a bonus you've got actual documentation showing how the box should be set up.
- michh 12y ago
  If an attacker can get elevated privileges through an user account, the users themselves can do so as well. And you shouldn't trust them either, so you need to assume they have if they could have. That means definitely reinstall when you find out you've been vulnerable to e.g. a local root exploit, regardless of user accounts have been hacked.

jnbiche 12y ago

I'm always amazed at the number of people who simply re-use servers that have been owned. It explains a lot.

artursapek 12y ago

I know. I'm taking the risk with it for a few more days because I was already in the process of moving those apps to fresh boxes (the other app is http://mondrian.io http://mondrian.io). I'd rather not take them down because they get considerable traffic. At this point I'm just keeping the server alive until I can fully replace it - not using it to do anything new. I did immediately change www's password again and de-privilege the ssh key that that box had been equipped with. In fact I've recently started always ssh-ing into servers with the -A flag so I don't have to give them keys at all. So far I don't see any more mysterious activity. Thanks for the word of caution.

ceffry 12y ago
Another word of caution. If someone can gain root access on a machine you log into with the -A flag they can "borrow" your authentication agent and use your keys. It's only borrowing because the connection to the agent disappears once you log off. But by then they could already have added new keys for themselves on your other severs.