6 ms·

Apache.org hit by targeted XSS attack, passwords compromised

29 points by cedsav 16y ago

cedsav 16y ago
A real case illustrating what Jerf helpfully explained to me just 45 days ago: http://news.ycombinator.com/item?id=1154379 http://news.ycombinator.com/item?id=1154379
dfranke 16y ago
Today is the day that tptacek gets to run around screaming "I told you so" :-). I don't remember when or why I created it, but I have an account on the compromised server. It was a non-unique password, but fortunately one that I've never used on any account of value, and one that I rotated out a while ago on every account that I ever use at all.
brown9-2 16y ago
The Apache team was nice enough to be pro-active and send me an email alert: You are receiving this email because you have a login, '...', on the Apache JIRA installation, https://issues.apache.org/jira/ https://issues.apache.org/jira/ On April 6 the issues.apache.org server was hacked. The attackers were able to install a trojan JIRA login screen and later get full root access: https://blogs.apache.org/infra/entry/apache_org_04_09_2010 https://blogs.apache.org/infra/entry/apache_org_04_09_2010 We are assuming that the attackers have a copy of the JIRA database, which includes a hash (SHA-512 unsalted) of the password you set when signing up as '...' to JIRA. If the password you set was not of great quality (eg. based on a dictionary word), it should be assumed that the attackers can guess your password from the password hash via brute force. The upshot is that someone malicious may know both your email address and a password of yours. This is a problem because many people reuse passwords across online services. If you reuse passwords across systems, we urge you to change your passwords on ALL SYSTEMS that might be using the compromised JIRA password. Prime examples might be gmail or hotmail accounts, online banking sites, or sites known to be related to your email's domain, gmail.com.
- huherto 16y ago
  I also got the same letter. But I haven't used it in years. Are you supposed to use your email to log in? I need to find out which password I was using in this account.
  - brown9-2 16y ago
    the actual email contained my login/username, I assume you could just use that
- kanwisher 16y ago
  UGH got the same email today, not fun!
tptacek 16y ago
A little disturbing that Slicehost wasn't responsive to Apache.org telling them they had a compromised image.
- justinsb 16y ago
  Really? * Passwords hashed with SHA-512 _unsalted_ * No lockout / notification after hundreds of thousands of failed logins * XSS vulnerability * Able to change the configuration to upload executable scripts * SSH password authentication enabled And your top concern was that Slicehost didn't immediately shut down a machine? Would it have really slowed the hackers down for more than an hour anyway?
  - pquerna 16y ago
    fwiw, 4/5 of your points are specific issues to Atlassian JIRA, a commercial product :) The 5th point, as explained in the post... We actually tried to disable Password based authentication -- It is the ASF's standard policy for it to be disabled! -- we just failed at testing the configuration on Brutus. On our other machines, we had a block of configuration options we appended to the END of the sshd configuration, and these options successfully turned off password authentication, requiring ssh keys only. On brutus however, we were missing a "UsePAM no", which meant pam went in and fell back to password authentication. This was a mistake, and we didn't realize until after the vulnerability when we were testing the machine, that its sshd was misconfigured.
    - justinsb 16y ago
      I didn't mean to criticize. There were mistakes made - there always are. But amongst the list of mistakes, I would not put Slicehost's slow response anywhere near the top, and I'm surprised that anyone (particularly someone whose opinions I normally respect) could choose that as the lesson to highlight. Anyway, when is Apache going to get serious about a bug tracking project? :-)
    - epochwolf 16y ago
      * Able to change the configuration to upload executable scripts Not familiar with JIRA but... couldn't you prevent this by either changing the ownership of the uploaded files or configure the web server not to execute any files in uploaded files directory.
gojomo 16y ago
Every shortened URL is a dangerous URL!
rapind 16y ago
To me this seems pretty frikin huge. The number of sysadmins and developer's using bugzilla, jira, confluence... I would assume a good number of them are using very crackable passwords (common words etc.), even if it's just a minority, which could then expose countless other systems managed by each of those sysadmins / devs who had their password hacked, because they probably also use a rotation of 1-3 passwords for other services... - or - Am I just on crack?
- gacba 16y ago
  I told them not to mess with their .htaccess unless they knew what they were doing!
ErrantX 16y ago
I always find it fascinating how fast security falls when you find the one weak spot. They were lucky to spot them so fast (relatively) I'm sure minotaur would have fallen with time too.
- epochwolf 16y ago
  This wasn't the case of a single weak spot. There where numerous issues with their security setup on this server. See http://news.ycombinator.com/item?id=1262784 http://news.ycombinator.com/item?id=1262784 for a short list.
  - ErrantX 16y ago
    It's a lot lot easier to exploit those weaknesses when your inside some of them. The XSS exploit was like the stopper in the dam.
jdc 16y ago
"The attack was crafted to steal the session cookie from the user logged-in to JIRA. When this issue was opened against the Infrastructure team, several of our administators clicked on the link. This compromised their sessions, including their JIRA administrator rights." I may be wrong here, but isn't using a session cookie to authenticate a login which allows one to upload files to an executable directory a bad idea? Wouldn't asking for the user's password again when the he needs higher privileges (a la popular bulletin board software) stop the whole attack in its tracks?

29 points by cedsav 16y ago

A real case illustrating what Jerf helpfully explained to me just 45 days ago: http://news.ycombinator.com/item?id=1154379 http://news.ycombinator.com/item?id=1154379

dfranke 16y ago

Today is the day that tptacek gets to run around screaming "I told you so" :-). I don't remember when or why I created it, but I have an account on the compromised server. It was a non-unique password, but fortunately one that I've never used on any account of value, and one that I rotated out a while ago on every account that I ever use at all.

brown9-2 16y ago

The Apache team was nice enough to be pro-active and send me an email alert: You are receiving this email because you have a login, '...', on the Apache JIRA installation, https://issues.apache.org/jira/ https://issues.apache.org/jira/ On April 6 the issues.apache.org server was hacked. The attackers were able to install a trojan JIRA login screen and later get full root access: https://blogs.apache.org/infra/entry/apache_org_04_09_2010 https://blogs.apache.org/infra/entry/apache_org_04_09_2010 We are assuming that the attackers have a copy of the JIRA database, which includes a hash (SHA-512 unsalted) of the password you set when signing up as '...' to JIRA. If the password you set was not of great quality (eg. based on a dictionary word), it should be assumed that the attackers can guess your password from the password hash via brute force. The upshot is that someone malicious may know both your email address and a password of yours. This is a problem because many people reuse passwords across online services. If you reuse passwords across systems, we urge you to change your passwords on ALL SYSTEMS that might be using the compromised JIRA password. Prime examples might be gmail or hotmail accounts, online banking sites, or sites known to be related to your email's domain, gmail.com.

huherto 16y ago
I also got the same letter. But I haven't used it in years. Are you supposed to use your email to log in? I need to find out which password I was using in this account.
- brown9-2 16y ago
  the actual email contained my login/username, I assume you could just use that
kanwisher 16y ago
UGH got the same email today, not fun!

tptacek 16y ago

A little disturbing that Slicehost wasn't responsive to Apache.org telling them they had a compromised image.

justinsb 16y ago
Really? * Passwords hashed with SHA-512 _unsalted_ * No lockout / notification after hundreds of thousands of failed logins * XSS vulnerability * Able to change the configuration to upload executable scripts * SSH password authentication enabled And your top concern was that Slicehost didn't immediately shut down a machine? Would it have really slowed the hackers down for more than an hour anyway?
- pquerna 16y ago
  fwiw, 4/5 of your points are specific issues to Atlassian JIRA, a commercial product :) The 5th point, as explained in the post... We actually tried to disable Password based authentication -- It is the ASF's standard policy for it to be disabled! -- we just failed at testing the configuration on Brutus. On our other machines, we had a block of configuration options we appended to the END of the sshd configuration, and these options successfully turned off password authentication, requiring ssh keys only. On brutus however, we were missing a "UsePAM no", which meant pam went in and fell back to password authentication. This was a mistake, and we didn't realize until after the vulnerability when we were testing the machine, that its sshd was misconfigured.
  - justinsb 16y ago
    I didn't mean to criticize. There were mistakes made - there always are. But amongst the list of mistakes, I would not put Slicehost's slow response anywhere near the top, and I'm surprised that anyone (particularly someone whose opinions I normally respect) could choose that as the lesson to highlight. Anyway, when is Apache going to get serious about a bug tracking project? :-)
  - epochwolf 16y ago
    * Able to change the configuration to upload executable scripts Not familiar with JIRA but... couldn't you prevent this by either changing the ownership of the uploaded files or configure the web server not to execute any files in uploaded files directory.

gojomo 16y ago

Every shortened URL is a dangerous URL!

rapind 16y ago

To me this seems pretty frikin huge. The number of sysadmins and developer's using bugzilla, jira, confluence... I would assume a good number of them are using very crackable passwords (common words etc.), even if it's just a minority, which could then expose countless other systems managed by each of those sysadmins / devs who had their password hacked, because they probably also use a rotation of 1-3 passwords for other services... - or - Am I just on crack?

gacba 16y ago
I told them not to mess with their .htaccess unless they knew what they were doing!

ErrantX 16y ago

I always find it fascinating how fast security falls when you find the one weak spot. They were lucky to spot them so fast (relatively) I'm sure minotaur would have fallen with time too.

epochwolf 16y ago
This wasn't the case of a single weak spot. There where numerous issues with their security setup on this server. See http://news.ycombinator.com/item?id=1262784 http://news.ycombinator.com/item?id=1262784 for a short list.
- ErrantX 16y ago
  It's a lot lot easier to exploit those weaknesses when your inside some of them. The XSS exploit was like the stopper in the dam.

jdc 16y ago

"The attack was crafted to steal the session cookie from the user logged-in to JIRA. When this issue was opened against the Infrastructure team, several of our administators clicked on the link. This compromised their sessions, including their JIRA administrator rights." I may be wrong here, but isn't using a session cookie to authenticate a login which allows one to upload files to an executable directory a bad idea? Wouldn't asking for the user's password again when the he needs higher privileges (a la popular bulletin board software) stop the whole attack in its tracks?