7 ms·

Sony was running unpatched Apache with no firewall for months before breach

188 points by joshes 15y ago

heyrhett 15y ago
What version was it running? Can anyone point to an explanation of the exploit?
- justincormack 15y ago
  There are very few recent exploits, unless they were running on Windows, so not sure I believe this. http://httpd.apache.org/security/vulnerabilities_20.html http://httpd.apache.org/security/vulnerabilities_20.html
- muppetman 15y ago
  Here is the alledged IRC chat of hackers discussing it. No idea how real it is. http://pastebin.com/m0ZxsjAb http://pastebin.com/m0ZxsjAb
  - pyre 15y ago
    I thought it was shown that those people are discussing hacking the actual device (PS3), and not hacking PSN.
mrcharles 15y ago
I have a feeling the upcoming lawsuits against sony aren't going to go well.
phlux 15y ago
I would wonder if whomever their sys ad was, deliberately left their perimeter weak. Also, did they never do a security audit??
- mrcharles 15y ago
  I've been dealing with Sony platforms as a game developer for over a decade, and their primary method of interacting with others is one of arrogance. From sample code that doesn't work and still has japanese comments, to incorrect documentation, to requiring developers to build all their own systems, sony often doesn't seem to give a shit about the outside world. Given what I know about sony as a game developer, I would not be even remotely surprised to learn that they've never done a security audit.
  - corysama 15y ago
    I know some guys who are ex-SCEA dev support. According to them, the attitude of Sony's American and European teams was one of frustration that the Japanese headquarters are hardware guys with no interest in software. They have always been severely underfunded compared to the Xbox dev support team and they've had to make do by pushing off a lot of the work to the third parties.
    - mrcharles 15y ago
      Yeah that meshes with what I know as well. However I'm pretty sure when I started on the PS2 pre-launch, SCEA dev support didn't even exist. Sony Japan probably didn't feel it necessary. Which is probably why all our docs were in Japanese for the first three months.
      - estel 15y ago
        It's remarkable in this context how one of Sony's primary strengths for the Playstation franchise has always (certainly PS1/PS2) been having the broadest third-party support of any console out there.
        mrcharles 15y ago
        Well, this started with the PS1 because it was actually quite easy to develop for. It had good tools, the tools were cheap, and you could build your game in C. Contrast this to the N64 at the time, which had a $1000000 buy in for a developer license, or the Saturn which was, by all accounts, a nightmare to develop for that made the PS2 look like child's play. After that, the support comes down to the economics of numbers. Most devs I know would have gladly made games on Dreamcast forever, but (piracy/marketing/apathy) killed it, and the PS2 was all that was left.
JoachimSchipper 15y ago
Not part of this article: Sony ran unpatched Apache on a system actually containing sensitive data, Sony was actually hacked via unpatched Apache.
- devindotcom 15y ago
  Yeah, I saw this rumor a while back and I wasn't convinced it was related. It's like saying Area 51 had a gap in the fence. That said, it's obviously indicative of bad security practice and will likely count against them either way.
  - ZoFreX 15y ago
    Los Alamos did have a hole in the fence! Because everyone working there was a US citizen the censorship was voluntary and had limits, so Feynman was able to write a letter out describing where the hole was.
  - sabat 15y ago
    What's funny: I've been as close as you can legally get to Area 51 (right at the warning signs, cammo dudes in sight). There is no fence, surprisingly. The reason, most likely: it would have to be a really long fence. That's a lot of land they've got out there.
dirtyhand 15y ago
No phoenix firewall? pft
ZoFreX 15y ago
If I see one more article on this incident that abuses the word "firewall" I'm going to hurt someone. Surely Apache is either accessible via port 80, or it isn't. What would a firewall do to mitigate vulnerabilities in a webserver?
- muppetman 15y ago
  Look at modsecurity.org. That's what people call (rightly or wrongly) a web application firewall. You can put a bunch of rules in and if it seens certainly bad incoming requests or certain outgoing requests (all of which are configurable) it'll take whatever action you've got configured. For example, if you try and submit javascript tags to my websites, they'll just drop the connection. SQL injection attempts (at least, very obvious ones) are also logged and dropped. There are commerical hardware devices that'll do the same sort of thing modsecurity does - I guess it's being suggested Sony didn't use any, which IMHO is very stupid. If you look at the definition of firewall, modsecurity seems to fit it: "A firewall is a part of a computer system or network that is designed to block unauthorized access while permitting authorized communications." I don't think the term is being abused, just used in a way that people aren't familiar with. Most people seem to think a firewall is only a network (IP or Ethernet) level device.
  - Sephr 15y ago
    What if I want to actually submit JavaScript tags to your website such as in a plaintext comment in a blog, to help illustrate my comment? This is not how you handle input. You sanitize output, not filter/drop input. And by sanitize, I mean encode safely for the medium, not just completely block anything unsafe before encoding.
    - muppetman 15y ago
      Like any firewall, you write the correct rules for it! Allow what you want to allow and block what you don't. I was just giving examples, the rules themselves are quite complex, same as with most firewalls but even more so with the variation of good/bad code out there! It's an impossible game. That, however, wasn't my point. My point is a firewall doesn't just have to refer to a network device as the OP (seems to) suggsest.
PatrickTulskie 15y ago
An unpatched apache is hardly an apache at all.
- meatsock 15y ago
  hey that's catchy
teyc 15y ago
There is no mention of missing firewall in the report. http://republicans.energycommerce.house.gov/Media/file/Hearings/CTCP/050411/Spafford.pdf http://republicans.energycommerce.house.gov/Media/file/Heari... Quote: In the Sony case, the majority of the victims are likely young people whose sense of risk, privacy and consequence are not yet fully developed, and thus they may also not understand the full ramiﬁcations of what has happened. Presumably, both companies are large enough that they could have aﬀorded to spend an appropriate amount on security and privacy protections of their data; I have no information about what protections they had in place, although some news reports indicate that Sony was running software that was badly out of date, and had been warned about that risk.
jswanson 15y ago
I've worked in IT in Japan for a little over 5 years now. Getting people to /allow/ you to patch servers is like pulling teeth. Seriously. If the OS itself is so far out of date that you can hardly find patches for it anymore, the issue is even worse. The mere specter of something possibly breaking is usually reason enough in many people's minds to not prioritize security updates, or in some case, flat out disallow them. Sadly. Edit: keep in mind that this is anecdotal, I'm sure there are companies that patch their servers properly.
foobarbazetc 15y ago
This is bullshit. If they're running RHEL (which is likely), the version number doesn't mean anything, since RedHat back ports all security patches.
fosk 15y ago
Does anybody know what those hackers did to breach the servers?

188 points by joshes 15y ago

heyrhett 15y ago

What version was it running? Can anyone point to an explanation of the exploit?

justincormack 15y ago
There are very few recent exploits, unless they were running on Windows, so not sure I believe this. http://httpd.apache.org/security/vulnerabilities_20.html http://httpd.apache.org/security/vulnerabilities_20.html
muppetman 15y ago
Here is the alledged IRC chat of hackers discussing it. No idea how real it is. http://pastebin.com/m0ZxsjAb http://pastebin.com/m0ZxsjAb
- pyre 15y ago
  I thought it was shown that those people are discussing hacking the actual device (PS3), and not hacking PSN.

mrcharles 15y ago

I have a feeling the upcoming lawsuits against sony aren't going to go well.

phlux 15y ago

I would wonder if whomever their sys ad was, deliberately left their perimeter weak. Also, did they never do a security audit??

mrcharles 15y ago
I've been dealing with Sony platforms as a game developer for over a decade, and their primary method of interacting with others is one of arrogance. From sample code that doesn't work and still has japanese comments, to incorrect documentation, to requiring developers to build all their own systems, sony often doesn't seem to give a shit about the outside world. Given what I know about sony as a game developer, I would not be even remotely surprised to learn that they've never done a security audit.
- corysama 15y ago
  I know some guys who are ex-SCEA dev support. According to them, the attitude of Sony's American and European teams was one of frustration that the Japanese headquarters are hardware guys with no interest in software. They have always been severely underfunded compared to the Xbox dev support team and they've had to make do by pushing off a lot of the work to the third parties.
  - mrcharles 15y ago
    Yeah that meshes with what I know as well. However I'm pretty sure when I started on the PS2 pre-launch, SCEA dev support didn't even exist. Sony Japan probably didn't feel it necessary. Which is probably why all our docs were in Japanese for the first three months.
    - estel 15y ago
      It's remarkable in this context how one of Sony's primary strengths for the Playstation franchise has always (certainly PS1/PS2) been having the broadest third-party support of any console out there.
      - mrcharles 15y ago
        Well, this started with the PS1 because it was actually quite easy to develop for. It had good tools, the tools were cheap, and you could build your game in C. Contrast this to the N64 at the time, which had a $1000000 buy in for a developer license, or the Saturn which was, by all accounts, a nightmare to develop for that made the PS2 look like child's play. After that, the support comes down to the economics of numbers. Most devs I know would have gladly made games on Dreamcast forever, but (piracy/marketing/apathy) killed it, and the PS2 was all that was left.

JoachimSchipper 15y ago

Not part of this article: Sony ran unpatched Apache on a system actually containing sensitive data, Sony was actually hacked via unpatched Apache.

devindotcom 15y ago
Yeah, I saw this rumor a while back and I wasn't convinced it was related. It's like saying Area 51 had a gap in the fence. That said, it's obviously indicative of bad security practice and will likely count against them either way.
- ZoFreX 15y ago
  Los Alamos did have a hole in the fence! Because everyone working there was a US citizen the censorship was voluntary and had limits, so Feynman was able to write a letter out describing where the hole was.
- sabat 15y ago
  What's funny: I've been as close as you can legally get to Area 51 (right at the warning signs, cammo dudes in sight). There is no fence, surprisingly. The reason, most likely: it would have to be a really long fence. That's a lot of land they've got out there.

dirtyhand 15y ago

No phoenix firewall? pft

ZoFreX 15y ago

If I see one more article on this incident that abuses the word "firewall" I'm going to hurt someone. Surely Apache is either accessible via port 80, or it isn't. What would a firewall do to mitigate vulnerabilities in a webserver?

muppetman 15y ago
Look at modsecurity.org. That's what people call (rightly or wrongly) a web application firewall. You can put a bunch of rules in and if it seens certainly bad incoming requests or certain outgoing requests (all of which are configurable) it'll take whatever action you've got configured. For example, if you try and submit javascript tags to my websites, they'll just drop the connection. SQL injection attempts (at least, very obvious ones) are also logged and dropped. There are commerical hardware devices that'll do the same sort of thing modsecurity does - I guess it's being suggested Sony didn't use any, which IMHO is very stupid. If you look at the definition of firewall, modsecurity seems to fit it: "A firewall is a part of a computer system or network that is designed to block unauthorized access while permitting authorized communications." I don't think the term is being abused, just used in a way that people aren't familiar with. Most people seem to think a firewall is only a network (IP or Ethernet) level device.
- Sephr 15y ago
  What if I want to actually submit JavaScript tags to your website such as in a plaintext comment in a blog, to help illustrate my comment? This is not how you handle input. You sanitize output, not filter/drop input. And by sanitize, I mean encode safely for the medium, not just completely block anything unsafe before encoding.
  - muppetman 15y ago
    Like any firewall, you write the correct rules for it! Allow what you want to allow and block what you don't. I was just giving examples, the rules themselves are quite complex, same as with most firewalls but even more so with the variation of good/bad code out there! It's an impossible game. That, however, wasn't my point. My point is a firewall doesn't just have to refer to a network device as the OP (seems to) suggsest.

PatrickTulskie 15y ago

An unpatched apache is hardly an apache at all.

meatsock 15y ago
hey that's catchy

teyc 15y ago

There is no mention of missing firewall in the report. http://republicans.energycommerce.house.gov/Media/file/Hearings/CTCP/050411/Spafford.pdf http://republicans.energycommerce.house.gov/Media/file/Heari... Quote: In the Sony case, the majority of the victims are likely young people whose sense of risk, privacy and consequence are not yet fully developed, and thus they may also not understand the full ramiﬁcations of what has happened. Presumably, both companies are large enough that they could have aﬀorded to spend an appropriate amount on security and privacy protections of their data; I have no information about what protections they had in place, although some news reports indicate that Sony was running software that was badly out of date, and had been warned about that risk.

jswanson 15y ago

I've worked in IT in Japan for a little over 5 years now. Getting people to /allow/ you to patch servers is like pulling teeth. Seriously. If the OS itself is so far out of date that you can hardly find patches for it anymore, the issue is even worse. The mere specter of something possibly breaking is usually reason enough in many people's minds to not prioritize security updates, or in some case, flat out disallow them. Sadly. Edit: keep in mind that this is anecdotal, I'm sure there are companies that patch their servers properly.

foobarbazetc 15y ago

This is bullshit. If they're running RHEL (which is likely), the version number doesn't mean anything, since RedHat back ports all security patches.

fosk 15y ago

Does anybody know what those hackers did to breach the servers?