4 ms·

"It then replaced every compromised computer and set up new defenses in hopes of keeping hackers out." I hope that's just poor reporting, or does the Times' IT

by sc00ter 14y ago

"It then replaced every compromised computer and set up new defenses in hopes of keeping hackers out." I hope that's just poor reporting, or does the Times' IT department really have that poor an understanding of how computers work? No wonder they got pwned. And I'm not buying the "we gave them free reign for four months on purpose" line. It makes no sense.

damian2000 14y ago
Maybe they actually meant every compromised computer OS (i.e. hard disk)?
- onedognight 14y ago
  Maybe they mean OS and BIOS?
  - beagle3 14y ago
    And network card firmware. And graphics card firmware. And in some cases, also ILO firmware, DVD drive firmware, and maybe a few other pieces. If you trust your IT supplier, and the equipment is a two years old, it's probably more economical to replace everything than try to fix it. But why would you trust your IT supplier - who sources all their stuff from China in the first place?
    - m0nastic 14y ago
      All of those things are technically true, but don't match up with the M.O. of the perpetrators in question (they're not actually using any super-fancy BIOS rootkits). Also, the remediation process is exactly that, a process. It involves a pre-planned, direct remediation effort at a specific time, after which, egress traffic is monitored to look for any other outbound connections that pop up that were missed in the first "sweep." Passwords are all changed. You "rinse, lather, repeat" that process until you stop seeing the communications. It can take several times before you sound an "all-clear".
      - btilly 14y ago
        The MO is that they are a state attacker, not a one trick pony. Until proven otherwise, you should assume that in addition to what you know they did, they did everything you can think of that is within their known capabilities. Super fancy BIOS rootkits are not outside their known capabilities. Also monitoring egress traffic is easier said than done. For example you could have a special gmail account that you connect to over https while the user is actively using the computer. This account receives and sends commands as email messages. Since it is normal for that computer to connect to gmail, and the connection is normally encrypted, that communications channel could be rather hard to detect.
        m0nastic 14y ago
        Sorry, I should have stated that better. I'm not talking about, "best practice for an advanced attack from an unknown perpetrator". I absolutely agree with you in general. I'm saying that, this particular attacker, is a known, identifiable actor. They have names, they have huge reams of files in manilla envelopes. If you are privy, you get to know their actual names, see photos of them. They have a very specific methodology which they use. You use that methodology to determine the extent of what they've accomplished. This specific actor is not using any super fancy BIOS rootkits. If one of their campaigns ever gets to the point that they are unable to obtain repeatable persistent access, that campaign is sent to a different actor (with a completely different big manilla envelope about) who will then attempt a more advanced campaign. The tools and tactics for these intrusions are very specific to the actual actors responsible.
        btilly 14y ago
        Your reply has a lot of very specific information about the attack, attacker, and state of the compromise that I find rather dubious and cannot find in the article. Do you have a source that you would care to share? Furthermore in this case we've been told that the attacker managed to achieve a rather thorough compromise of the network. And managed to persist through multiple attempts to remove them. Even if the attack proceeded by the rules that you describe, it would be foolhardy to assume that they were not subjected to the advanced campaign.
beagle3 14y ago
Someone has poor understanding of how computers work, but it isn't necessarily the NY Times. Once a computer is compromised, you can't trust anything about it. You may believe reinstalling the OS is enough, but it is possible that some remote control tool is still lurking in a main BIOS reflashed while compromised, or in the GPU firmware, or tens of other places. While it should potentially be possible to reflash everything, it is practically cheaper to replace the computers. Do YOU know how to reflash your bios with a trusted version, your GPU firmware, etc? I don't mean "I know how to look it up on Google, and I'm sure I can do it if needed". This thing is hard to automate and do at scale even if you do know how to do it, especially if not all your computer models are uniform. Depending on how old and varied the hardware is, it is very likely that the economical solution, (assuming you suspect an attacker capable of these feats), is to replace all the computers. [Though, all the hardware they replaced it with has been, most likely, built and QAd in China. Why would you trust _that_? The rabbit hole goes very deep. Practically too deep for anyone without a billion dollar R&D budget these days]
- sc00ter 14y ago
  You may believe reinstalling the OS is enough I made no such claim, but verifying bios and firmware signatures (and indeed detecting changes when they happen), and reinstalling them at scale is not a major challenge with a well managed IT infrastructure. I can accept however that the Times may well have been running 10 year old PCs, with manual IT management processes, and outdated security software, and that replacement may have been overdue and economically more viable.
  - beagle3 14y ago
    > verifying bios and firmware signatures (and indeed detecting changes when they happen), and reinstalling them at scale is not a major challenge with a well managed IT infrastructure. Can you back up that claim with reference to a system that does that? EVERY single management system I can think of trusts the system to report its status. You can't trust a compromised system to report its status. Assume you have 5,000 desktop computers. How do you set them up so you can verify bios and firmware signatures without forcing a good bios reflash in the first place? (An action that does require soldering or jumper setting on modern motherboards!) > I can accept however that the Times may well have been running 10 year old PCs If you're running your business properly, 3-4 years is the oldest any PC should ever get. If you know a business running 10 year old PCs, tell them to get a new accountant. Today's $300 ATOM netbook (with your 10 year old screen and keyboard) will have positive ROI compared to maintaining a 10 year old machine (The best 2002 Pentium 4 is comparable to a modern ATOM, but needs 5-10 times as much power). You'll be saving money just with energy/cooling costs.
untog 14y ago
And I'm not buying the "we gave them free reign for four months on purpose" line. It makes no sense. There's a sort of interesting journalistic gamble at work there. If you're confident that your backup systems are untouchable and you're able to track exactly what is going on, you could gather evidence for a truly ground-breaking story. Unfortunately like usual there's no iron-clad evidence that the Chinese government is behind the hacking, so it's not the story it could have been.
sillysaurus 14y ago
"It then replaced every compromised computer and set up new defenses in hopes of keeping hackers out." I hope that's just poor reporting, or does the Times' IT department really have that poor an understanding of how computers work? The audience of NYT understands "We replaced the compromised computers." Whereas the audience won't understand "We rebuilt each compromised computer, taking care to ensure they weren't exposed to the internet (and, additionally, were isolated from our internal network) until we were certain we'd patched every security flaw the attackers had previously exploited. We've also made policy changes to minimize the attack surface of our new infrastructure." Therefore, those are wasted words. In fact, those words are an unnecessary risk. It's a risk because it's a strategic mistake for a newspaper to publish confusing articles.

by sc00ter 14y ago

"It then replaced every compromised computer and set up new defenses in hopes of keeping hackers out." I hope that's just poor reporting, or does the Times' IT department really have that poor an understanding of how computers work? No wonder they got pwned. And I'm not buying the "we gave them free reign for four months on purpose" line. It makes no sense.

damian2000 14y ago

Maybe they actually meant every compromised computer OS (i.e. hard disk)?

onedognight 14y ago
Maybe they mean OS and BIOS?
- beagle3 14y ago
  And network card firmware. And graphics card firmware. And in some cases, also ILO firmware, DVD drive firmware, and maybe a few other pieces. If you trust your IT supplier, and the equipment is a two years old, it's probably more economical to replace everything than try to fix it. But why would you trust your IT supplier - who sources all their stuff from China in the first place?
  - m0nastic 14y ago
    All of those things are technically true, but don't match up with the M.O. of the perpetrators in question (they're not actually using any super-fancy BIOS rootkits). Also, the remediation process is exactly that, a process. It involves a pre-planned, direct remediation effort at a specific time, after which, egress traffic is monitored to look for any other outbound connections that pop up that were missed in the first "sweep." Passwords are all changed. You "rinse, lather, repeat" that process until you stop seeing the communications. It can take several times before you sound an "all-clear".
    - btilly 14y ago
      The MO is that they are a state attacker, not a one trick pony. Until proven otherwise, you should assume that in addition to what you know they did, they did everything you can think of that is within their known capabilities. Super fancy BIOS rootkits are not outside their known capabilities. Also monitoring egress traffic is easier said than done. For example you could have a special gmail account that you connect to over https while the user is actively using the computer. This account receives and sends commands as email messages. Since it is normal for that computer to connect to gmail, and the connection is normally encrypted, that communications channel could be rather hard to detect.
      - m0nastic 14y ago
        Sorry, I should have stated that better. I'm not talking about, "best practice for an advanced attack from an unknown perpetrator". I absolutely agree with you in general. I'm saying that, this particular attacker, is a known, identifiable actor. They have names, they have huge reams of files in manilla envelopes. If you are privy, you get to know their actual names, see photos of them. They have a very specific methodology which they use. You use that methodology to determine the extent of what they've accomplished. This specific actor is not using any super fancy BIOS rootkits. If one of their campaigns ever gets to the point that they are unable to obtain repeatable persistent access, that campaign is sent to a different actor (with a completely different big manilla envelope about) who will then attempt a more advanced campaign. The tools and tactics for these intrusions are very specific to the actual actors responsible.
        btilly 14y ago
        Your reply has a lot of very specific information about the attack, attacker, and state of the compromise that I find rather dubious and cannot find in the article. Do you have a source that you would care to share? Furthermore in this case we've been told that the attacker managed to achieve a rather thorough compromise of the network. And managed to persist through multiple attempts to remove them. Even if the attack proceeded by the rules that you describe, it would be foolhardy to assume that they were not subjected to the advanced campaign.

beagle3 14y ago

Someone has poor understanding of how computers work, but it isn't necessarily the NY Times. Once a computer is compromised, you can't trust anything about it. You may believe reinstalling the OS is enough, but it is possible that some remote control tool is still lurking in a main BIOS reflashed while compromised, or in the GPU firmware, or tens of other places. While it should potentially be possible to reflash everything, it is practically cheaper to replace the computers. Do YOU know how to reflash your bios with a trusted version, your GPU firmware, etc? I don't mean "I know how to look it up on Google, and I'm sure I can do it if needed". This thing is hard to automate and do at scale even if you do know how to do it, especially if not all your computer models are uniform. Depending on how old and varied the hardware is, it is very likely that the economical solution, (assuming you suspect an attacker capable of these feats), is to replace all the computers. [Though, all the hardware they replaced it with has been, most likely, built and QAd in China. Why would you trust _that_? The rabbit hole goes very deep. Practically too deep for anyone without a billion dollar R&D budget these days]

sc00ter 14y ago
You may believe reinstalling the OS is enough I made no such claim, but verifying bios and firmware signatures (and indeed detecting changes when they happen), and reinstalling them at scale is not a major challenge with a well managed IT infrastructure. I can accept however that the Times may well have been running 10 year old PCs, with manual IT management processes, and outdated security software, and that replacement may have been overdue and economically more viable.
- beagle3 14y ago
  > verifying bios and firmware signatures (and indeed detecting changes when they happen), and reinstalling them at scale is not a major challenge with a well managed IT infrastructure. Can you back up that claim with reference to a system that does that? EVERY single management system I can think of trusts the system to report its status. You can't trust a compromised system to report its status. Assume you have 5,000 desktop computers. How do you set them up so you can verify bios and firmware signatures without forcing a good bios reflash in the first place? (An action that does require soldering or jumper setting on modern motherboards!) > I can accept however that the Times may well have been running 10 year old PCs If you're running your business properly, 3-4 years is the oldest any PC should ever get. If you know a business running 10 year old PCs, tell them to get a new accountant. Today's $300 ATOM netbook (with your 10 year old screen and keyboard) will have positive ROI compared to maintaining a 10 year old machine (The best 2002 Pentium 4 is comparable to a modern ATOM, but needs 5-10 times as much power). You'll be saving money just with energy/cooling costs.

untog 14y ago

And I'm not buying the "we gave them free reign for four months on purpose" line. It makes no sense. There's a sort of interesting journalistic gamble at work there. If you're confident that your backup systems are untouchable and you're able to track exactly what is going on, you could gather evidence for a truly ground-breaking story. Unfortunately like usual there's no iron-clad evidence that the Chinese government is behind the hacking, so it's not the story it could have been.

sillysaurus 14y ago

"It then replaced every compromised computer and set up new defenses in hopes of keeping hackers out." I hope that's just poor reporting, or does the Times' IT department really have that poor an understanding of how computers work? The audience of NYT understands "We replaced the compromised computers." Whereas the audience won't understand "We rebuilt each compromised computer, taking care to ensure they weren't exposed to the internet (and, additionally, were isolated from our internal network) until we were certain we'd patched every security flaw the attackers had previously exploited. We've also made policy changes to minimize the attack surface of our new infrastructure." Therefore, those are wasted words. In fact, those words are an unnecessary risk. It's a risk because it's a strategic mistake for a newspaper to publish confusing articles.