7 ms·

It does not seem like this story is true. If it's true, it makes absolutely no sense for Apple and Amazon to attack Bloomberg. Sure, a national security letter

by lacker 8y ago

It does not seem like this story is true. If it's true, it makes absolutely no sense for Apple and Amazon to attack Bloomberg. Sure, a national security letter could force them to stay quiet, or maybe even to lie to the public and say it didn't happen, but it can't make them criticize Bloomberg. Attacking Bloomberg if the story is true is only going to convince Bloomberg to dig deeper. And the story isn't even that bad for Amazon or Apple - it's much worse for US-China relations than it is for either of those companies. The key technical detail of what these chips are allegedly doing also does not make sense. From the article: the chips allowed the attackers to create a stealth doorway into any network that included the altered machines How can you get around a firewall by using a compromised machine that's part of the internal network? I don't think Bloomberg reporters are just making stuff up. But the technical confusion here makes me suspect that the government officials who leaked this story just didn't understand the details of a real incident that happened, and in the leaking the story got mangled into inaccuracy.

bubblethink 8y ago
I find the lack of real technical information quite surprising, not just on bloomberg, but here as well. The previous big thread that you can scroll endlessly has anecdotes and chit-chat but nothing on the actual technical details.
QuinnyPig 8y ago
It goes beyond that. If this story were in fact true, what we're witnessing is large companies whose entire business is built upon user trust setting fire to themselves. Who'd ever trust AWS again with anything of consequence if it turns out the denial is false? I don't buy it as written. There's something else here.
- deleted 8y ago
  [deleted]
- addicted 8y ago
  What would lead to more of a loss of trust? Revealing to your customers that you hid a massive hack for a few years (even if the government required you to) or insisting that Bloomberg is lying, when you know all the other relevant players will also insist on the same.
- ethbro 8y ago
  How would one falsify a denial?
  - lacker 8y ago
    The article claims there is an email trail about this. Those communications could be leaked - that would make it pretty obvious this story is true, if it is. There could also be simply a picture of the spy chip! It's pretty easy to have stronger evidence than this article has.
    - ethbro 8y ago
      Both of those are in control of the companies in question. I think many comments have gotten far too conspiracy theoryish, but I can't think of any plausible answers to "If they were lying, how would we know?"
- Jach 8y ago
  > Who'd ever trust AWS again with anything of consequence if it turns out the denial is false? I agree it makes little sense to attack and deny rather than just stay quiet or keep it to a minimal PR blurb. But I think you're overestimating the amount of blowback they would get if they're lying. Most people just wouldn't care. It might be the decisive factor in a bigger company deciding where to put something new and them going to Oracle Metal or something, but I am really skeptical their bottom line would suffer because of this one thing.
  - 394549 8y ago
    Also, almost everyone who lies thinks they'll get away with it. Otherwise, why lie? Maybe AWS and Apple are counting on the top-secret nature of all this to enable their lies. Would US intelligence agencies really come out on the record to contradict the denials and confirm the hack if it meant destroying the credibility of a couple of American companies? The Bloomberg article detailed how the government was not willing to sound the same alarms against an American company as they were against Huawei and ZTE for just that reason. I'm guessing that creates enough ambiguity for false denials to be effective.
    - morpheuskafka 8y ago
      Yeah I think people don’t understand how big of a deal this would be. Remember Joseph McCarthy? Now picture that, in today’s politics, only with even a shred of real evidence that US companies and the military were infiltrated. That’s an act of war between two nuclear powers. There is literally no telling what kind of chaos could follow. I’m not saying it’s guranteed that this story is good—-for one, I find the glossed over technical details a little questionable. But it is definitely plausible that this is a false denial, under orders or not. Regardless, two points are very clear. Remember Stallman who uses that old X60 with a free BIOS? We all called him crazy when he insisted on not having management firmwares and secretive back doors. Now we have a story claiming they were used in an attack by a US adversary (as well as a long chain of confirmed recent security flaws). The second point is that we have to stop trusting China as the sole source for all electronics. It’s a national security crisis that there isn’t a single US facility making many of the components needed for basic computing platform. The federal government needs to immediately and strongly incentive more domestic foundries. And not the trump style “here be factory, jobs good” but actual targeted planning to bring the whole supply chain back, not just the final assembly everyone talks about.
parliament32 8y ago
It depends. These chips were specifically connected to the BMC/IPMI: >The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off. Is your OOB-management VLAN/network firewalled off from outgoing access to the internet? If not, this chip can make requests out to another host, who gives it code to run. This code can then either collect data and send it back out, or do other nefarious things.
- bubblethink 8y ago
  The BMC is always on a maintenance vlan for any organization worth its salt. The software stack on those things doesn't deserve to see the light of the day. If you open it up to the internet, you will get pwned, china or not. So there seems to be something missing. It can't just be a compromised BMC. Edit: Now that I think of it, a compromised BMC can inject payloads into the running OS, depending on the sophistication of the attack and the defenses of the OS. So it does seem possible.
  - parliament32 8y ago
    Keep in mind that the BCM already has the ability to piggyback on the production interface if it decides the dedicated interface isn't connected to anything... what's to stop it from doing this in a mailicious scenario and fire off some garbage-looking dns/icmp/whatever?
- jeletonskelly 8y ago
  Unless you're air-gapped you can usually assume that if the machines are syncing to public NTP then there's a way in and out.
  - pixl97 8y ago
    How many organizations actually watch what is coming in and out of their DNS?
Thriptic 8y ago
> How can you get around a firewall by using a compromised machine that's part of the internal network? Reverse shell depending on how you have the firewall set up. Most firewalls don't block outbound packets, and even if they do you can tunnel over an open port. > If it's true, it makes absolutely no sense for Apple and Amazon to attack Bloomberg. If you are a major company, are you going to use AWS if you believe their hardware is pwned?
- pixl97 8y ago
  Almost evreything has DNS resolvers setup. How many places verify their DNS queries are not full of evil?
jcmeyrignac 8y ago
I believe it's possible. This story reminded me about Lenovo's drivers automatic installation: https://www.theregister.co.uk/2015/08/12/lenovo_firmware_nasty/ https://www.theregister.co.uk/2015/08/12/lenovo_firmware_nas... And yes, I had the problem with my Lenovo desktop.
monksy 8y ago
You can get around that firewall if the firewall is compromised as well.
- late2part 8y ago
  Turtles all the way down...
wetpaws 8y ago
>If it's true, it makes absolutely no sense for Apple and Amazon to attack Bloomberg. It makes a perfect sense to protect from a potential devastating blow to stock price.
- 21 8y ago
  There would be no blow. China (or the NSA) hacking into something is non-news. Everybody expects that, especially since Snowden. It's their job, and the news would be them failing to do it.
- astrange 8y ago
  What you're describing is illegal.
Hikikomori 8y ago
It seems that it required internet access to download additional code, if it didn't have that it couldn't do much at all.
akiselev 8y ago
The actual details on the hardware are also sketchy. Based on my reading of the article, this wasn't a chip swap where one chip is replaced by a backdoored version. The article implies that this was an extra chip so either dozens (possibly hundreds) of engineers were in on the operation from the beginning to slip the chip into the design undetected or the chip was mounted without any changes to the board design. The former is much riskier than backdooring the chips and the latter, as far as I know, has not been done before with a nontrivial chip. Fitting any chip capable of exfiltrating a nontrivial amount of data onto a modern motherboard without going through many rounds of simulation or significantly impacting performance, while also putting it in a place it is capable of intercepting valuable data is practically impossible. Hell, just getting the right power domains wired to the chip is going to be tough enough.
- throwaway2048 8y ago
  Placing it on a low speed SPI read/write trace to the flash chip containing the BMC firmware is entirely plausible, and in no way would require "dozens (possibly hundreds) of engineers." EDIT: https://news.ycombinator.com/item?id=18138638 https://news.ycombinator.com/item?id=18138638
  - akiselev 8y ago
    It's entirely plausible as long as no one is inspecting the final product. The hard part isn't connecting the traces, it's doing so without immediately arousing suspicion when a tech open the server and sees a ghetto engineered chip hanging off the PCB at a weird angle. I hadn't read the detail about the second flash chip. Obviously, if they're just putting compromised flash into a footprint left empty by the manufacturer on purpose for recovery, it's much easier, cleaner, and can be done with a small group.
  - calandoa 8y ago
    Similar attacks on the BMC firmware have been described here, with the same AST2400 BMC chip: https://blog.eclypsium.com/2018/09/06/insecure-firmware-updates-in-server-management-systems/ https://blog.eclypsium.com/2018/09/06/insecure-firmware-upda... https://blog.eclypsium.com/2018/06/07/firmware-vulnerabilities-in-supermicro-systems/ https://blog.eclypsium.com/2018/06/07/firmware-vulnerabiliti...
tanilama 8y ago
That is my question as well, without getting their hands on the machine, how do the attackers start the attacks? And in case of AWS, since everything is so virtualized, does it make sense to install a hardware backdoor if it cant even map to your true target？ It could probably make more sense as a backdoor when the hackers get hold the access to the physical device, it is however quite incredible to me that this hack is designed for a remote network access, and could go unnoticed from infosec within the company if it is truly sending packets outside the firewall...
- 394549 8y ago
  > That is my question as well, without getting their hands on the machine, how do the attackers start the attacks? And in case of AWS, since everything is so virtualized, does it make sense to install a hardware backdoor if it cant even map to your true target？ Nation states have the manpower for spray-and-pray attacks. They have less need for precise targeting. > It could probably make more sense as a backdoor when the hackers get hold the access to the physical device, it is however quite incredible to me that this hack is designed for a remote network access, and could go unnoticed from infosec within the company if it is truly sending packets outside the firewall... How is it incredible? Infosec isn't perfect. Equifax was hacked in part because their IDS system was offline due to an out-of-date certificate [1]. That system would have caught the exfiltration of data if it had been active. Also, Amazon AWS must have all kinds of crazy traffic flowing through its network due to its customers. My gut feel is that it would be incredibly hard to characterize that traffic in order to proactively detect many kinds of nefarious traffic. The Bloomberg article stated they were easily able to find the traffic from these implants...but only after they knew what to look for. [1] https://www.bankinfosecurity.com/postmortem-behind-equifax-breach-multiple-failures-a-11480 https://www.bankinfosecurity.com/postmortem-behind-equifax-b...
AnthonyMouse 8y ago
> And the story isn't even that bad for Amazon or Apple - it's much worse for US-China relations than it is for either of those companies. If true, it's pretty bad. For one thing customers will be more wary of buying hardware from China. If they move manufacturing elsewhere (or, for Amazon, source hardware elsewhere) they would have higher costs and lower margins, if they don't and a competitor does they lose business. On top of that, people are already wary of the privacy implications of these companies having all your data, but most are willing to trust major US companies. If it looks like they can't keep it secure against random Chinese companies engaged in industrial espionage, more customers will conclude that operating servers in-house is the better choice.
- Swenrekcah 8y ago
  Silver linong is it’s probably better for both IT people and the economy as a whole that companies generally operate their own servers.
  - jermy 8y ago
    Unless we use Supermicro kit, presumably?
    - AnthonyMouse 8y ago
      Unless we use made-in-China kit, presumably. It wouldn't be that surprising to see Supermicro move their production back out of China after this. Though they've been having a hard time of it recently already -- something about getting delisted from the NASDAQ. I wonder if the two incidents are related at all?

by lacker 8y ago

bubblethink 8y ago

I find the lack of real technical information quite surprising, not just on bloomberg, but here as well. The previous big thread that you can scroll endlessly has anecdotes and chit-chat but nothing on the actual technical details.

QuinnyPig 8y ago

It goes beyond that. If this story were in fact true, what we're witnessing is large companies whose entire business is built upon user trust setting fire to themselves. Who'd ever trust AWS again with anything of consequence if it turns out the denial is false? I don't buy it as written. There's something else here.

deleted 8y ago
[deleted]
addicted 8y ago
What would lead to more of a loss of trust? Revealing to your customers that you hid a massive hack for a few years (even if the government required you to) or insisting that Bloomberg is lying, when you know all the other relevant players will also insist on the same.
ethbro 8y ago
How would one falsify a denial?
- lacker 8y ago
  The article claims there is an email trail about this. Those communications could be leaked - that would make it pretty obvious this story is true, if it is. There could also be simply a picture of the spy chip! It's pretty easy to have stronger evidence than this article has.
  - ethbro 8y ago
    Both of those are in control of the companies in question. I think many comments have gotten far too conspiracy theoryish, but I can't think of any plausible answers to "If they were lying, how would we know?"
Jach 8y ago
> Who'd ever trust AWS again with anything of consequence if it turns out the denial is false? I agree it makes little sense to attack and deny rather than just stay quiet or keep it to a minimal PR blurb. But I think you're overestimating the amount of blowback they would get if they're lying. Most people just wouldn't care. It might be the decisive factor in a bigger company deciding where to put something new and them going to Oracle Metal or something, but I am really skeptical their bottom line would suffer because of this one thing.
- 394549 8y ago
  Also, almost everyone who lies thinks they'll get away with it. Otherwise, why lie? Maybe AWS and Apple are counting on the top-secret nature of all this to enable their lies. Would US intelligence agencies really come out on the record to contradict the denials and confirm the hack if it meant destroying the credibility of a couple of American companies? The Bloomberg article detailed how the government was not willing to sound the same alarms against an American company as they were against Huawei and ZTE for just that reason. I'm guessing that creates enough ambiguity for false denials to be effective.
  - morpheuskafka 8y ago
    Yeah I think people don’t understand how big of a deal this would be. Remember Joseph McCarthy? Now picture that, in today’s politics, only with even a shred of real evidence that US companies and the military were infiltrated. That’s an act of war between two nuclear powers. There is literally no telling what kind of chaos could follow. I’m not saying it’s guranteed that this story is good—-for one, I find the glossed over technical details a little questionable. But it is definitely plausible that this is a false denial, under orders or not. Regardless, two points are very clear. Remember Stallman who uses that old X60 with a free BIOS? We all called him crazy when he insisted on not having management firmwares and secretive back doors. Now we have a story claiming they were used in an attack by a US adversary (as well as a long chain of confirmed recent security flaws). The second point is that we have to stop trusting China as the sole source for all electronics. It’s a national security crisis that there isn’t a single US facility making many of the components needed for basic computing platform. The federal government needs to immediately and strongly incentive more domestic foundries. And not the trump style “here be factory, jobs good” but actual targeted planning to bring the whole supply chain back, not just the final assembly everyone talks about.

parliament32 8y ago

It depends. These chips were specifically connected to the BMC/IPMI: >The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off. Is your OOB-management VLAN/network firewalled off from outgoing access to the internet? If not, this chip can make requests out to another host, who gives it code to run. This code can then either collect data and send it back out, or do other nefarious things.

bubblethink 8y ago
The BMC is always on a maintenance vlan for any organization worth its salt. The software stack on those things doesn't deserve to see the light of the day. If you open it up to the internet, you will get pwned, china or not. So there seems to be something missing. It can't just be a compromised BMC. Edit: Now that I think of it, a compromised BMC can inject payloads into the running OS, depending on the sophistication of the attack and the defenses of the OS. So it does seem possible.
- parliament32 8y ago
  Keep in mind that the BCM already has the ability to piggyback on the production interface if it decides the dedicated interface isn't connected to anything... what's to stop it from doing this in a mailicious scenario and fire off some garbage-looking dns/icmp/whatever?
jeletonskelly 8y ago
Unless you're air-gapped you can usually assume that if the machines are syncing to public NTP then there's a way in and out.
- pixl97 8y ago
  How many organizations actually watch what is coming in and out of their DNS?

Thriptic 8y ago

> How can you get around a firewall by using a compromised machine that's part of the internal network? Reverse shell depending on how you have the firewall set up. Most firewalls don't block outbound packets, and even if they do you can tunnel over an open port. > If it's true, it makes absolutely no sense for Apple and Amazon to attack Bloomberg. If you are a major company, are you going to use AWS if you believe their hardware is pwned?

pixl97 8y ago
Almost evreything has DNS resolvers setup. How many places verify their DNS queries are not full of evil?

jcmeyrignac 8y ago

I believe it's possible. This story reminded me about Lenovo's drivers automatic installation: https://www.theregister.co.uk/2015/08/12/lenovo_firmware_nasty/ https://www.theregister.co.uk/2015/08/12/lenovo_firmware_nas... And yes, I had the problem with my Lenovo desktop.

monksy 8y ago

You can get around that firewall if the firewall is compromised as well.

late2part 8y ago
Turtles all the way down...

wetpaws 8y ago

>If it's true, it makes absolutely no sense for Apple and Amazon to attack Bloomberg. It makes a perfect sense to protect from a potential devastating blow to stock price.

21 8y ago
There would be no blow. China (or the NSA) hacking into something is non-news. Everybody expects that, especially since Snowden. It's their job, and the news would be them failing to do it.
astrange 8y ago
What you're describing is illegal.

Hikikomori 8y ago

It seems that it required internet access to download additional code, if it didn't have that it couldn't do much at all.

akiselev 8y ago

The actual details on the hardware are also sketchy. Based on my reading of the article, this wasn't a chip swap where one chip is replaced by a backdoored version. The article implies that this was an extra chip so either dozens (possibly hundreds) of engineers were in on the operation from the beginning to slip the chip into the design undetected or the chip was mounted without any changes to the board design. The former is much riskier than backdooring the chips and the latter, as far as I know, has not been done before with a nontrivial chip. Fitting any chip capable of exfiltrating a nontrivial amount of data onto a modern motherboard without going through many rounds of simulation or significantly impacting performance, while also putting it in a place it is capable of intercepting valuable data is practically impossible. Hell, just getting the right power domains wired to the chip is going to be tough enough.

throwaway2048 8y ago
Placing it on a low speed SPI read/write trace to the flash chip containing the BMC firmware is entirely plausible, and in no way would require "dozens (possibly hundreds) of engineers." EDIT: https://news.ycombinator.com/item?id=18138638 https://news.ycombinator.com/item?id=18138638
- akiselev 8y ago
  It's entirely plausible as long as no one is inspecting the final product. The hard part isn't connecting the traces, it's doing so without immediately arousing suspicion when a tech open the server and sees a ghetto engineered chip hanging off the PCB at a weird angle. I hadn't read the detail about the second flash chip. Obviously, if they're just putting compromised flash into a footprint left empty by the manufacturer on purpose for recovery, it's much easier, cleaner, and can be done with a small group.
- calandoa 8y ago
  Similar attacks on the BMC firmware have been described here, with the same AST2400 BMC chip: https://blog.eclypsium.com/2018/09/06/insecure-firmware-updates-in-server-management-systems/ https://blog.eclypsium.com/2018/09/06/insecure-firmware-upda... https://blog.eclypsium.com/2018/06/07/firmware-vulnerabilities-in-supermicro-systems/ https://blog.eclypsium.com/2018/06/07/firmware-vulnerabiliti...

tanilama 8y ago

That is my question as well, without getting their hands on the machine, how do the attackers start the attacks? And in case of AWS, since everything is so virtualized, does it make sense to install a hardware backdoor if it cant even map to your true target？ It could probably make more sense as a backdoor when the hackers get hold the access to the physical device, it is however quite incredible to me that this hack is designed for a remote network access, and could go unnoticed from infosec within the company if it is truly sending packets outside the firewall...

394549 8y ago
> That is my question as well, without getting their hands on the machine, how do the attackers start the attacks? And in case of AWS, since everything is so virtualized, does it make sense to install a hardware backdoor if it cant even map to your true target？ Nation states have the manpower for spray-and-pray attacks. They have less need for precise targeting. > It could probably make more sense as a backdoor when the hackers get hold the access to the physical device, it is however quite incredible to me that this hack is designed for a remote network access, and could go unnoticed from infosec within the company if it is truly sending packets outside the firewall... How is it incredible? Infosec isn't perfect. Equifax was hacked in part because their IDS system was offline due to an out-of-date certificate [1]. That system would have caught the exfiltration of data if it had been active. Also, Amazon AWS must have all kinds of crazy traffic flowing through its network due to its customers. My gut feel is that it would be incredibly hard to characterize that traffic in order to proactively detect many kinds of nefarious traffic. The Bloomberg article stated they were easily able to find the traffic from these implants...but only after they knew what to look for. [1] https://www.bankinfosecurity.com/postmortem-behind-equifax-breach-multiple-failures-a-11480 https://www.bankinfosecurity.com/postmortem-behind-equifax-b...

AnthonyMouse 8y ago

> And the story isn't even that bad for Amazon or Apple - it's much worse for US-China relations than it is for either of those companies. If true, it's pretty bad. For one thing customers will be more wary of buying hardware from China. If they move manufacturing elsewhere (or, for Amazon, source hardware elsewhere) they would have higher costs and lower margins, if they don't and a competitor does they lose business. On top of that, people are already wary of the privacy implications of these companies having all your data, but most are willing to trust major US companies. If it looks like they can't keep it secure against random Chinese companies engaged in industrial espionage, more customers will conclude that operating servers in-house is the better choice.

Swenrekcah 8y ago
Silver linong is it’s probably better for both IT people and the economy as a whole that companies generally operate their own servers.
- jermy 8y ago
  Unless we use Supermicro kit, presumably?
  - AnthonyMouse 8y ago
    Unless we use made-in-China kit, presumably. It wouldn't be that surprising to see Supermicro move their production back out of China after this. Though they've been having a hard time of it recently already -- something about getting delisted from the NASDAQ. I wonder if the two incidents are related at all?