10 ms·

Is there an article that describes a bit more in detail what the chips actually did (or were capable of doing)? They only say "the microchip altered the operati

Is there an article that describes a bit more in detail what the chips actually did (or were capable of doing)? They only say "the microchip altered the operating system’s core so it could accept modifications.", which I might interpret as circumventing signature checks to allow installing modified firmware on the systems? But how does the chip connect to the network and how does it receive commands? That said, it's pretty scary that you can hide so much malicious functionality in such a small device, makes me wonder what might be hidden in my Lenovo. In any case it speaks highly of the auditing firm that they were able to locate this. I wonder if they performed an x-ray analysis of the board, as given the size of these chips it should be possible to embed such devices in one of the internal layers of the board as well, making them essentially invisible to optical inspection.

RL_Quine 8y ago
SuperMicro hardware has very extensive IPMI integration into the mothebroard, which amongst other things can take over and inject frames into the network interface, emulate a VGA device, talk to the CPUs serial lines directly, flash firmware, control the state of a number of physical devices- and this is what it supports just from the web interface it presents by default with the password "ADMIN:ADMIN". My money, based on experience attempting to harden their devices, is that any modification were injected into the IPMI hardware where most of this was already supported. This stuff ends up being extremely difficult to disable. The naive approach would be to not connect to the dedicated NIC that's indicated on the back and in the instruction manual, but if you do this it masquerades onto the main NIC invisibly to the OS and DHCPs on its own to open up an administration port, web interface, and some assorted call homes. You have to explicitly tell it to use the non-connected port, change credentials, and modify it so that it is not accessible within operating system as well. Hopefully while the machine is offline to prevented any automated scanning finding it within your network. The number of times I'd end up nmaping our local networks and being able to remotely access production hardware with an interface that allowed me to reach this interface was maddening. The system is basically designed to be as insecure as possible by default, and allow for the maximum possible persistent threats with BIOS flashing, IPMI flashing, and other completely nu-authenticated avenues exposed. The course of action was always just to write off the hardware and bin it, because god knows what impact you could actually have using that interface.
- transpute 8y ago
  OpenBMC (Facebook, Google, Microsoft, Intel, IBM and others) is working on open-source baseboard management software, https://www.linuxfoundation.org/blog/2018/03/openbmc-project-community-comes-together-at-the-linux-foundation-to-define-open-source-implementation-of-bmc-firmware-stack/ https://www.linuxfoundation.org/blog/2018/03/openbmc-project... > The organizations behind the new project each have already made substantial contributions to creating open source baseboard management controller (BMC) firmware. Now, working together, they will define the vision for a standard stack that can be used across systems and computing environments. LinuxBoot and Open Compute OSF are working on open-source server firmware that can be measured on every boot and validated against hardware root of trust keys controlled by the server owner instead of the server OEM, https://www.platformsecuritysummit.com/2018/speaker/hudson/ https://www.platformsecuritysummit.com/2018/speaker/hudson/
  - alfalfasprout 8y ago
    Wouldn't help. The BMC hardware has direct serial access to CPUs and other hardware in the machine. Communication is unencrypted. A hardware modification attack wouldn't touch the firmware at all and could still compromise IPMI functionality.
    - deleted 8y ago
      [deleted]
    - greglindahl 8y ago
      Reducing the attack surface does help -- you're making perfection the enemy of the "somewhat better".
      - tomaskafka 8y ago
        No, against determined adversary it doesn't.
    - transpute 8y ago
      Even now, supply chain hardware modification attacks remind people of fiction. However, the number of people known to be affected by buggy BMC firmware is orders of magnitude larger, as described two comments upthread: > The system is basically designed to be as insecure as possible by default, and allow for the maximum possible persistent threats with BIOS flashing, IPMI flashing, and other completely un-authenticated avenues exposed.
newsDerp 8y ago
5. When a server was installed and switched on, the microchip altered the operating system’s core so it could accept modifications. The chip could also contact computers controlled by the attackers in search of further instructions and code. So, in typical vulnerability/payload/exploit fashion, the board's bus is vulnerable by default, because the chip pierces all the usual lines of defense protecting against network and operator I/O. It carries a payload intended to target very common features used everywhere commodity servers are used, one that likely listens for DMA traffic on the bus, and alters the signal stream, by escaping upon the occurrence of a magic sequence, and inserting its own signal, before resuming the authentic stream in flight. The payload could be pretty small, since the server boards are likely using OS packages that match the chipset. This limits the software to a small set of well known targets, Linux, Windows, Apple. Target their kernels, and you only have to snip out a small chunk of bytes, and splice your own pre-defined package in. Splice in a miniature runtime, that operates a turing complete set of operations, and open up a listener that waits for network access, and now, your payload can enable arbitrary code execution, irrespective of permissions. Now, to exploit, the payload needs to time the opportunity to splice itself onto the disk correctly. If certain well-known chunks of code will always exist in each given operating system, then with every disk access event one just needs to wait for the inevitable moment those magic system-specific bytes travel over the bus, in order to replace the known bytes with the poisoned modification. Events might target when the bytes are originally installed with the OS, or every time the OS reads those known bytes back into live memory, from any source. The total payload package could probably fit inside a couple of megabytes, pack on a few more for the "listen & splice" part of the attack to round out the entire mass, and all we know how much data an SD card can fit into say... five grains of rice?
- RL_Quine 8y ago
  It's not particularly magical, there's consumer chips around which are not a whole lot bigger (though obviously in a more standard package). You don't get a lot of resources, but you don't really need it if all the other frameworks are in place in other software. If this sort of thing is something you can buy on Mouser for a few cents, the espionage grade material is probably an order or magnitude more higher quality. https://www.microchip.com/wwwproducts/en/ATtiny4 https://www.microchip.com/wwwproducts/en/ATtiny4 For scale, this alone is about the size of a large SMD capacitor and would basically be lost in most designs today.
07d046 8y ago
The article says: But they were capable of doing two very important things: telling the device to communicate with one of several anonymous computers elsewhere on the internet that were loaded with more complex code; and preparing the device’s operating system to accept this new code. The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off. To me, that makes it sound like they could download from a remote host and inject code and do literally anything.
- jeletonskelly 8y ago
  If this is indeed the case, I'm surprised someone didn't catch something earlier when the device was calling "home" over their network. I'm wondering if China stole BlackRidge First Packet Authentication tech [1] to keep things dark. BlackRidge is... "involved" in IC and defense projects. [1]https://patents.google.com/patent/US8346951B2/en https://patents.google.com/patent/US8346951B2/en
mankash666 8y ago
This! I wish we had a more technical deep dive.
jjcc 8y ago
Not the chip itself but there's an investigation which I've already submitted: https://news.ycombinator.com/item?id=18144519 https://news.ycombinator.com/item?id=18144519
dreamcompiler 8y ago
You're not the only one worried about Lenovo. https://freebeacon.com/national-security/military-warns-chinese-computer-gear-poses-cyber-spy-threat/ https://freebeacon.com/national-security/military-warns-chin...

RL_Quine 8y ago

SuperMicro hardware has very extensive IPMI integration into the mothebroard, which amongst other things can take over and inject frames into the network interface, emulate a VGA device, talk to the CPUs serial lines directly, flash firmware, control the state of a number of physical devices- and this is what it supports just from the web interface it presents by default with the password "ADMIN:ADMIN". My money, based on experience attempting to harden their devices, is that any modification were injected into the IPMI hardware where most of this was already supported. This stuff ends up being extremely difficult to disable. The naive approach would be to not connect to the dedicated NIC that's indicated on the back and in the instruction manual, but if you do this it masquerades onto the main NIC invisibly to the OS and DHCPs on its own to open up an administration port, web interface, and some assorted call homes. You have to explicitly tell it to use the non-connected port, change credentials, and modify it so that it is not accessible within operating system as well. Hopefully while the machine is offline to prevented any automated scanning finding it within your network. The number of times I'd end up nmaping our local networks and being able to remotely access production hardware with an interface that allowed me to reach this interface was maddening. The system is basically designed to be as insecure as possible by default, and allow for the maximum possible persistent threats with BIOS flashing, IPMI flashing, and other completely nu-authenticated avenues exposed. The course of action was always just to write off the hardware and bin it, because god knows what impact you could actually have using that interface.

transpute 8y ago
OpenBMC (Facebook, Google, Microsoft, Intel, IBM and others) is working on open-source baseboard management software, https://www.linuxfoundation.org/blog/2018/03/openbmc-project-community-comes-together-at-the-linux-foundation-to-define-open-source-implementation-of-bmc-firmware-stack/ https://www.linuxfoundation.org/blog/2018/03/openbmc-project... > The organizations behind the new project each have already made substantial contributions to creating open source baseboard management controller (BMC) firmware. Now, working together, they will define the vision for a standard stack that can be used across systems and computing environments. LinuxBoot and Open Compute OSF are working on open-source server firmware that can be measured on every boot and validated against hardware root of trust keys controlled by the server owner instead of the server OEM, https://www.platformsecuritysummit.com/2018/speaker/hudson/ https://www.platformsecuritysummit.com/2018/speaker/hudson/
- alfalfasprout 8y ago
  Wouldn't help. The BMC hardware has direct serial access to CPUs and other hardware in the machine. Communication is unencrypted. A hardware modification attack wouldn't touch the firmware at all and could still compromise IPMI functionality.
  - deleted 8y ago
    [deleted]
  - greglindahl 8y ago
    Reducing the attack surface does help -- you're making perfection the enemy of the "somewhat better".
    - tomaskafka 8y ago
      No, against determined adversary it doesn't.
  - transpute 8y ago
    Even now, supply chain hardware modification attacks remind people of fiction. However, the number of people known to be affected by buggy BMC firmware is orders of magnitude larger, as described two comments upthread: > The system is basically designed to be as insecure as possible by default, and allow for the maximum possible persistent threats with BIOS flashing, IPMI flashing, and other completely un-authenticated avenues exposed.

newsDerp 8y ago

5. When a server was installed and switched on, the microchip altered the operating system’s core so it could accept modifications. The chip could also contact computers controlled by the attackers in search of further instructions and code. So, in typical vulnerability/payload/exploit fashion, the board's bus is vulnerable by default, because the chip pierces all the usual lines of defense protecting against network and operator I/O. It carries a payload intended to target very common features used everywhere commodity servers are used, one that likely listens for DMA traffic on the bus, and alters the signal stream, by escaping upon the occurrence of a magic sequence, and inserting its own signal, before resuming the authentic stream in flight. The payload could be pretty small, since the server boards are likely using OS packages that match the chipset. This limits the software to a small set of well known targets, Linux, Windows, Apple. Target their kernels, and you only have to snip out a small chunk of bytes, and splice your own pre-defined package in. Splice in a miniature runtime, that operates a turing complete set of operations, and open up a listener that waits for network access, and now, your payload can enable arbitrary code execution, irrespective of permissions. Now, to exploit, the payload needs to time the opportunity to splice itself onto the disk correctly. If certain well-known chunks of code will always exist in each given operating system, then with every disk access event one just needs to wait for the inevitable moment those magic system-specific bytes travel over the bus, in order to replace the known bytes with the poisoned modification. Events might target when the bytes are originally installed with the OS, or every time the OS reads those known bytes back into live memory, from any source. The total payload package could probably fit inside a couple of megabytes, pack on a few more for the "listen & splice" part of the attack to round out the entire mass, and all we know how much data an SD card can fit into say... five grains of rice?

RL_Quine 8y ago
It's not particularly magical, there's consumer chips around which are not a whole lot bigger (though obviously in a more standard package). You don't get a lot of resources, but you don't really need it if all the other frameworks are in place in other software. If this sort of thing is something you can buy on Mouser for a few cents, the espionage grade material is probably an order or magnitude more higher quality. https://www.microchip.com/wwwproducts/en/ATtiny4 https://www.microchip.com/wwwproducts/en/ATtiny4 For scale, this alone is about the size of a large SMD capacitor and would basically be lost in most designs today.

07d046 8y ago

The article says: But they were capable of doing two very important things: telling the device to communicate with one of several anonymous computers elsewhere on the internet that were loaded with more complex code; and preparing the device’s operating system to accept this new code. The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off. To me, that makes it sound like they could download from a remote host and inject code and do literally anything.

jeletonskelly 8y ago
If this is indeed the case, I'm surprised someone didn't catch something earlier when the device was calling "home" over their network. I'm wondering if China stole BlackRidge First Packet Authentication tech [1] to keep things dark. BlackRidge is... "involved" in IC and defense projects. [1]https://patents.google.com/patent/US8346951B2/en https://patents.google.com/patent/US8346951B2/en

mankash666 8y ago

This! I wish we had a more technical deep dive.

jjcc 8y ago

Not the chip itself but there's an investigation which I've already submitted: https://news.ycombinator.com/item?id=18144519 https://news.ycombinator.com/item?id=18144519

dreamcompiler 8y ago

You're not the only one worried about Lenovo. https://freebeacon.com/national-security/military-warns-chinese-computer-gear-poses-cyber-spy-threat/ https://freebeacon.com/national-security/military-warns-chin...