4 ms·

You could but a base level firmware on ROM, with a hardware trigger, and all that does on boot is listen and receive a signed firmware to write to the system. I

You could but a base level firmware on ROM, with a hardware trigger, and all that does on boot is listen and receive a signed firmware to write to the system. It needs a way to be triggered through hardware examining traffic and that also needs to require the seen command be signed. That recovery boot system needs to be as simple and minimal as possibly so you can have good assurance that there aren't problems with it, and should be written in the safest language you can get away with. Guard that signing key with your life, and lock it away for a rainy day, only to be used if much of your fleet of devices is hosed entirely. It should not be the same as a firmware signing key which needs to be pulled out and used sometimes. I think that could work, to a degree. There's always the risk that your recovery mechanism itself it exploited, so you need to make it as small and hardened a target as possible and reduce its complexity to the bare minimum. That doesn't solve the problem, which might be inherently unsolvable, but it may reduce that likelihood of it to levels where it's not a problem until long past the lifecycle of the devices.

ajross 2y ago
> You could but a base level firmware on ROM, with a hardware trigger, and all that does on boot is listen and receive a signed firmware to write to the system. Almost all devices have something like that already in the form of a bootloader or SOC bootstrapping mode. But the idea breaks down if you want to do it OTA. The full storage/kernel/network/UI stack required to make that happen isn't ever going to run under "ROM" in the sense of truly immutable storage. The best you get is a read-only backup partition (shipped in some form on pretty much all laptops today), but that's no less exploitable really.
- stacktrust 2y ago
  Apple has a robust recovery mechanism on their laptops, via T2 security coprocessor. https://www.macrumors.com/2020/06/25/apple-silicon-macs-new-startup-recovery-options/ https://www.macrumors.com/2020/06/25/apple-silicon-macs-new-... https://support.apple.com/guide/security/ https://support.apple.com/guide/security/
  - ajross 2y ago
    Which is still running out of mutable storage. The point isn't whether you can verify the boot, it's whether you can prevent a compromised device (compromised to the point of being able to write to its own storage) from bricking itself. Now, as it happens Apple (everyone really, but Apple is a leader for sure) has some great protections in place to prevent that. And that's great. But if you feel you can rely on those protections there's no need to demand the ROM recovery demanded upthread.
    - stacktrust 2y ago
      There's also Apple DFU mode to restore OS with help from immutable ROM and external device, without depending on installed OS or mutable "Recovery OS" partition, https://theapplewiki.com/wiki/DFU_Mode https://theapplewiki.com/wiki/DFU_Mode & https://support.apple.com/en-us/108900 https://support.apple.com/en-us/108900 > DFU or Device Firmware Upgrade mode allows all devices to be restored from any state. It is essentially a mode where the BootROM can accept iBSS. DFU is part of the SecureROM which is burned into the hardware, so it cannot be removed.
      - ajross 2y ago
        ... right, which as mentioned requires physical access and external hardware and doesn't meet the requirements above either. And it's not particularly notable either: every flashable SOC in every market has something like this, at different levels of sophistication. Again it's just not a solvable problem at the level of integration imagined.
- kbenson 2y ago
  > The full storage/kernel/network/UI stack required to make that happen isn't ever going to run under "ROM" in the sense of truly immutable storage. Why not? I'm essentially describing a specialized OOB system, and it would just use a carved out small chunk of system RAM or ship with a minimal amount RAM of its own. If you mean actually impossible to change because it's physical ROM ("truly immutable"), that's less important to the design than there's no mechanism that allows that storage area to be written to from the system itself, whether that's just the very locked down and minimal recovery kernel it houses not allowing it, or a jumper.
  - ajross 2y ago
    Sure, but now your device needs two eMMC chips or whatever to store all that extra junk, and it's been priced out of the market. FWIW: an alternative to designs like this is just to ship your customers an extra router to keep in a box if the first stops working: it's exactly the same principle, and fails for the same reasons.
    - AnthonyMouse 2y ago
      There is a simple solution to this: Make the flash removable. The firmware is stored on some SD card or M.2 device, if it becomes corrupt then you take it out and flash it with clean firmware using any PC. You don't even need the rest of the device to contain any signing mechanism with keys that could be compromised, because using this method requires physical access, and any compromise that occurs from physical access can be detected or undone with same by checksumming or re-flashing the storage device again from a clean PC. And you can also do signed firmware updates OTA without worrying that the device can be bricked by a vulnerability or signing key compromise, because it can always be restored via physical access.
    - kbenson 2y ago
      > and it's been priced out of the market. Whether that's likely is entirely based on the cost of the device. Some things are simple and cheap and extra hardware cuts deeply into the profit. Others are not but this sort of thing is also important because they are remote and you don't want to have a person go out on site. When the device is expensive enough or sending someone to the site is expensive enough, "just ship a replacement" is not really a viable solution, unless you're installing it in a high-availability capacity where you can fail over to it without physical intervention. Obviously it's not a solution for every circumstance. Nothing really is. I don't think it's useful for us to assume that a solution has to be, as that doesn't really help us in the many instances when it's good enough.

ajross 2y ago

> You could but a base level firmware on ROM, with a hardware trigger, and all that does on boot is listen and receive a signed firmware to write to the system. Almost all devices have something like that already in the form of a bootloader or SOC bootstrapping mode. But the idea breaks down if you want to do it OTA. The full storage/kernel/network/UI stack required to make that happen isn't ever going to run under "ROM" in the sense of truly immutable storage. The best you get is a read-only backup partition (shipped in some form on pretty much all laptops today), but that's no less exploitable really.

stacktrust 2y ago
Apple has a robust recovery mechanism on their laptops, via T2 security coprocessor. https://www.macrumors.com/2020/06/25/apple-silicon-macs-new-startup-recovery-options/ https://www.macrumors.com/2020/06/25/apple-silicon-macs-new-... https://support.apple.com/guide/security/ https://support.apple.com/guide/security/
- ajross 2y ago
  Which is still running out of mutable storage. The point isn't whether you can verify the boot, it's whether you can prevent a compromised device (compromised to the point of being able to write to its own storage) from bricking itself. Now, as it happens Apple (everyone really, but Apple is a leader for sure) has some great protections in place to prevent that. And that's great. But if you feel you can rely on those protections there's no need to demand the ROM recovery demanded upthread.
  - stacktrust 2y ago
    There's also Apple DFU mode to restore OS with help from immutable ROM and external device, without depending on installed OS or mutable "Recovery OS" partition, https://theapplewiki.com/wiki/DFU_Mode https://theapplewiki.com/wiki/DFU_Mode & https://support.apple.com/en-us/108900 https://support.apple.com/en-us/108900 > DFU or Device Firmware Upgrade mode allows all devices to be restored from any state. It is essentially a mode where the BootROM can accept iBSS. DFU is part of the SecureROM which is burned into the hardware, so it cannot be removed.
    - ajross 2y ago
      ... right, which as mentioned requires physical access and external hardware and doesn't meet the requirements above either. And it's not particularly notable either: every flashable SOC in every market has something like this, at different levels of sophistication. Again it's just not a solvable problem at the level of integration imagined.
kbenson 2y ago
> The full storage/kernel/network/UI stack required to make that happen isn't ever going to run under "ROM" in the sense of truly immutable storage. Why not? I'm essentially describing a specialized OOB system, and it would just use a carved out small chunk of system RAM or ship with a minimal amount RAM of its own. If you mean actually impossible to change because it's physical ROM ("truly immutable"), that's less important to the design than there's no mechanism that allows that storage area to be written to from the system itself, whether that's just the very locked down and minimal recovery kernel it houses not allowing it, or a jumper.
- ajross 2y ago
  Sure, but now your device needs two eMMC chips or whatever to store all that extra junk, and it's been priced out of the market. FWIW: an alternative to designs like this is just to ship your customers an extra router to keep in a box if the first stops working: it's exactly the same principle, and fails for the same reasons.
  - AnthonyMouse 2y ago
    There is a simple solution to this: Make the flash removable. The firmware is stored on some SD card or M.2 device, if it becomes corrupt then you take it out and flash it with clean firmware using any PC. You don't even need the rest of the device to contain any signing mechanism with keys that could be compromised, because using this method requires physical access, and any compromise that occurs from physical access can be detected or undone with same by checksumming or re-flashing the storage device again from a clean PC. And you can also do signed firmware updates OTA without worrying that the device can be bricked by a vulnerability or signing key compromise, because it can always be restored via physical access.
  - kbenson 2y ago
    > and it's been priced out of the market. Whether that's likely is entirely based on the cost of the device. Some things are simple and cheap and extra hardware cuts deeply into the profit. Others are not but this sort of thing is also important because they are remote and you don't want to have a person go out on site. When the device is expensive enough or sending someone to the site is expensive enough, "just ship a replacement" is not really a viable solution, unless you're installing it in a high-availability capacity where you can fail over to it without physical intervention. Obviously it's not a solution for every circumstance. Nothing really is. I don't think it's useful for us to assume that a solution has to be, as that doesn't really help us in the many instances when it's good enough.