6 ms·

> These reports led us to believe the problem was likely a firmware issue, as most other issues could be resolved through a factory reset. My dream is to inter

> These reports led us to believe the problem was likely a firmware issue, as most other issues could be resolved through a factory reset. My dream is to intercept the write-enable lines on the flash chips holding these firmwares so I can lock out updates. And schedule a daily reboot for any memory-resident-only crap. That’s what we used to do on, ahem, satellite receivers, 20 years ago and maybe we all need to treat every device attached to the internet as having a similar susceptibility to “electronic counter-measures”. Or at least monitor them for updates and light up a light when an update happens if it was my own equipment and I’d know if it should go off or not.

iknowstuff 2y ago
[flagged]
- whatevaa 2y ago
  He just wants to watch the world burn.
- Denvercoder9 2y ago
  Most non-programmers don't give a shit about their router beyond "the wifi must work". Something completely stateless that can't be broken or messed with actually sounds like something they'd want.
  - fiedzia 2y ago
    For "wifi to work" you will need updates. Though updating all devices at once is a really dumb policy.
  - starttoaster 2y ago
    What about when the router has a security update that is actually useful to stop people from getting owned? Such as to address: https://thehackernews.com/2023/07/critical-mikrotik-routeros.html https://thehackernews.com/2023/07/critical-mikrotik-routeros... Blocking updates seems useful for blocking malicious and helpful updates. I wonder which camp the majority of updates fall into.
    - stacktrust 2y ago
      A physical switch can be locally toggled by the device owner/admin. Some motherboards offer a physical jumper for firmware updates, including x86 PC Engines APU2 coreboot router.
      - starttoaster 2y ago
        So here's another problem. Going back to the point made earlier in this thread "users don't care much about their routers." The average user opens their router page exceedingly rarely, which is the benefit of automatic updates. I set up my mom's router, and she works in IT (more in project management these days, so she's fallen into being a mostly non-technical user.) And she still texts me every time she needs to get into it to ask me what the password I set up was... once every few years. The downfall of automatic updates is obviously something like the case of the article this discussion is about. But weigh up the costs and benefits, in my opinion, the scale tips more towards automatic updates when you factor in all the critical level vulnerabilities that router OS's have accrued over the years.
        stacktrust 2y ago
        Some consumer routers (e.g. Amazon eero) have moved to cloud/app config and automatic updates. A read-only partition can bootstrap recovery from cloud, if the main firmware or config is damaged. Some updates are possible with live kernel patching of the memory-resident OS, which can also be used by malware.
    - labcomputer 2y ago
      Well, considering that most home users never update their router's firmware, I'm going to go out on a limb and suggest that the majority of applied updates are malicious. If you did want to go this route, a simple fix would be to have the write enable line gated by a hardware one-shot timer (think like a 555 timer) triggered by a physical button on the front (and the button would also reboot the router). The firmware update sequence would go like: Router prompts user to update using button -> user presses update button -> router reboots to clear malicious software -> when the router comes back up, the write enable gate remains open for $n minutes (and maybe there's a GPIO that can hold the gate open if it is already open) -> router performs software update. The problem is that there's pretty much no way to do this without adding a $1-2 to the BOM, and no manufacturer of (pro|con)sumer routers will do that. Edit: Or do what stacktrust wrote and just have a toggle switch (update/secure).
  - dymk 2y ago
    Bummer then, SSIDs and WiFi passwords are state.
    - q0uaur 2y ago
      plenty of routers in my country (id go on a limb and say the majority) have both written on a sticker, different password for each device generated when the firmware first gets installed, and people NEVER change it. edit they also come with a QR code you can scan on your phone to connect to the wifi without manually copy the password, so people just put up that qr code somewhere and connecting is easy enough.
    - WhyNotHugo 2y ago
      To be fair, most people never change these. In any case, you can have a separate storage (you need less than 1KB) for these two things.
      - Scoundreller 2y ago
        I think that’s usually the case: that will be stored in a small 24c or 93c (i2c or SPI) chip that is separate from the firmware that may only handle 1000 flashes and takes a lot longer to flash (often requiring risky and long downtime).
    - Denvercoder9 2y ago
      Half of my technical friends haven't changed theirs from the factory defaults. None of my non-technical friends have changed them. A non-representative sample of SSIDs I receive in my apartment gives 9 default ones, and 2 custom ones (plus mine).
  - deleted 2y ago
    [deleted]
  - hifromwork 2y ago
    Wormable vulnerability in a router is 10x bigger issue in practice than attacks that brick a router. By "in practice" I mean judging by the real world attacks that I know of. Hacked routers are used as residential proxies for criminals, for DDoS attacks, fast flux networks, credential stuffing attacks, and more. Bricking routers is rare (and very loud), that's why it's news on HN. Many router malware families don't even try to be persistent. Even Mirai - arguably the most famous router botnet - is not persistent [1]. In case the device is rebooted, it just gets infected again in a few minutes. It's very important that all network connected devices have an update mechanism, working automatically in the background. [1] At least the original version. After the code leak people were doing all kinds of updates, so there are some variants that try to be persistent in some cases.
    - stacktrust 2y ago
      > In case the device is rebooted, it just gets infected again in a few minutes. Can Suricata detect and block known router botnets?
- ghshephard 2y ago
  I have a friend who used to do classified military work - and a lot of the firmware on munitions is designed in precisely this way to avoid countermeasures. Advanced systems have a lot of software, and the systems they are shipped on have fuses and circuit traces that are melted to avoid any possible countermeasures or modification once they've passed acceptance.
  - starttoaster 2y ago
    When the military finds a flaw in their immutable firmware, what do they do with those munitions? Throw them away and build new ones? Routers are networked and critical vulnerabilities that should be addressed get patched on them over time. Is the proposal to just tell the user to buy a new router?
    - ThrowawayTestr 2y ago
      >what do they do with those munitions? They're sent back to the manufacturer and the boards are replaced.
      - rvnx 2y ago
        Here is the same with the routers that got bricked. They probably need to be reflashed. Most likely they are leased by the ISP so under their scope of responsibility and maintenance. "internet doesn't work" -> "send me a new router, it broke", problem solved.
      - Cthulhu_ 2y ago
        Given what little I know of military stuff, swapping out a hardware board is cheap compared to the cost of the whole thing. Would they be sent back or would an engineer travel to wherever stuff is stored to do the job?
        ThrowawayTestr 2y ago
        >swapping out a hardware board is cheap The labor is cheap, the boards are expensive as hell.
- bongodongobob 2y ago
  Same type of person who blocks Windows updates, messes with their registry, removes system files and processes, and then complains when Windows gets flaky on them and breaks in strange and unpredictable ways.
  - Scoundreller 2y ago
    If it’s a switch, you could unlock it and allow for the provider’s (or your own if it’s your own equipment) updates before re-locking it.
  - stacktrust 2y ago
    Windows has a stateless mode. https://learn.microsoft.com/en-us/windows/iot/iot-enterprise/customize/unified-write-filter https://learn.microsoft.com/en-us/windows/iot/iot-enterprise... > Unified Write Filter (UWF) is an optional Windows 10 feature that helps to protect your drives by intercepting and redirecting any writes to the drive (app installations, settings changes, saved data) to a virtual overlay. The virtual overlay is a temporary location that is cleared during a reboot or when a guest user logs off.. Increases security and reliability where new apps aren't frequently added. Can be used to reduce wear on solid-state drives and other write-sensitive media. Optimizing Application load timing on boot – it can be faster to resume from a HORM file on every boot rather than reloading the system on each boot. UWF replaces the Windows 7 Enhanced Write Filter (EWF) and the File Based Write Filter (FBWF).
    - Cthulhu_ 2y ago
      Reminds me of some management software we ran on school computers some 20 odd years ago, it basically made the computer immutable and it'd be back to how it was after a reboot. I forgot what it was called and never knew how it worked though, at the time I was impressed. It needed an admin password to make changes that persisted.
      - faefox 2y ago
        Deep Freeze, maybe? :)
aidenn0 2y ago
I suppose from the point of view of someone with a black-market HU card, DirecTV was an example of an Advanced Persistent Threat. Never thought of it that way before.
- Scoundreller 2y ago
  Funny thing about directv is that because they allowed for many manufacturers to build receivers, directv had little control over the receiver firmware, so these counter-counter measures weren’t necessary at the receiver level. Other providers that rolled out their own receivers had high control over the receiver firmware and once users figured out how to protect their cards, the receivers became an effective attack vector for the lazy. But that’s where a lot of the public knowledge about JTAGs really started coming to light. Awfully nice of them to put in a cutout at the bottom of the receiver.
napierzaza 2y ago
[dead]
schmidtleonard 2y ago
> maybe we all need to treat every device attached to the internet as having a similar susceptibility to “electronic counter-measures” "First party malware"
luma 2y ago
I'm not too familiar with customer DSL solutions but for cable modems, that firmware and configuration is managed by the CMTS because technology and configuration changes on the head end may require customer-side changes to ensure continued operation. The config is a pretty dynamic thing as frequency plans, signal rate, etc change over time as the cable plant and head end equipment is upgraded and maintained. I'd expect that any attempt to lock write enable to the EEPROM would eventually result in your modem failing to provision.
- Scoundreller 2y ago
  When your provider cuts you off, that’s when you know that your provider has a legit upgrade you need to take. Take the update and then lock stuff up again. Of course, I don’t think you’re supposed to make mods to your vendor provided equipment… In the satellite world, this would happen too: old firmware would be cut off. That’s when you go legit for a while with your sub’d card, take the update, and watch your sub’d channels until the new update could be reverse engineered. And probably have some heroes learn the hard way of taking the update and having some negative impacts that are harder to reverse.
  - luma 2y ago
    I'm not sure what such an approach would accomplish. If the goal is to prevent the kind of problem seen in the OP (which, let's be real - is a rare occurrence) in order to avoid an unplanned outage, you've instead created a situation where it'll fail to connect far more regularly as you're kicked off the network for not correctly handling the provisioning process. You're trading a rare unplanned outage for a common unplanned outage.
    - Scoundreller 2y ago
      Depends how often the provider pushes out updates (and the purpose/necessity of them). And it’s only that “rare unplanned outage” when a malicious update bricks your device. Much worse is a malicious update that doesn’t result in an outage. Probably still rare but that impact though. Edit: would also add that there’s probably a big firmware chip that changes infrequently, and frequently changing config stored on a separate and smaller chip (like a 24c or 93 series eeprom that holds a few kilobytes). That way you don’t risk bricking your modem by unplugging it at the wrong time.
not2b 2y ago
It's a no-win situation. Sure, disabling firmware updates would have prevented this attack, but it would also prevent security fixes that keep the routers from being turned into a botnet. But what I don't get in this case is why it was not possible to reset the device to its original state. It seems like a misdesign if it's possible to destroy all of the firmware, including the backup.
- utensil4778 2y ago
  Generally the way this works is you have two partitions in your flash chip. One contains the current firmware and the second is a place to drop new firmware. Then the bootloader twiddles a bit somewhere and boots to one partition or the other. There's really nothing stopping you from wiping the previous partition once you're done. I think some routers still have a single flash partition and the update process here is a lot more hairy and will obviously not retain the previous version after an update. Apart from attacks like this, there's absolutely no reason to have a protected read only copy of the factory firmware. 99.9999% all you would ever need to do to recover from a bad flash is to just fail back to the previous image. A proper read only factory image would require an extra ROM chip to store it, as well as extra bootloader complexity required to load from ROM or copy to flash on failure. It's just barely expensive enough at scale to not be worth it for an extremely rare event.
- yjftsjthsd-h 2y ago
  > It seems like a misdesign if it's possible to destroy all of the firmware, including the backup. Humor me; how would that work? If anything, I'd expect it to be easier to overwrite the inactive slot (assuming an A/B setup, ideally with read-only root). If you really wanted, you could have a separate chip that was read-only enforced by hardware, and I've seen that done for really low level firmware (ex. Chromebook boot firmware) but it's usually really limited precisely because the inability to update it means you get stuck with any bugs so it's usually only used to boot to the real (rw) storage.
- sounds 2y ago
  It's an interesting challenge because the device is nominally "under ISP control" but any device located in a customer's home is under the physical control of the customer. The mistrust between the ISP and the customer leads to "trusted" devices where the firmware, including the backup, can be overwritten by the ISP, but then cannot recover if it gets corrupted. And believe me, the corrupt firmware scenario happens a lot due to incompetence. This is getting attention because it wasn't incompetence this time. But how does blank, unprovisioned equipment discover a path to its provisioning server? Especially in light of the new "trusted" push, this is an arms race in a market segment such as routers where there isn't any money for high end solutions - only the cheapest option is even considered. tl;dr: a social and economic problem, likely can't be fixed with a purely technical solution
stacktrust 2y ago
> My dream is to intercept the write-enable lines on the flash chips holding these firmwares so I can lock out updates. And schedule a daily reboot for any memory-resident-only crap. There was an open hardware project for SD card emulation, where the emulator could reject writes, https://github.com/racklet/meeting-notes/blob/main/community/2021.md https://github.com/racklet/meeting-notes/blob/main/community... OSS emulation for SPI flash, https://trmm.net/Spispy/ https://trmm.net/Spispy/ Some USB drives (Kanguru) and SSD enclosures (ElecGear M.2 2230 NVME) have firmware and physical switch to block writes, useful to boot custom "live ISOs" that run from RAM.
- Scoundreller 2y ago
  Eventually in the satellite world, card emulators took over and only the receiver was a vector of attack, but then the receivers started getting simulated too. The nice thing about emulators is that you could intercept calls that you wanted and send your own response while still taking any and all updates. Hard to break when you have more control than they do.
- ThePowerOfFuet 2y ago
  > Some USB drives (Kanguru) and SSD enclosures (ElecGear M.2 2230 NVME) have firmware and physical switch to block writes, useful to boot custom "live ISOs" that run from RAM. For the rest of us, there's Ventoy. https://www.ventoy.net/ https://www.ventoy.net/
russdill 2y ago
Secure boot schemes can already "fix" this. If a boot image is programmed that isn't signed, the system boots to a write protected backup image. The system can also to some degree block the programming images that aren't signed, but presumably malware has gained root access.
ck2 2y ago
ISPs can send any firmware to a docsis cablemodem, without the user knowing or accepting. Imagine the damage that could be done by a malicious actor via the ISPs computers. Or imagine someone being able to hack the system that does that update even without the ISP. 600K users would be a toy, they could do it to 6 Million. Doesn't even have to be clever, just brick millions of cablemodems. North Korea or some other government level entity could manage the resources to figure that out.
- stacktrust 2y ago
  Do ISPs and modem vendors roll their own OTA infrastructure and signing key management, or contract it out?

iknowstuff 2y ago

[flagged]

whatevaa 2y ago
He just wants to watch the world burn.
Denvercoder9 2y ago
Most non-programmers don't give a shit about their router beyond "the wifi must work". Something completely stateless that can't be broken or messed with actually sounds like something they'd want.
- fiedzia 2y ago
  For "wifi to work" you will need updates. Though updating all devices at once is a really dumb policy.
- starttoaster 2y ago
  What about when the router has a security update that is actually useful to stop people from getting owned? Such as to address: https://thehackernews.com/2023/07/critical-mikrotik-routeros.html https://thehackernews.com/2023/07/critical-mikrotik-routeros... Blocking updates seems useful for blocking malicious and helpful updates. I wonder which camp the majority of updates fall into.
  - stacktrust 2y ago
    A physical switch can be locally toggled by the device owner/admin. Some motherboards offer a physical jumper for firmware updates, including x86 PC Engines APU2 coreboot router.
    - starttoaster 2y ago
      So here's another problem. Going back to the point made earlier in this thread "users don't care much about their routers." The average user opens their router page exceedingly rarely, which is the benefit of automatic updates. I set up my mom's router, and she works in IT (more in project management these days, so she's fallen into being a mostly non-technical user.) And she still texts me every time she needs to get into it to ask me what the password I set up was... once every few years. The downfall of automatic updates is obviously something like the case of the article this discussion is about. But weigh up the costs and benefits, in my opinion, the scale tips more towards automatic updates when you factor in all the critical level vulnerabilities that router OS's have accrued over the years.
      - stacktrust 2y ago
        Some consumer routers (e.g. Amazon eero) have moved to cloud/app config and automatic updates. A read-only partition can bootstrap recovery from cloud, if the main firmware or config is damaged. Some updates are possible with live kernel patching of the memory-resident OS, which can also be used by malware.
  - labcomputer 2y ago
    Well, considering that most home users never update their router's firmware, I'm going to go out on a limb and suggest that the majority of applied updates are malicious. If you did want to go this route, a simple fix would be to have the write enable line gated by a hardware one-shot timer (think like a 555 timer) triggered by a physical button on the front (and the button would also reboot the router). The firmware update sequence would go like: Router prompts user to update using button -> user presses update button -> router reboots to clear malicious software -> when the router comes back up, the write enable gate remains open for $n minutes (and maybe there's a GPIO that can hold the gate open if it is already open) -> router performs software update. The problem is that there's pretty much no way to do this without adding a $1-2 to the BOM, and no manufacturer of (pro|con)sumer routers will do that. Edit: Or do what stacktrust wrote and just have a toggle switch (update/secure).
- dymk 2y ago
  Bummer then, SSIDs and WiFi passwords are state.
  - q0uaur 2y ago
    plenty of routers in my country (id go on a limb and say the majority) have both written on a sticker, different password for each device generated when the firmware first gets installed, and people NEVER change it. edit they also come with a QR code you can scan on your phone to connect to the wifi without manually copy the password, so people just put up that qr code somewhere and connecting is easy enough.
  - WhyNotHugo 2y ago
    To be fair, most people never change these. In any case, you can have a separate storage (you need less than 1KB) for these two things.
    - Scoundreller 2y ago
      I think that’s usually the case: that will be stored in a small 24c or 93c (i2c or SPI) chip that is separate from the firmware that may only handle 1000 flashes and takes a lot longer to flash (often requiring risky and long downtime).
  - Denvercoder9 2y ago
    Half of my technical friends haven't changed theirs from the factory defaults. None of my non-technical friends have changed them. A non-representative sample of SSIDs I receive in my apartment gives 9 default ones, and 2 custom ones (plus mine).
- deleted 2y ago
  [deleted]
- hifromwork 2y ago
  Wormable vulnerability in a router is 10x bigger issue in practice than attacks that brick a router. By "in practice" I mean judging by the real world attacks that I know of. Hacked routers are used as residential proxies for criminals, for DDoS attacks, fast flux networks, credential stuffing attacks, and more. Bricking routers is rare (and very loud), that's why it's news on HN. Many router malware families don't even try to be persistent. Even Mirai - arguably the most famous router botnet - is not persistent [1]. In case the device is rebooted, it just gets infected again in a few minutes. It's very important that all network connected devices have an update mechanism, working automatically in the background. [1] At least the original version. After the code leak people were doing all kinds of updates, so there are some variants that try to be persistent in some cases.
  - stacktrust 2y ago
    > In case the device is rebooted, it just gets infected again in a few minutes. Can Suricata detect and block known router botnets?
ghshephard 2y ago
I have a friend who used to do classified military work - and a lot of the firmware on munitions is designed in precisely this way to avoid countermeasures. Advanced systems have a lot of software, and the systems they are shipped on have fuses and circuit traces that are melted to avoid any possible countermeasures or modification once they've passed acceptance.
- starttoaster 2y ago
  When the military finds a flaw in their immutable firmware, what do they do with those munitions? Throw them away and build new ones? Routers are networked and critical vulnerabilities that should be addressed get patched on them over time. Is the proposal to just tell the user to buy a new router?
  - ThrowawayTestr 2y ago
    >what do they do with those munitions? They're sent back to the manufacturer and the boards are replaced.
    - rvnx 2y ago
      Here is the same with the routers that got bricked. They probably need to be reflashed. Most likely they are leased by the ISP so under their scope of responsibility and maintenance. "internet doesn't work" -> "send me a new router, it broke", problem solved.
    - Cthulhu_ 2y ago
      Given what little I know of military stuff, swapping out a hardware board is cheap compared to the cost of the whole thing. Would they be sent back or would an engineer travel to wherever stuff is stored to do the job?
      - ThrowawayTestr 2y ago
        >swapping out a hardware board is cheap The labor is cheap, the boards are expensive as hell.
bongodongobob 2y ago
Same type of person who blocks Windows updates, messes with their registry, removes system files and processes, and then complains when Windows gets flaky on them and breaks in strange and unpredictable ways.
- Scoundreller 2y ago
  If it’s a switch, you could unlock it and allow for the provider’s (or your own if it’s your own equipment) updates before re-locking it.
- stacktrust 2y ago
  Windows has a stateless mode. https://learn.microsoft.com/en-us/windows/iot/iot-enterprise/customize/unified-write-filter https://learn.microsoft.com/en-us/windows/iot/iot-enterprise... > Unified Write Filter (UWF) is an optional Windows 10 feature that helps to protect your drives by intercepting and redirecting any writes to the drive (app installations, settings changes, saved data) to a virtual overlay. The virtual overlay is a temporary location that is cleared during a reboot or when a guest user logs off.. Increases security and reliability where new apps aren't frequently added. Can be used to reduce wear on solid-state drives and other write-sensitive media. Optimizing Application load timing on boot – it can be faster to resume from a HORM file on every boot rather than reloading the system on each boot. UWF replaces the Windows 7 Enhanced Write Filter (EWF) and the File Based Write Filter (FBWF).
  - Cthulhu_ 2y ago
    Reminds me of some management software we ran on school computers some 20 odd years ago, it basically made the computer immutable and it'd be back to how it was after a reboot. I forgot what it was called and never knew how it worked though, at the time I was impressed. It needed an admin password to make changes that persisted.
    - faefox 2y ago
      Deep Freeze, maybe? :)

aidenn0 2y ago

I suppose from the point of view of someone with a black-market HU card, DirecTV was an example of an Advanced Persistent Threat. Never thought of it that way before.

Scoundreller 2y ago
Funny thing about directv is that because they allowed for many manufacturers to build receivers, directv had little control over the receiver firmware, so these counter-counter measures weren’t necessary at the receiver level. Other providers that rolled out their own receivers had high control over the receiver firmware and once users figured out how to protect their cards, the receivers became an effective attack vector for the lazy. But that’s where a lot of the public knowledge about JTAGs really started coming to light. Awfully nice of them to put in a cutout at the bottom of the receiver.

napierzaza 2y ago

[dead]

schmidtleonard 2y ago

> maybe we all need to treat every device attached to the internet as having a similar susceptibility to “electronic counter-measures” "First party malware"

luma 2y ago

I'm not too familiar with customer DSL solutions but for cable modems, that firmware and configuration is managed by the CMTS because technology and configuration changes on the head end may require customer-side changes to ensure continued operation. The config is a pretty dynamic thing as frequency plans, signal rate, etc change over time as the cable plant and head end equipment is upgraded and maintained. I'd expect that any attempt to lock write enable to the EEPROM would eventually result in your modem failing to provision.

Scoundreller 2y ago
When your provider cuts you off, that’s when you know that your provider has a legit upgrade you need to take. Take the update and then lock stuff up again. Of course, I don’t think you’re supposed to make mods to your vendor provided equipment… In the satellite world, this would happen too: old firmware would be cut off. That’s when you go legit for a while with your sub’d card, take the update, and watch your sub’d channels until the new update could be reverse engineered. And probably have some heroes learn the hard way of taking the update and having some negative impacts that are harder to reverse.
- luma 2y ago
  I'm not sure what such an approach would accomplish. If the goal is to prevent the kind of problem seen in the OP (which, let's be real - is a rare occurrence) in order to avoid an unplanned outage, you've instead created a situation where it'll fail to connect far more regularly as you're kicked off the network for not correctly handling the provisioning process. You're trading a rare unplanned outage for a common unplanned outage.
  - Scoundreller 2y ago
    Depends how often the provider pushes out updates (and the purpose/necessity of them). And it’s only that “rare unplanned outage” when a malicious update bricks your device. Much worse is a malicious update that doesn’t result in an outage. Probably still rare but that impact though. Edit: would also add that there’s probably a big firmware chip that changes infrequently, and frequently changing config stored on a separate and smaller chip (like a 24c or 93 series eeprom that holds a few kilobytes). That way you don’t risk bricking your modem by unplugging it at the wrong time.

not2b 2y ago

It's a no-win situation. Sure, disabling firmware updates would have prevented this attack, but it would also prevent security fixes that keep the routers from being turned into a botnet. But what I don't get in this case is why it was not possible to reset the device to its original state. It seems like a misdesign if it's possible to destroy all of the firmware, including the backup.

utensil4778 2y ago
Generally the way this works is you have two partitions in your flash chip. One contains the current firmware and the second is a place to drop new firmware. Then the bootloader twiddles a bit somewhere and boots to one partition or the other. There's really nothing stopping you from wiping the previous partition once you're done. I think some routers still have a single flash partition and the update process here is a lot more hairy and will obviously not retain the previous version after an update. Apart from attacks like this, there's absolutely no reason to have a protected read only copy of the factory firmware. 99.9999% all you would ever need to do to recover from a bad flash is to just fail back to the previous image. A proper read only factory image would require an extra ROM chip to store it, as well as extra bootloader complexity required to load from ROM or copy to flash on failure. It's just barely expensive enough at scale to not be worth it for an extremely rare event.
yjftsjthsd-h 2y ago
> It seems like a misdesign if it's possible to destroy all of the firmware, including the backup. Humor me; how would that work? If anything, I'd expect it to be easier to overwrite the inactive slot (assuming an A/B setup, ideally with read-only root). If you really wanted, you could have a separate chip that was read-only enforced by hardware, and I've seen that done for really low level firmware (ex. Chromebook boot firmware) but it's usually really limited precisely because the inability to update it means you get stuck with any bugs so it's usually only used to boot to the real (rw) storage.
sounds 2y ago
It's an interesting challenge because the device is nominally "under ISP control" but any device located in a customer's home is under the physical control of the customer. The mistrust between the ISP and the customer leads to "trusted" devices where the firmware, including the backup, can be overwritten by the ISP, but then cannot recover if it gets corrupted. And believe me, the corrupt firmware scenario happens a lot due to incompetence. This is getting attention because it wasn't incompetence this time. But how does blank, unprovisioned equipment discover a path to its provisioning server? Especially in light of the new "trusted" push, this is an arms race in a market segment such as routers where there isn't any money for high end solutions - only the cheapest option is even considered. tl;dr: a social and economic problem, likely can't be fixed with a purely technical solution

stacktrust 2y ago

> My dream is to intercept the write-enable lines on the flash chips holding these firmwares so I can lock out updates. And schedule a daily reboot for any memory-resident-only crap. There was an open hardware project for SD card emulation, where the emulator could reject writes, https://github.com/racklet/meeting-notes/blob/main/community/2021.md https://github.com/racklet/meeting-notes/blob/main/community... OSS emulation for SPI flash, https://trmm.net/Spispy/ https://trmm.net/Spispy/ Some USB drives (Kanguru) and SSD enclosures (ElecGear M.2 2230 NVME) have firmware and physical switch to block writes, useful to boot custom "live ISOs" that run from RAM.

Scoundreller 2y ago
Eventually in the satellite world, card emulators took over and only the receiver was a vector of attack, but then the receivers started getting simulated too. The nice thing about emulators is that you could intercept calls that you wanted and send your own response while still taking any and all updates. Hard to break when you have more control than they do.
ThePowerOfFuet 2y ago
> Some USB drives (Kanguru) and SSD enclosures (ElecGear M.2 2230 NVME) have firmware and physical switch to block writes, useful to boot custom "live ISOs" that run from RAM. For the rest of us, there's Ventoy. https://www.ventoy.net/ https://www.ventoy.net/

russdill 2y ago

Secure boot schemes can already "fix" this. If a boot image is programmed that isn't signed, the system boots to a write protected backup image. The system can also to some degree block the programming images that aren't signed, but presumably malware has gained root access.

ck2 2y ago

ISPs can send any firmware to a docsis cablemodem, without the user knowing or accepting. Imagine the damage that could be done by a malicious actor via the ISPs computers. Or imagine someone being able to hack the system that does that update even without the ISP. 600K users would be a toy, they could do it to 6 Million. Doesn't even have to be clever, just brick millions of cablemodems. North Korea or some other government level entity could manage the resources to figure that out.

stacktrust 2y ago
Do ISPs and modem vendors roll their own OTA infrastructure and signing key management, or contract it out?