4 ms·

A work-from-home colleague just lost a work day or two when the manufacturer of his home wi-fi network reached down from the cloud, silently updated the firmwar

A work-from-home colleague just lost a work day or two when the manufacturer of his home wi-fi network reached down from the cloud, silently updated the firmware, and introduced flakiness in the process. Imagine that happening to your washing machine, or stove, or fridge. By malice or incompetence, the invisible hand of the cloud reaching down and making a device work differently than you're used to. No thanks. But an easy fix for those recalcitrant 50% like me. Just make the device refuse to function at all unless it has (recent) internet connectivity. Problem solved.

happymellon 4y ago
Solved, or a return.
olivierduval 4y ago
Actually, you'll get a direct return to the store with a big red label "Doesn't work"... and refund
- AnimalMuppet 4y ago
  Yeah. 50% won't voluntarily connect. Maybe 25% will return for refund instead of connect. (And maybe 10% don't even have internet, and aren't going to get it for a washing machine...)
H1Supreme 4y ago
Care to list the brand (model)? That seems insane to me. Firmware upgrades should be deliberate, and done with caution (hopefully starting with a backup).
- 2muchcoffeeman 4y ago
  Any update should be deliberate. Windows will now just update it self without asking over night. I often wake up, and see that my PC laptop has rebooted into Linux since that’s at the top in the bootloader.
  - eatsyourtacos 4y ago
    Eh I think that is honestly just not true with regards to Windows, sorry. There are like 1.5 billion computers running it. Them forcing updates is simply the best procedure because we live in a world full of complete asshats that want to create chaos and do malicious shit. The average person would never update their computers "deliberately" if that was the default. It's simply the lesser of all evils to ensure people are relatively up to date on security. I don't really like it either, but I understand it completely.
    - alistairSH 4y ago
      There's a massive difference between a completely silent update and the more common "Update available, install now Y/N?" Apple, MS, etc should do the second. Even with a "do it now, or within 7 days, or we do it for you." Same for the router. For goods that have safety implications (cars), the updates should NEVER be silent. And probably need a long notification period with a detailed "What's New".
      - TylerE 4y ago
        What if it's a fix for an actively-exploited zero day?
        alistairSH 4y ago
        Maybe give me a day? I absolutely don't want my laptop updating/rebooting while I'm in the middle of work. I understand the need to keep up to date for security reasons, but at some point, I just need my device to work without interruption.
        7speter 4y ago
        Have the press make noise about it. This has happened a few times with i devices and intel cpus. I shouldn’t have to fight my computer to stop it from upgrading itself to Windows 11
    - domador 4y ago
- nerdponx 4y ago
  I've read that silently pushing firmware updates to broadband modems is standard ISP practice, at least in the USA.
  - ArchOversight 4y ago
    It's part of the DOCSIS standard. Users are not allowed to modify/update the firmware on a DOCSIS modem because that could potentially be used to unlock additional speed outside of the users paid for speeds due to the way that the standard does bandwidth limiting on the modem instead of further upstream. That is why ISPs have lists of cable modems that are certified for their networks, and thus the ones allowed to connect to them as that is the list they have firmware updates for.
- rnjesus 4y ago
  my eero pro does this. about once a month it will decide to update itself around two in the morning without any prompt, notification, or any way to defer or delay. there is no option to adjust update times. you can check the app manually and install available updates, but if you don’t do this within a certain amount of time, it will do it automatically within a few days. three times now i have been up late working on a project and it has kicked me offline to update. it’s so goddamned annoying. i wouldn’t have bought this router if i’d known it did this. (i also had to call them on the phone to get a custom ip range enabled on it, which, for a “pro” product, is just ridiculous).
- lrem 4y ago
  Ever heard of Tesla? ;)
  - m463 4y ago
    firmware update - now all the buttons are smaller and harder to hit!
- yencabulator 4y ago
  > Firmware upgrades should be deliberate, and done with caution (hopefully starting with a backup). Many, many, devices don't allow firmware downgrades, on purpose. On the extreme end, Microsoft spent a huge amount of development effort to forbid that on the hardware level. I con't find the Xbox 360 link right now, but here's a Nintendo Switch one: https://news.ycombinator.com/item?id=23534793 https://news.ycombinator.com/item?id=23534793
robinsonb5 4y ago
Or perhaps making a microwave believe it's a steam oven? https://hackaday.com/2022/03/18/welcome-to-the-future-where-your-microwave-thinks-its-a-steam-oven/ https://hackaday.com/2022/03/18/welcome-to-the-future-where-...
nullc 4y ago
I had a solar inverter mfgr destroy ~$20k in batteries through a unannounced irregular remote update (that was defective) after I inadvertently left open the firewall rule that lets their monitoring platform talk to the inverter after temporarily using it. Had I known there was an update I could and likely would have caught the issue before it had a chance to over discharge the batteries, so the unauthorized remote upgrade was a causative factor-- and not just the fact that the upgrade was defective. My cable modem, which I own, is installed in a difficult to access location due to my lack of desire to run coax between buildings. In the summer every couple months it has to be rebooted when it loses lock and can't regain it on its own. I used to have a cronjob that would automatically handle it, until comcast without my authorization reached inside my property and applied a change to completely remove the ability to remotely reboot it. The remote reboot stuff was XSS vulnerable, so customers whos client PCs could access the cable modem could have had websites randomly rebooting it on them. This wasn't a problem for me, and could have been fixed by eliminating the XSS vulnerability instead. Screw giving remote access to third parties. So I think I'm pretty much 2 for 2 in terms of devices allowed remote access ending up screwing me over because of it. I can only imagine how many additional problems I would have had if I wasn't diligent in preventing any kind of remote access and avoiding having internet connected devices in the first place.
- 1123581321 4y ago
  What inverter was that?
  - nullc 4y ago
    Solaredge. I didn't even know their monitoring system could push firmware-- every prior update done had been via a SD card. It was taken care of but geesh, and left me without batteries for about a year due to parts shortages. I wouldn't use their product again for anything involving energy storage. Solar only it seems to be fine.
snarfy 4y ago
It sounds like basically every Android update I've had.
grishka 4y ago
I call this "a deliberate RCE vulnerability" because that's what it is in essence — someone else running arbitrary code on your own device without your consent.
thedanbob 4y ago
For quite a while I suspected that my ISP was pushing flaky updates to my cable modem as my internet would randomly drop until I rebooted it. My solution was to wire it into an esp8266 relay and program my home automation to power cycle it if the internet went down for more than a few minutes. Thankfully, they seem to have gotten their act together; the automation hasn’t triggered since September.
m463 4y ago
I just don't buy into the connected-to-manufacturer iot silliness. But I was bit in by this in another way. This happened with an AT&T dsl modem I had years ago. AT&T pushed an update and then my network stopped working. I had static ip addresses and all of a sudden I got no traffic. I went round and round and because the static ip devices were not their equipment, and I could connect out (on their dynamic ip) they washed their hands of it. What a mess. (I had to delve into the internal router logs, and eventually figured out the pace 5268ac modem had a bug. even though the firewall was DISABLED, packets were blocked unless I enabled firewall "stealth mode")

happymellon 4y ago

Solved, or a return.

olivierduval 4y ago

Actually, you'll get a direct return to the store with a big red label "Doesn't work"... and refund

AnimalMuppet 4y ago
Yeah. 50% won't voluntarily connect. Maybe 25% will return for refund instead of connect. (And maybe 10% don't even have internet, and aren't going to get it for a washing machine...)

H1Supreme 4y ago

Care to list the brand (model)? That seems insane to me. Firmware upgrades should be deliberate, and done with caution (hopefully starting with a backup).

2muchcoffeeman 4y ago
Any update should be deliberate. Windows will now just update it self without asking over night. I often wake up, and see that my PC laptop has rebooted into Linux since that’s at the top in the bootloader.
- eatsyourtacos 4y ago
  Eh I think that is honestly just not true with regards to Windows, sorry. There are like 1.5 billion computers running it. Them forcing updates is simply the best procedure because we live in a world full of complete asshats that want to create chaos and do malicious shit. The average person would never update their computers "deliberately" if that was the default. It's simply the lesser of all evils to ensure people are relatively up to date on security. I don't really like it either, but I understand it completely.
  - alistairSH 4y ago
    There's a massive difference between a completely silent update and the more common "Update available, install now Y/N?" Apple, MS, etc should do the second. Even with a "do it now, or within 7 days, or we do it for you." Same for the router. For goods that have safety implications (cars), the updates should NEVER be silent. And probably need a long notification period with a detailed "What's New".
    - TylerE 4y ago
      What if it's a fix for an actively-exploited zero day?
      - alistairSH 4y ago
        Maybe give me a day? I absolutely don't want my laptop updating/rebooting while I'm in the middle of work. I understand the need to keep up to date for security reasons, but at some point, I just need my device to work without interruption.
      - 7speter 4y ago
        Have the press make noise about it. This has happened a few times with i devices and intel cpus. I shouldn’t have to fight my computer to stop it from upgrading itself to Windows 11
  - domador 4y ago
nerdponx 4y ago
I've read that silently pushing firmware updates to broadband modems is standard ISP practice, at least in the USA.
- ArchOversight 4y ago
  It's part of the DOCSIS standard. Users are not allowed to modify/update the firmware on a DOCSIS modem because that could potentially be used to unlock additional speed outside of the users paid for speeds due to the way that the standard does bandwidth limiting on the modem instead of further upstream. That is why ISPs have lists of cable modems that are certified for their networks, and thus the ones allowed to connect to them as that is the list they have firmware updates for.
rnjesus 4y ago
my eero pro does this. about once a month it will decide to update itself around two in the morning without any prompt, notification, or any way to defer or delay. there is no option to adjust update times. you can check the app manually and install available updates, but if you don’t do this within a certain amount of time, it will do it automatically within a few days. three times now i have been up late working on a project and it has kicked me offline to update. it’s so goddamned annoying. i wouldn’t have bought this router if i’d known it did this. (i also had to call them on the phone to get a custom ip range enabled on it, which, for a “pro” product, is just ridiculous).
lrem 4y ago
Ever heard of Tesla? ;)
- m463 4y ago
  firmware update - now all the buttons are smaller and harder to hit!
yencabulator 4y ago
> Firmware upgrades should be deliberate, and done with caution (hopefully starting with a backup). Many, many, devices don't allow firmware downgrades, on purpose. On the extreme end, Microsoft spent a huge amount of development effort to forbid that on the hardware level. I con't find the Xbox 360 link right now, but here's a Nintendo Switch one: https://news.ycombinator.com/item?id=23534793 https://news.ycombinator.com/item?id=23534793

robinsonb5 4y ago

Or perhaps making a microwave believe it's a steam oven? https://hackaday.com/2022/03/18/welcome-to-the-future-where-your-microwave-thinks-its-a-steam-oven/ https://hackaday.com/2022/03/18/welcome-to-the-future-where-...

nullc 4y ago

I had a solar inverter mfgr destroy ~$20k in batteries through a unannounced irregular remote update (that was defective) after I inadvertently left open the firewall rule that lets their monitoring platform talk to the inverter after temporarily using it. Had I known there was an update I could and likely would have caught the issue before it had a chance to over discharge the batteries, so the unauthorized remote upgrade was a causative factor-- and not just the fact that the upgrade was defective. My cable modem, which I own, is installed in a difficult to access location due to my lack of desire to run coax between buildings. In the summer every couple months it has to be rebooted when it loses lock and can't regain it on its own. I used to have a cronjob that would automatically handle it, until comcast without my authorization reached inside my property and applied a change to completely remove the ability to remotely reboot it. The remote reboot stuff was XSS vulnerable, so customers whos client PCs could access the cable modem could have had websites randomly rebooting it on them. This wasn't a problem for me, and could have been fixed by eliminating the XSS vulnerability instead. Screw giving remote access to third parties. So I think I'm pretty much 2 for 2 in terms of devices allowed remote access ending up screwing me over because of it. I can only imagine how many additional problems I would have had if I wasn't diligent in preventing any kind of remote access and avoiding having internet connected devices in the first place.

1123581321 4y ago
What inverter was that?
- nullc 4y ago
  Solaredge. I didn't even know their monitoring system could push firmware-- every prior update done had been via a SD card. It was taken care of but geesh, and left me without batteries for about a year due to parts shortages. I wouldn't use their product again for anything involving energy storage. Solar only it seems to be fine.

snarfy 4y ago

It sounds like basically every Android update I've had.

grishka 4y ago

I call this "a deliberate RCE vulnerability" because that's what it is in essence — someone else running arbitrary code on your own device without your consent.

thedanbob 4y ago

For quite a while I suspected that my ISP was pushing flaky updates to my cable modem as my internet would randomly drop until I rebooted it. My solution was to wire it into an esp8266 relay and program my home automation to power cycle it if the internet went down for more than a few minutes. Thankfully, they seem to have gotten their act together; the automation hasn’t triggered since September.

m463 4y ago

I just don't buy into the connected-to-manufacturer iot silliness. But I was bit in by this in another way. This happened with an AT&T dsl modem I had years ago. AT&T pushed an update and then my network stopped working. I had static ip addresses and all of a sudden I got no traffic. I went round and round and because the static ip devices were not their equipment, and I could connect out (on their dynamic ip) they washed their hands of it. What a mess. (I had to delve into the internal router logs, and eventually figured out the pace 5268ac modem had a bug. even though the firewall was DISABLED, packets were blocked unless I enabled firewall "stealth mode")