10 ms·

I can see this being an issue for embedded systems that were set up before Let’s Encrypt added their new root authority. Some of these embedded systems won’t ha

I can see this being an issue for embedded systems that were set up before Let’s Encrypt added their new root authority. Some of these embedded systems won’t have an automatic way to update their trust anchors like browsers and operating systems typically do and so 30 September 2021 will be their Y2K date. RIP oh ye IOT toaster!

CircleSpokes 5y ago
Yeah such systems will be problematic. Those type of systems also have similar problems in cases where a certificate has been revoked. Thankfully that isn't a very big problem (at least at the moment).
jeroenhd 5y ago
If those devices require LE and still haven't received (automatic) updates already, I'd consider them IoT trash regardless. It's probably part of some IoT botnet already. Embedded devices that aren't updated regularly for long periods of time are the reason we can't have nice things on the internet.
- DelightOne 5y ago
  I imagine Botnets may even have an interest in updating the devices themselves..
  - adrianN 5y ago
    The Invisible Hand keeping your IoT devices chugging along... interesting thought!
    - sysadmindotfail 5y ago
      It's not hard to imagine someone patching firmware out of sheer annoyance/efficiency. In fact, I bet someone can provide examples of product improvement along the journey of nefarious takeover.
  - jeroenhd 5y ago
    Botnets definitely tend to patch or secure devices they've taken over. Plenty of them kill the ssh or telnet daemon after getting access, and some even patch known vulnerabilities in the web interface. It's a weird battle for control over someone else's hardware.
- nobody9999 5y ago
  >Embedded devices that aren't updated regularly for long periods of time are the reason we can't have nice things on the internet. No. Embedded devices should generally not be on the Internet at all. Rather, they should have local management/update points and zero access to the Internet. We still wouldn't be able to have nice things on the Internet, due to all the general purpose devices that have been compromised, but there's no reason a light bulb, a refrigerator, a TV or (especially) a camera or microphone should have any contact with the external world. But that breaks the "we're going to display ads on stuff you paid for, and if we spy on you, we can charge more for those ads" business model, and as such, I won't hold my breath and will continue to aggressively manage my DNS and network egress rules.
  - quaintdev 5y ago
    > No. Embedded devices should generally not be on the Internet at all. Rather, they should have local management/update points and zero access to the Internet. This! So much this. There should exist a gateway between these devices and Internet. IoT devices should not have to send data across the Interent in the first place unless user wants to access his home remotely.
- White_Wolf 5y ago
  I would consider IOT trash any IOT device that demands/requires an internet connection to be functional. The same would apply to any smartphone that requires a sim on first start (or after factory reset). Embedded devices should not even be alowed online. Automatic updates I have mixed feelings about because they (sometimes) have a bad habit of breaking things and give me a ton of hassle.
  - xvector 5y ago
    Thread is the future of IoT. Finally a local mesh network that works.
    - Avamander 5y ago
      Not with that idiotic name. The authors deserve a strong paddling.
  - jeroenhd 5y ago
    Honestly, if the internet isn't required for the device to function, I think it shouldn't even be IoT in the first place. I think automatic updates should be part of the device's default lifecycle because nobody will download a ZIP and stick it onto a flash drive to update their fridge. If an automatic update bricks a device, it should be exchanged for free by either the point of sale or the manufacturer because it broke during normal operation. IoT vendors have been given far too much leeway and it's time they're held responsible for the crap they put on the internet.
    - White_Wolf 5y ago
      You can have IoT devices with LAN only. IoT doesn't mean that it needs to understand DNS and access WAN. For that you can have IoT gateways with proper security in place. The problem with "part of the device lifecycle" is planned obsolescence. I would be up for it though if manufacturers are forced to release the source code and all service information they have the same day they drop support/updates for a product. Edit: I don't think you realise how many applications of lot of "IoT" things have where they are only on a LAN with 0 internet access. Internet access should be optional, not mandatory.
_jal 5y ago
Let's hope for can't-connect-to-anything failure modes. The only thing worse than bricked electronic trash is functional, malicious electronic trash.
- teddyh 5y ago
  This reminds me of how forests actually benefit from the occasional wildfire.
  - ehnto 5y ago
    It's integral to many ecosystems. I am also reminded of ChaosMonkey, the tool that would periodically take parts of your infrastructure down so you can make sure your failsafes actually work in the real world.
SilverRed 5y ago
>[moved] to their own root, ISRG Root X1, that expires on 4th June 2035, giving us quite a number of years. This seems like it makes the problem even worse. If these certs expired every year or so, embedded/IoT vendors would not get away with fixing something that eventually expires since it would expire too quickly.
- michaelt 5y ago
  Do you have a solution for these certificates expiring which: (a) Doesn't involve creating a different certificate which never expires, and (b) Still works if a user leaves a device powered off in storage for a few years If so I'd love to hear it!
  - xxpor 5y ago
    We should move to never expires. What good does a cert that expires in 15 years do other than setup a ticking time bomb. On the other hand, leaf certs should expire every day. Or at most a week. Move to new root certs when the tech calls for it (like SHA1->SHA256)
    - devrand 5y ago
      The only practical benefit I can imagine is that it puts a deadline on the damage from a root compromise. Not sure how much that's actually worth though. If you're designing a device that cannot be updated (or won't be), you probably shouldn't use Web PKI. Even if the roots didn't expire, I'm sure the BRs would eventually evolve to preclude issuing leaf certs compatible with the devices.
      - deleted 5y ago
        [deleted]
    - notriddle 5y ago
      The old Let's encrypt root certificate (DST Root CA X3 [1]) was from 2001, was signed with sha1, and is 2048 bit RSA. What criteria would you use to decide expiration dates for 20 years in the future? [1]: https://crt.sh/?id=8395 https://crt.sh/?id=8395
KronisLV 5y ago
Why not just use self-signed certificates for those sorts of devices with a 50 year expiry or something like that? Surely setting up your CA or even embedding a single long term certificate (or chain) within your own device isn't too much a hassle, right? Realistically, your toaster, fridge or lightbulbs will only talk to a particular set of hosts within the vendor's walled garden, so there's no need to have a trust chain for the entirety of the Internet anyways. Provided that your IoT devices aren't doing anything shady otherwise. Assuming that you even need or want IoT devices.
- lultimouomo 5y ago
  I am deploying a system where IoT devices first contact a bootstrap endpoint, authenticated by a private CA whose root cert is hardcoded in the device firmware. The bootstrap endpoint communicates the URL and CA root cert of the actual web services. This allows us to have the services themselves implemented as serverless functions using the cloud provider HTTPS/MQTT facilities. Does anyone see potential issues with that?
  - konschubert 5y ago
    The only issue is if the private key of your root Cert gets leaked. Then you won't have a way to issue a new one to your device.
  - teddyh 5y ago
    Even if the root cert is hardcoded, the root cert might still have an expiration date, and the firmware code might still check that.
    - lultimouomo 5y ago
      Good point. When this happens I'll be dead though, and so will be those devices. If we both turn out to be immortal I'll need to provide a firmware upgrade with a new CA root, possibly 10 years or so ahead of the expiration so every device has a chance to auto-upgrade.

CircleSpokes 5y ago

Yeah such systems will be problematic. Those type of systems also have similar problems in cases where a certificate has been revoked. Thankfully that isn't a very big problem (at least at the moment).

jeroenhd 5y ago

If those devices require LE and still haven't received (automatic) updates already, I'd consider them IoT trash regardless. It's probably part of some IoT botnet already. Embedded devices that aren't updated regularly for long periods of time are the reason we can't have nice things on the internet.

DelightOne 5y ago
I imagine Botnets may even have an interest in updating the devices themselves..
- adrianN 5y ago
  The Invisible Hand keeping your IoT devices chugging along... interesting thought!
  - sysadmindotfail 5y ago
    It's not hard to imagine someone patching firmware out of sheer annoyance/efficiency. In fact, I bet someone can provide examples of product improvement along the journey of nefarious takeover.
- jeroenhd 5y ago
  Botnets definitely tend to patch or secure devices they've taken over. Plenty of them kill the ssh or telnet daemon after getting access, and some even patch known vulnerabilities in the web interface. It's a weird battle for control over someone else's hardware.
nobody9999 5y ago
>Embedded devices that aren't updated regularly for long periods of time are the reason we can't have nice things on the internet. No. Embedded devices should generally not be on the Internet at all. Rather, they should have local management/update points and zero access to the Internet. We still wouldn't be able to have nice things on the Internet, due to all the general purpose devices that have been compromised, but there's no reason a light bulb, a refrigerator, a TV or (especially) a camera or microphone should have any contact with the external world. But that breaks the "we're going to display ads on stuff you paid for, and if we spy on you, we can charge more for those ads" business model, and as such, I won't hold my breath and will continue to aggressively manage my DNS and network egress rules.
- quaintdev 5y ago
  > No. Embedded devices should generally not be on the Internet at all. Rather, they should have local management/update points and zero access to the Internet. This! So much this. There should exist a gateway between these devices and Internet. IoT devices should not have to send data across the Interent in the first place unless user wants to access his home remotely.
White_Wolf 5y ago
I would consider IOT trash any IOT device that demands/requires an internet connection to be functional. The same would apply to any smartphone that requires a sim on first start (or after factory reset). Embedded devices should not even be alowed online. Automatic updates I have mixed feelings about because they (sometimes) have a bad habit of breaking things and give me a ton of hassle.
- xvector 5y ago
  Thread is the future of IoT. Finally a local mesh network that works.
  - Avamander 5y ago
    Not with that idiotic name. The authors deserve a strong paddling.
- jeroenhd 5y ago
  Honestly, if the internet isn't required for the device to function, I think it shouldn't even be IoT in the first place. I think automatic updates should be part of the device's default lifecycle because nobody will download a ZIP and stick it onto a flash drive to update their fridge. If an automatic update bricks a device, it should be exchanged for free by either the point of sale or the manufacturer because it broke during normal operation. IoT vendors have been given far too much leeway and it's time they're held responsible for the crap they put on the internet.
  - White_Wolf 5y ago
    You can have IoT devices with LAN only. IoT doesn't mean that it needs to understand DNS and access WAN. For that you can have IoT gateways with proper security in place. The problem with "part of the device lifecycle" is planned obsolescence. I would be up for it though if manufacturers are forced to release the source code and all service information they have the same day they drop support/updates for a product. Edit: I don't think you realise how many applications of lot of "IoT" things have where they are only on a LAN with 0 internet access. Internet access should be optional, not mandatory.

_jal 5y ago

Let's hope for can't-connect-to-anything failure modes. The only thing worse than bricked electronic trash is functional, malicious electronic trash.

teddyh 5y ago
This reminds me of how forests actually benefit from the occasional wildfire.
- ehnto 5y ago
  It's integral to many ecosystems. I am also reminded of ChaosMonkey, the tool that would periodically take parts of your infrastructure down so you can make sure your failsafes actually work in the real world.

SilverRed 5y ago

>[moved] to their own root, ISRG Root X1, that expires on 4th June 2035, giving us quite a number of years. This seems like it makes the problem even worse. If these certs expired every year or so, embedded/IoT vendors would not get away with fixing something that eventually expires since it would expire too quickly.

michaelt 5y ago
Do you have a solution for these certificates expiring which: (a) Doesn't involve creating a different certificate which never expires, and (b) Still works if a user leaves a device powered off in storage for a few years If so I'd love to hear it!
- xxpor 5y ago
  We should move to never expires. What good does a cert that expires in 15 years do other than setup a ticking time bomb. On the other hand, leaf certs should expire every day. Or at most a week. Move to new root certs when the tech calls for it (like SHA1->SHA256)
  - devrand 5y ago
    The only practical benefit I can imagine is that it puts a deadline on the damage from a root compromise. Not sure how much that's actually worth though. If you're designing a device that cannot be updated (or won't be), you probably shouldn't use Web PKI. Even if the roots didn't expire, I'm sure the BRs would eventually evolve to preclude issuing leaf certs compatible with the devices.
    - deleted 5y ago
      [deleted]
  - notriddle 5y ago
    The old Let's encrypt root certificate (DST Root CA X3 [1]) was from 2001, was signed with sha1, and is 2048 bit RSA. What criteria would you use to decide expiration dates for 20 years in the future? [1]: https://crt.sh/?id=8395 https://crt.sh/?id=8395

KronisLV 5y ago

Why not just use self-signed certificates for those sorts of devices with a 50 year expiry or something like that? Surely setting up your CA or even embedding a single long term certificate (or chain) within your own device isn't too much a hassle, right? Realistically, your toaster, fridge or lightbulbs will only talk to a particular set of hosts within the vendor's walled garden, so there's no need to have a trust chain for the entirety of the Internet anyways. Provided that your IoT devices aren't doing anything shady otherwise. Assuming that you even need or want IoT devices.

lultimouomo 5y ago
I am deploying a system where IoT devices first contact a bootstrap endpoint, authenticated by a private CA whose root cert is hardcoded in the device firmware. The bootstrap endpoint communicates the URL and CA root cert of the actual web services. This allows us to have the services themselves implemented as serverless functions using the cloud provider HTTPS/MQTT facilities. Does anyone see potential issues with that?
- konschubert 5y ago
  The only issue is if the private key of your root Cert gets leaked. Then you won't have a way to issue a new one to your device.
- teddyh 5y ago
  Even if the root cert is hardcoded, the root cert might still have an expiration date, and the firmware code might still check that.
  - lultimouomo 5y ago
    Good point. When this happens I'll be dead though, and so will be those devices. If we both turn out to be immortal I'll need to provide a firmware upgrade with a new CA root, possibly 10 years or so ahead of the expiration so every device has a chance to auto-upgrade.