6 ms·

One thing that regulators need to be very careful about is how "security updates" are defined, and exactly what manufacturer obligations for issuing security up

by tkfu 3y ago

One thing that regulators need to be very careful about is how "security updates" are defined, and exactly what manufacturer obligations for issuing security updates should be. CVEs are a notoriously terrible representation of actual security risks, so a measure like "manufacturer must issue new releases that include any released patches for CVEs with a severity rating greater than 9" would be a clear non-starter. There are also often practical issues related to security patching embedded devices: for example, a downstream supplier's driver can make it impossible to upgrade a kernel unless/until the supplier provides a fix. Of course, strong regulation here could help to drive bad practices like that out of the industry, but I'm not going to hold my breath on that one. The effect of regulation like this would make it harder for manufacturers who don't have the market power to lean on their suppliers to provide security patches. Finally, it's important that any regulation that mandates or strongly encourages software updates also mandates that the update system itself be implemented in a secure way. This is my specific area of expertise, and I can tell you that it's very often done very badly. A bad update system is a gigantic, flashing red target for attack. So something like mandating signatures (and sig validation) on software update images would be a good start. Mandating the use of TUF-compliant repositories would be even better.

SimingtonFCC 3y ago
My two cents is that this would be an excellent comment on the record -- I'd love a discussion at the level of defining security risks to be part of the official federal commentary, because this is going to be a thorny implementation problem.
- holmesworcester 3y ago
  It would be great for people to post an update like "comment submitted" on threads like this one to make sure it was entered as a comment into the official record. I'm sure these comments in themselves are helpful to @SimingtonFCC individually, but having them be part of the official record gives the FCC legal grounds to consider them and incorporate them into rules.
  - SimingtonFCC 3y ago
    Completely agree! The public record in this case is going to be what agencies and industry looks to, far more than whatever I might happen to personally believe. I'm going to get as much information from this discussion as I can, but every participant should feel free to comment on the record, or to get their employers, companies, trade associations, ad-hoc working groups, concerned citizens congregating on Discord to complain, etc. to do so as well.
MattPalmer1086 3y ago
While I acknowledge that CVE scoring of risk can be inconsistent and sometimes wildly wrong, what would you suggest in its place?
- tkfu 3y ago
  That's the problem, there isn't a good objective measure. Some type of "reasonableness" standard is usually invoked in situations like this, but that kinda just takes us back to square one: what's currently considered reasonable in the industry is pretty terrible.
  - MattPalmer1086 3y ago
    I'm not sure we will ever have a universally accepted objective measure of risk. Risk is, by its nature, somewhat subjective. Most organisations will use CVEs and the CVSS system as a starting point, but will triage them and produce their own assessment of the actual risk to them and their products given how the software is used.
  - whats_a_quasar 3y ago
    I don't think a legal reasonableness standard would be the same as "common industry behavior." Regulation would hold companies to a real reasonableness standard, as determined in the text of the regulation or by a court.
- slt2021 3y ago
  just go by past incidents. Quite often it is not software vuln that enables hacker's attack - it is insecure default config that user never changes and manufacturer supplies same default user/pw with each device. also insecure backdoors left by developers for debug purposes (or is it really debug or maybe espionage?)
  - Animats 3y ago
    > also insecure backdoors left by developers for debug purposes (or is it really debug or maybe espionage?) It should be made clear that any "backdoor" is a criminal offense under the "unauthorized access" provision of the Computer Fraud and Abuse act, unless the device is covered by an explicit remote maintenance agreement which imposes duties upon the maintainer.
btown 3y ago
I'm curious about your thoughts on balancing the damage of another Mirai with the damage of another SolarWinds. A regulation where every IoT device must accept a signed OTA update would make update servers an extremely valuable target for supply chain compromises. On the one hand, without updates, a world of IoT devices will inevitably get infected slowly and permanently (as long as they're physically active). But on the other hand, with mandatory updates, a world of IoT devices can get infected all at once (in the case of a supply chain attack) and possibly just as permanently (if the attacker's payload can disable or re-route the update system)? Do you think that prevailing security standards for IoT manufacturers are good enough that this balance falls in favor of a mandatory-update regulation?
- SimingtonFCC 3y ago
  I don't know about a mandatory update regulation -- one way or the other, that isn't on the table right now. I would love extensive discussion on the record, however, of the costs and benefits of requiring updates to get the label.
rollcat 3y ago
> There are also often practical issues related to security patching embedded devices: for example, a downstream supplier's driver can make it impossible to upgrade a kernel unless/until the supplier provides a fix. Of course, strong regulation here could help to drive bad practices like that out of the industry, but I'm not going to hold my breath on that one. The effect of regulation like this would make it harder for manufacturers who don't have the market power to lean on their suppliers to provide security patches. This. We were building an IoT product that was effectively stuck on a derivative of Ubuntu 18.04; we couldn't upgrade because vendor wouldn't rebase on a new LTS for a very long time. As our project was being developed in Python, we were stuck on 3.6, and as it reached EOL, many third-party libraries dropped support and wouldn't even release security fixes; we needed to stay on that particular OS because of hardware support; and moving off the distribution-provided Python packages would increase maintenance burden beyond what we were able to handle. Even if the vendor would continue to provide security updates to the base OS and its packages, any real-world software solution will rely on third party packages, which may choose to drop support. I would love it if the lawmakers considered this scenario.
- oezi 3y ago
  Then fire your shitty vendor or refund your customers. Nothing will change unless everybody changes.
  - jnwatson 3y ago
    You can't fire your SoC vendor especially once the product ships. And their are all PITA about security updates.
    - oezi 3y ago
      If you buy from a supplier with a contract that stipulates security updates then you certainly would define the damages which failure to fix will cause you, wouldn't you?
      - AnthonyMouse 3y ago
MarcoPerazaFCC 3y ago
Thank you for these thoughtful points. Some relevant responses from other threads: From https://news.ycombinator.com/item?id=37394188 https://news.ycombinator.com/item?id=37394188 : I think you're right that it would be difficult for the FCC to precisely define exactly when security updates are required. This is a problem in law generally, one that is usually resolved by imposing a reasonableness standard. Maybe here, a vulnerability needs to be patched if it might reasonably be expected to allow an attacker to take control of a device, or to do so when combined with other known or unknown vulnerabilities. Or maybe a different standard. Then when enforcement/lawsuits come around, the judge/jury/regulator has to evaluate the reasonableness of the manufacturer's actions in light of that standard. We'd love to see commentary on the record as to what the right legal standard might be. From https://news.ycombinator.com/item?id=37394793 https://news.ycombinator.com/item?id=37394793 : Agreed. Building an automatic firmware update system from scratch would be burdensome for many IoT makers, but as it becomes necessary or encouraged, we would expect the market to provide a packaged solution/framework that manufacturers could fold into their products. It would be really helpful have to discussion of this on the record. How generalizable do you think such a solution could be? We are aware of the Uptane project, an OTA firmware update framework being jointly worked on by several car manufacturers, but would love to hear more about the feasibility of a solution for IoT devices generally, or particular classes of IoT devices. From https://news.ycombinator.com/item?id=37393926 https://news.ycombinator.com/item?id=37393926 : [...] companies wanting to put a label on their product would probably want to extract similar guarantees up their supply chain. Especially with a voluntary program like the one the FCC is proposing, good practices won't become the norm across the market overnight. But maybe, at the very least, the segment of product and component makers that take security seriously will begin to grow. I encourage you to share your thoughts in an official comment.
- perihelions 3y ago
  You're the lawyer guy? What statutory authority are you drawing on that you believe allows you, the FCC, to regulate this stuff? Thanks!
  - MarcoPerazaFCC 3y ago
xp84 3y ago
> regulation like this would make it harder for manufacturers who don't have the market power to lean on their suppliers to provide security patches. Thought question (I’m asking, I don’t know the “answer”): Today, many of these devices are marketed and sold by a company that has little to no involvement in the creation of the firmware or software, besides maybe sending over an image of their logo to be rolled into some turnkey “app.” Would we actually be better off if companies couldn’t really afford to basically dropship some sketchy white-label Chinese product, and instead could only sell a product here if they were confident they (acting alone) would be fully capable of supporting and updating it for a reasonable lifetime? Yes, it would raise the barrier to entry above basically the floor where it is today, but I don’t imagine there is a way to have it both ways.

by tkfu 3y ago

SimingtonFCC 3y ago

My two cents is that this would be an excellent comment on the record -- I'd love a discussion at the level of defining security risks to be part of the official federal commentary, because this is going to be a thorny implementation problem.

holmesworcester 3y ago
It would be great for people to post an update like "comment submitted" on threads like this one to make sure it was entered as a comment into the official record. I'm sure these comments in themselves are helpful to @SimingtonFCC individually, but having them be part of the official record gives the FCC legal grounds to consider them and incorporate them into rules.
- SimingtonFCC 3y ago
  Completely agree! The public record in this case is going to be what agencies and industry looks to, far more than whatever I might happen to personally believe. I'm going to get as much information from this discussion as I can, but every participant should feel free to comment on the record, or to get their employers, companies, trade associations, ad-hoc working groups, concerned citizens congregating on Discord to complain, etc. to do so as well.

MattPalmer1086 3y ago

While I acknowledge that CVE scoring of risk can be inconsistent and sometimes wildly wrong, what would you suggest in its place?

tkfu 3y ago
That's the problem, there isn't a good objective measure. Some type of "reasonableness" standard is usually invoked in situations like this, but that kinda just takes us back to square one: what's currently considered reasonable in the industry is pretty terrible.
- MattPalmer1086 3y ago
  I'm not sure we will ever have a universally accepted objective measure of risk. Risk is, by its nature, somewhat subjective. Most organisations will use CVEs and the CVSS system as a starting point, but will triage them and produce their own assessment of the actual risk to them and their products given how the software is used.
- whats_a_quasar 3y ago
  I don't think a legal reasonableness standard would be the same as "common industry behavior." Regulation would hold companies to a real reasonableness standard, as determined in the text of the regulation or by a court.
slt2021 3y ago
just go by past incidents. Quite often it is not software vuln that enables hacker's attack - it is insecure default config that user never changes and manufacturer supplies same default user/pw with each device. also insecure backdoors left by developers for debug purposes (or is it really debug or maybe espionage?)
- Animats 3y ago
  > also insecure backdoors left by developers for debug purposes (or is it really debug or maybe espionage?) It should be made clear that any "backdoor" is a criminal offense under the "unauthorized access" provision of the Computer Fraud and Abuse act, unless the device is covered by an explicit remote maintenance agreement which imposes duties upon the maintainer.

btown 3y ago

I'm curious about your thoughts on balancing the damage of another Mirai with the damage of another SolarWinds. A regulation where every IoT device must accept a signed OTA update would make update servers an extremely valuable target for supply chain compromises. On the one hand, without updates, a world of IoT devices will inevitably get infected slowly and permanently (as long as they're physically active). But on the other hand, with mandatory updates, a world of IoT devices can get infected all at once (in the case of a supply chain attack) and possibly just as permanently (if the attacker's payload can disable or re-route the update system)? Do you think that prevailing security standards for IoT manufacturers are good enough that this balance falls in favor of a mandatory-update regulation?

SimingtonFCC 3y ago
I don't know about a mandatory update regulation -- one way or the other, that isn't on the table right now. I would love extensive discussion on the record, however, of the costs and benefits of requiring updates to get the label.

rollcat 3y ago

> There are also often practical issues related to security patching embedded devices: for example, a downstream supplier's driver can make it impossible to upgrade a kernel unless/until the supplier provides a fix. Of course, strong regulation here could help to drive bad practices like that out of the industry, but I'm not going to hold my breath on that one. The effect of regulation like this would make it harder for manufacturers who don't have the market power to lean on their suppliers to provide security patches. This. We were building an IoT product that was effectively stuck on a derivative of Ubuntu 18.04; we couldn't upgrade because vendor wouldn't rebase on a new LTS for a very long time. As our project was being developed in Python, we were stuck on 3.6, and as it reached EOL, many third-party libraries dropped support and wouldn't even release security fixes; we needed to stay on that particular OS because of hardware support; and moving off the distribution-provided Python packages would increase maintenance burden beyond what we were able to handle. Even if the vendor would continue to provide security updates to the base OS and its packages, any real-world software solution will rely on third party packages, which may choose to drop support. I would love it if the lawmakers considered this scenario.

oezi 3y ago
Then fire your shitty vendor or refund your customers. Nothing will change unless everybody changes.
- jnwatson 3y ago
  You can't fire your SoC vendor especially once the product ships. And their are all PITA about security updates.
  - oezi 3y ago
    If you buy from a supplier with a contract that stipulates security updates then you certainly would define the damages which failure to fix will cause you, wouldn't you?
    - AnthonyMouse 3y ago

MarcoPerazaFCC 3y ago

Thank you for these thoughtful points. Some relevant responses from other threads: From https://news.ycombinator.com/item?id=37394188 https://news.ycombinator.com/item?id=37394188 : I think you're right that it would be difficult for the FCC to precisely define exactly when security updates are required. This is a problem in law generally, one that is usually resolved by imposing a reasonableness standard. Maybe here, a vulnerability needs to be patched if it might reasonably be expected to allow an attacker to take control of a device, or to do so when combined with other known or unknown vulnerabilities. Or maybe a different standard. Then when enforcement/lawsuits come around, the judge/jury/regulator has to evaluate the reasonableness of the manufacturer's actions in light of that standard. We'd love to see commentary on the record as to what the right legal standard might be. From https://news.ycombinator.com/item?id=37394793 https://news.ycombinator.com/item?id=37394793 : Agreed. Building an automatic firmware update system from scratch would be burdensome for many IoT makers, but as it becomes necessary or encouraged, we would expect the market to provide a packaged solution/framework that manufacturers could fold into their products. It would be really helpful have to discussion of this on the record. How generalizable do you think such a solution could be? We are aware of the Uptane project, an OTA firmware update framework being jointly worked on by several car manufacturers, but would love to hear more about the feasibility of a solution for IoT devices generally, or particular classes of IoT devices. From https://news.ycombinator.com/item?id=37393926 https://news.ycombinator.com/item?id=37393926 : [...] companies wanting to put a label on their product would probably want to extract similar guarantees up their supply chain. Especially with a voluntary program like the one the FCC is proposing, good practices won't become the norm across the market overnight. But maybe, at the very least, the segment of product and component makers that take security seriously will begin to grow. I encourage you to share your thoughts in an official comment.

perihelions 3y ago
You're the lawyer guy? What statutory authority are you drawing on that you believe allows you, the FCC, to regulate this stuff? Thanks!
- MarcoPerazaFCC 3y ago

xp84 3y ago

> regulation like this would make it harder for manufacturers who don't have the market power to lean on their suppliers to provide security patches. Thought question (I’m asking, I don’t know the “answer”): Today, many of these devices are marketed and sold by a company that has little to no involvement in the creation of the firmware or software, besides maybe sending over an image of their logo to be rolled into some turnkey “app.” Would we actually be better off if companies couldn’t really afford to basically dropship some sketchy white-label Chinese product, and instead could only sell a product here if they were confident they (acting alone) would be fully capable of supporting and updating it for a reasonable lifetime? Yes, it would raise the barrier to entry above basically the floor where it is today, but I don’t imagine there is a way to have it both ways.