5 ms·

At this point, I have to wonder what is even the point of missives like this. There are only two things that will solve the software quality problem: 1. Econom

At this point, I have to wonder what is even the point of missives like this. There are only two things that will solve the software quality problem: 1. Economic incentives. It's all just mindless blather unless you're actually talking about ways that software vendors will be held liable for bugs in their products. If you're not talking about that, what you're saying is basically "ok pretty please" useless. 2. Reducing the complexity of making products secure in the first place. Making truly secure software products is incredibly hard in this day and age, which is one reason why demanding software product liability is so scary. Professional structural engineers, for example, are used to taking liability for their designs and buildings. But with software security the complexity is nearly infinitely higher, and making it secure is much harder to guarantee. The other thing that people often ignore, or at least don't want to admit, is that the "move fast and break things" ethos has been phenomenally successful from a business perspective. The US software industry grew exponentially faster than anyplace else in the world, even places like India that doubled down on things like the "Software Capability Maturity Model" in the early 00s, and honestly have little to show for it.

juunpp 2y ago
3. Legal incentives. When somebody dies or fails to receive critical care at a hospital because a Windows XP machined got owned, somebody should probably face something along the lines of criminal negligence.
- rockskon 2y ago
  Will Microsoft face liability if someone dies or fails to receive critical care because some infrastructure system auto-rebooted to apply an update and lost state/data relating to that patient's care?
  - mkoubaa 2y ago
    That's the only thing that'll Microsoft chill with the reboots
  - juunpp 2y ago
    Yes.
- pasc1878 2y ago
  The person who should be fined etc is the person who said put the machine on the internet.
transpute 2y ago
> economic incentives EU CRA incoming, https://ubuntu.com/blog/the-cyber-resilience-act-what-it-means-for-open-source https://ubuntu.com/blog/the-cyber-resilience-act-what-it-mea... How will the US version differ?
thayne 2y ago
> But with software security the complexity is nearly infinitely higher, and making it secure is much harder to guarantee. The other thing is that software, especially software connected software (which these days is most software) has to have a much higher level of security than most other industries. When a structural engineer builds a bridge they don't have to worry abou a large number of criminals from all over the world trying to find weaknesses that can be exploited to cause the bridge to collapse. But software engineers do have to worry about hackers, including state sponsored ones, constantly trying to find and exploit weaknesses in their software. I think it absurd to put blame on software engineers for failing to make flawless software instead of the bad actors that exploit bugs that would never be noticed during normal operation, or the law enforcement that is ineffective at stopping such exploitation. Now, I do think that there should be more liability if you don't take security seriously. But there is a delicate balance there. If a single simple mistake has the potential to cause devastating liability, that will seriously slow down the software industry and substantially increase the cost.
- tastyfreeze 2y ago
  Lock makers aren't liable for making pickable locks. Punish the bad actors and leave the liability to civil suits.
- Veserv 2y ago
  If a single simple mistake has the potential to cause devastating harm, then that is precisely the standard that should be demanded. If you do not want that, then you can work on something where your mistakes do not cause immense harm to others or reduce the scope of the system so such harm is not possible. Imagine a mechanical engineer making industrial robots complaining about how unfair it is that they are held liable for single simple mistakes like pulping people. What injustice! If you want to make single simple mistakes, you can work on tasks where that is unlikely to cause harm like, I don't know, designing door handles? Nothing wrong with making door handles, the stakes are just different. "But I want to work on problems that can kill people (or other devastating harm), but not be responsible for killing them (or other devastating harm)" is a utterly insane position that has taken hold in software and has no place in serious, critical systems. If you can not make systems fit for the level of harm they can cause, then you have no place making such systems. That is irrespective of whether anybody in the entire world can do so; systems inadequate to minimize harm to the degree necessary to provide a positive societal cost-benefit over their lifetime are unfit for usage.
EnigmaFlare 2y ago
I agree with her about blaming developers, not hackers. Though not to the point of liability for all developers, but maybe for a few specialist professionals who take on that responsibility and are paid appropriately for it. Hackers are essentially a force of nature that will always exist and always be unstoppable by law enforcement because they can be in whatever country doesn't enforce such laws. You wouldn't blame the wind for destroying a bridge - it's up to the engineer to expect the wind and make it secure against that. Viewing them this way makes it clear that the people responsible for hacks are the developers in the same way developers are responsible for non-security bugs. Blame is only useful if you can actually use it to alter people's behavior - which you can't for international hackers, or the wind. Banging this drum could be effective if it leads to a culture change. We already see criticism of developers of software that has obvious vulnerabilities all the time on HN, so there's already some sense that developers shouldn't be extremely negligent/incompetent around security. You can't guarantee security 100% of course, but you can have a general awareness that it's wrong to make the stupid decisions that developers keep making and are generally known to be bad practice.
- pmontra 2y ago
  Developers build insecure software in part because themselves and in part because of the decisions made by their managers up to the CEO. So when you write "developers" we must read "software development companies".
  - EnigmaFlare 2y ago
    Yes, that's what I meant too, sorry.
- gljiva 2y ago
  > I agree with her about blaming developers, not hackers. They are clearly called "villains". Wind isn't a person capable of controlling their actions. There is no intention to do harm. They aren't senseless animals either. Yes, it's developers' fault if a product isn't secure enough, but it's also not wrong to put blame on those actively exploiting that. Let's not stop blaming those who do wrong --- and that kind of hackers is doing wrong, not just the developers "making stupid decisions". Those aren't mutually exclusive
Buttons840 2y ago
A third option is to empower security researchers and hope the good guys find the security holes before the bad guys. Currently, we threaten the good guys, security researchers, with jail way to quickly. If someone presses F12 and finds a bunch of SSNs in the raw HTML of the State's web page the Governor personally threatens to send them to jail[0]. The good security researchers tip-tow around, timidly asking permission to run pentests while the bad guys do whatever they want. Protect security researchers, change the laws to empower them and give them the benefit of the doubt. I think a big reason we don't do this is it would be a burden and an embarrassment to wealthy companies. It's literally a matter of national security and we current sacrifice national security for the convenience of companies. As you say, security is hard, and putting liability for security issues on anyone is probably unreasonable. The problem is companies can have their cake and eat it too. The companies get full control over their software, and they get to pick and choose who tests the security of their systems, while at the same time having no liability. The companies are basically saying "trust me to take care of the security, also, I accept no liability"; that doesn't inspire confidence. If the liability is too much to bear, then the companies should give up control over who can test the security of their systems. [0]: https://techcrunch.com/2021/10/15/f12-isnt-hacking-missouri-governor-threatens-to-prosecute-local-journalist-for-finding-exposed-state-data/ https://techcrunch.com/2021/10/15/f12-isnt-hacking-missouri-...
- ezoe 2y ago
  It suggest that insecure software should be simply called defective product. So the security audit should be called QA. A product, which don't spend a lot on QA, profit more. Unless there will be a catastrophic incident. Also, why haven't those so called security researchers jointly criticized EDR yet? A third-party closed source kernel driver which behave like, practically same as malware.
  - Thorrez 2y ago
    Software has tons of bugs. A fraction of those bugs are security vulnerabilities. Any type of bug can be considered a defect, and thus can be considered to make the product defective. By using the terminology "defective" instead of "vulnerable" we lose the distinction between security bugs and other bugs. I don't think we want to lose that distinction.
jiggawatts 2y ago
> Making truly secure software products is incredibly hard in this day and age I politely disagree. Writing secure software is easier than ever. For example, there are several mainstream and full-featured web frameworks that use managed virtual machine runtimes. Node.js and ASP.NET come to mind, but there are many other examples. These are largely immune to memory safety attacks and the like that plague older languages. Most popular languages also have a wide variety of ORMs available that prevent SQL injection by default. Don't like heavyweight ORMs? No problem! There's like a dozen micro-ORMs like Dapper that do nothing to your SQL other than block injection vulnerabilities and eliminate the boilerplate. Similarly, web templating frameworks like Razor pages prevent script injection by default. Cloud app platforms, containerisation, or even just virtual machines make it trivial to provide hardware enforced isolation on a per-app basis instead of relying on the much weaker process isolation within an OS with shared app hosting. TLS 1.3 has essentially eliminated cryptographic vulnerabilities in network protocols. You no longer have to "think" about this concern in normal circumstances. What's even better is that back end protocols have also uniformly adopted TLS 1.3. Even Microsoft Windows has started using for the wire protocol of Microsoft SQL Server and for the SMB file sharing protocol! Most modern queues, databases, and the like use at least TLS 1.2 or the equivalent. It's now safe to have SQL[1] and SMB shares[2] exposed to the Internet. Try telling that to someone in sec-ops twenty years ago! Most modern PaaS platforms such as cloud-native databases have very fine-grained RBAC, built-in auditing, read-only modes, and other security features on by default. Developers are spoiled with features such as SAS tokens that can be used to trivially generate signed URLs with the absolute bare minimum access required. Speaking of PaaS platforms like Azure App Service, these have completely eliminated the OS management aspect of security. Developers never again need to think about operating system security updates or OS-level configuration. Just deploy your code and go. Etc... You have to be deliberately making bad choices to end up writing insecure software in 2025. Purposefully staring at the smörgåsbord of secure options and saying: "I really don't care about security, I'm doing something else instead... just because." I know that might be controversial, but seriously, the writing has been on the wall for nearly a decade now for large swaths of the IT industry. If you're picking C or even C++ in 2025 for anything you're almost certainly making a mistake. Rust is available now even in the Linux kernel, and I hear the Windows kernel is not far behind. Don't like Rust? Use Go. Don't like Go? Use .NET 9. Seriously! It's open-source, supports AOT compilation, works on Linux just fine, and is within spitting distance of C++ for performance! [1] https://learn.microsoft.com/en-us/azure/azure-sql/database/network-access-controls-overview?view=azuresql https://learn.microsoft.com/en-us/azure/azure-sql/database/n... [2] https://learn.microsoft.com/en-us/azure/storage/files/files-smb-protocol?tabs=azure-portal https://learn.microsoft.com/en-us/azure/storage/files/files-...
gwd 2y ago
> At this point, I have to wonder what is even the point of missives like this. ...It's all just mindless blather unless you're actually talking about ways that software vendors will be held liable for bugs in their products. I think that liability for bugs is exactly where she's going with this. I'm not an expert, but it sounds from a few things I've heard on some Lawfare podcasts (e.g,. [1][2]) like the idea of software liability has been discussed for quite a while now in government policy circles. This sort of public statement may be laying the groundwork for building the political will to make it happen. [1] https://www.youtube.com/watch?v=9UneL5-Q98E&pp=ygUQbGF3ZmFyZSBzb2Z0d2FyZQ%3D%3D https://www.youtube.com/watch?v=9UneL5-Q98E&pp=ygUQbGF3ZmFyZ... [2] https://www.youtube.com/watch?v=zyNft-IZm_A&pp=ygUQbGF3ZmFyZSBzb2Z0d2FyZQ%3D%3D https://www.youtube.com/watch?v=zyNft-IZm_A&pp=ygUQbGF3ZmFyZ... EDIT: > Making truly secure software products is incredibly hard in this day and age, which is one reason why demanding software product liability is so scary. Loads of companies already are liable for bugs software that runs on their products: this includes cars, airplanes, and I would presume medical devices, and so on. The response has been what's called "safety certification": as an industry, you define a process which, if followed, you can in court say "we were reasonably careful", and then you hire an evaluator to confirm that you have followed that process. These processes don't prevent all bugs, naturally, but they certainly go a long way towards reducing them. Liability for companies who don't follow appropriate standard processes would essentially prevent cloud companies cutting security to get an edge in time-to-market or cost.
- generic92034 2y ago
  > The response has been what's called "safety certification": as an industry, you define a process which, if followed, you can in court say "we were reasonably careful", and then you hire an evaluator to confirm that you have followed that process. Can you still call it "liability" when all you have to do is performing some kind of compliance theater to get rid of it?
  - telgareith 2y ago
    Ask Boeing.
  - 2y ago
impossiblefork 2y ago
You can also choose to avoid complexity. Often a shorter computer program that is easy to understand can do exactly what a more complicated program can. We can simplify interfaces between systems and ensure that their specifications are short, readable and implementable without allocation, buffers or other things that can be implemented incorrectly. We can ensure that program code is stored separately from data. Now that LLMs are getting better we could probably have them go through all our code and introduce invariants, etc. to make sure it does exactly what it's supposed to, and if it can't then a human expert can intervene for the function for which the LLM can't find the proof. I think hardware manufacturers could help too. More isolation, Harvard architectures etc. would be quite appealing.
lelanthran 2y ago
> ways that software vendors will be held liable for bugs in their products. And if we do that then the state of software would grind to a halt, because there is no software that is completely bug-free.[1] Like it or not, the market has spoken on what it considers acceptable risk for general software. Software where human lives are at risk is already highly regulated, which is why so few human lives are lost to bugs, compared to lives lost to other engineering defects. [1] I feel we are already at a point where the market has, economically anyway, hit the point of diminishing returns for investment into reliability in software.
croes 2y ago
Do get laws to punish certain behavior that behavior must be considered as a bad thing in terms of being morally wrong. So it's a first step to liability.
pjmlp 2y ago
Making vendors understand that the time of EULAS waving responsibility is coming to pass, and like in any other grown up industry, liability is coming.
soerxpso 2y ago
> Professional structural engineers, for example, are used to taking liability for their designs and buildings. But with software security the complexity is nearly infinitely higher, and making it secure is much harder to guarantee. I'm not sure about your claim that structural engineering is less complex, but there's another (arguably much more significant) difference: structural safety is against an indifferent adversary (the weather, and physics); software security is against a malicious adversary. If someone with resources wants to take down a building (with exception for certain expensive military installations), no amount of structural engineering is going to stop them. Software that isn't vulnerable to cyberattacks should be compared to a bunker that isn't vulnerable to coordinated artillery strikes, not to your average building.
nradov 2y ago
If customers want their software vendors to take liability for software defects (including security vulnerabilities) then they can just negotiate that into licensing agreements. We don't need the federal government to get involved with new laws or regulations.
cratermoon 2y ago
3. Regulations. Yes I know the typical visitor to HN is deathly allergic to government regulations and regulatory bodies in general. That attitude in tech is how we got here.

juunpp 2y ago

3. Legal incentives. When somebody dies or fails to receive critical care at a hospital because a Windows XP machined got owned, somebody should probably face something along the lines of criminal negligence.

rockskon 2y ago
Will Microsoft face liability if someone dies or fails to receive critical care because some infrastructure system auto-rebooted to apply an update and lost state/data relating to that patient's care?
- mkoubaa 2y ago
  That's the only thing that'll Microsoft chill with the reboots
- juunpp 2y ago
  Yes.
pasc1878 2y ago
The person who should be fined etc is the person who said put the machine on the internet.

transpute 2y ago

> economic incentives EU CRA incoming, https://ubuntu.com/blog/the-cyber-resilience-act-what-it-means-for-open-source https://ubuntu.com/blog/the-cyber-resilience-act-what-it-mea... How will the US version differ?

thayne 2y ago

> But with software security the complexity is nearly infinitely higher, and making it secure is much harder to guarantee. The other thing is that software, especially software connected software (which these days is most software) has to have a much higher level of security than most other industries. When a structural engineer builds a bridge they don't have to worry abou a large number of criminals from all over the world trying to find weaknesses that can be exploited to cause the bridge to collapse. But software engineers do have to worry about hackers, including state sponsored ones, constantly trying to find and exploit weaknesses in their software. I think it absurd to put blame on software engineers for failing to make flawless software instead of the bad actors that exploit bugs that would never be noticed during normal operation, or the law enforcement that is ineffective at stopping such exploitation. Now, I do think that there should be more liability if you don't take security seriously. But there is a delicate balance there. If a single simple mistake has the potential to cause devastating liability, that will seriously slow down the software industry and substantially increase the cost.

tastyfreeze 2y ago
Lock makers aren't liable for making pickable locks. Punish the bad actors and leave the liability to civil suits.
Veserv 2y ago
If a single simple mistake has the potential to cause devastating harm, then that is precisely the standard that should be demanded. If you do not want that, then you can work on something where your mistakes do not cause immense harm to others or reduce the scope of the system so such harm is not possible. Imagine a mechanical engineer making industrial robots complaining about how unfair it is that they are held liable for single simple mistakes like pulping people. What injustice! If you want to make single simple mistakes, you can work on tasks where that is unlikely to cause harm like, I don't know, designing door handles? Nothing wrong with making door handles, the stakes are just different. "But I want to work on problems that can kill people (or other devastating harm), but not be responsible for killing them (or other devastating harm)" is a utterly insane position that has taken hold in software and has no place in serious, critical systems. If you can not make systems fit for the level of harm they can cause, then you have no place making such systems. That is irrespective of whether anybody in the entire world can do so; systems inadequate to minimize harm to the degree necessary to provide a positive societal cost-benefit over their lifetime are unfit for usage.

EnigmaFlare 2y ago

I agree with her about blaming developers, not hackers. Though not to the point of liability for all developers, but maybe for a few specialist professionals who take on that responsibility and are paid appropriately for it. Hackers are essentially a force of nature that will always exist and always be unstoppable by law enforcement because they can be in whatever country doesn't enforce such laws. You wouldn't blame the wind for destroying a bridge - it's up to the engineer to expect the wind and make it secure against that. Viewing them this way makes it clear that the people responsible for hacks are the developers in the same way developers are responsible for non-security bugs. Blame is only useful if you can actually use it to alter people's behavior - which you can't for international hackers, or the wind. Banging this drum could be effective if it leads to a culture change. We already see criticism of developers of software that has obvious vulnerabilities all the time on HN, so there's already some sense that developers shouldn't be extremely negligent/incompetent around security. You can't guarantee security 100% of course, but you can have a general awareness that it's wrong to make the stupid decisions that developers keep making and are generally known to be bad practice.

pmontra 2y ago
Developers build insecure software in part because themselves and in part because of the decisions made by their managers up to the CEO. So when you write "developers" we must read "software development companies".
- EnigmaFlare 2y ago
  Yes, that's what I meant too, sorry.
gljiva 2y ago
> I agree with her about blaming developers, not hackers. They are clearly called "villains". Wind isn't a person capable of controlling their actions. There is no intention to do harm. They aren't senseless animals either. Yes, it's developers' fault if a product isn't secure enough, but it's also not wrong to put blame on those actively exploiting that. Let's not stop blaming those who do wrong --- and that kind of hackers is doing wrong, not just the developers "making stupid decisions". Those aren't mutually exclusive

Buttons840 2y ago

A third option is to empower security researchers and hope the good guys find the security holes before the bad guys. Currently, we threaten the good guys, security researchers, with jail way to quickly. If someone presses F12 and finds a bunch of SSNs in the raw HTML of the State's web page the Governor personally threatens to send them to jail[0]. The good security researchers tip-tow around, timidly asking permission to run pentests while the bad guys do whatever they want. Protect security researchers, change the laws to empower them and give them the benefit of the doubt. I think a big reason we don't do this is it would be a burden and an embarrassment to wealthy companies. It's literally a matter of national security and we current sacrifice national security for the convenience of companies. As you say, security is hard, and putting liability for security issues on anyone is probably unreasonable. The problem is companies can have their cake and eat it too. The companies get full control over their software, and they get to pick and choose who tests the security of their systems, while at the same time having no liability. The companies are basically saying "trust me to take care of the security, also, I accept no liability"; that doesn't inspire confidence. If the liability is too much to bear, then the companies should give up control over who can test the security of their systems. [0]: https://techcrunch.com/2021/10/15/f12-isnt-hacking-missouri-governor-threatens-to-prosecute-local-journalist-for-finding-exposed-state-data/ https://techcrunch.com/2021/10/15/f12-isnt-hacking-missouri-...

ezoe 2y ago
It suggest that insecure software should be simply called defective product. So the security audit should be called QA. A product, which don't spend a lot on QA, profit more. Unless there will be a catastrophic incident. Also, why haven't those so called security researchers jointly criticized EDR yet? A third-party closed source kernel driver which behave like, practically same as malware.
- Thorrez 2y ago
  Software has tons of bugs. A fraction of those bugs are security vulnerabilities. Any type of bug can be considered a defect, and thus can be considered to make the product defective. By using the terminology "defective" instead of "vulnerable" we lose the distinction between security bugs and other bugs. I don't think we want to lose that distinction.

jiggawatts 2y ago

> Making truly secure software products is incredibly hard in this day and age I politely disagree. Writing secure software is easier than ever. For example, there are several mainstream and full-featured web frameworks that use managed virtual machine runtimes. Node.js and ASP.NET come to mind, but there are many other examples. These are largely immune to memory safety attacks and the like that plague older languages. Most popular languages also have a wide variety of ORMs available that prevent SQL injection by default. Don't like heavyweight ORMs? No problem! There's like a dozen micro-ORMs like Dapper that do nothing to your SQL other than block injection vulnerabilities and eliminate the boilerplate. Similarly, web templating frameworks like Razor pages prevent script injection by default. Cloud app platforms, containerisation, or even just virtual machines make it trivial to provide hardware enforced isolation on a per-app basis instead of relying on the much weaker process isolation within an OS with shared app hosting. TLS 1.3 has essentially eliminated cryptographic vulnerabilities in network protocols. You no longer have to "think" about this concern in normal circumstances. What's even better is that back end protocols have also uniformly adopted TLS 1.3. Even Microsoft Windows has started using for the wire protocol of Microsoft SQL Server and for the SMB file sharing protocol! Most modern queues, databases, and the like use at least TLS 1.2 or the equivalent. It's now safe to have SQL[1] and SMB shares[2] exposed to the Internet. Try telling that to someone in sec-ops twenty years ago! Most modern PaaS platforms such as cloud-native databases have very fine-grained RBAC, built-in auditing, read-only modes, and other security features on by default. Developers are spoiled with features such as SAS tokens that can be used to trivially generate signed URLs with the absolute bare minimum access required. Speaking of PaaS platforms like Azure App Service, these have completely eliminated the OS management aspect of security. Developers never again need to think about operating system security updates or OS-level configuration. Just deploy your code and go. Etc... You have to be deliberately making bad choices to end up writing insecure software in 2025. Purposefully staring at the smörgåsbord of secure options and saying: "I really don't care about security, I'm doing something else instead... just because." I know that might be controversial, but seriously, the writing has been on the wall for nearly a decade now for large swaths of the IT industry. If you're picking C or even C++ in 2025 for anything you're almost certainly making a mistake. Rust is available now even in the Linux kernel, and I hear the Windows kernel is not far behind. Don't like Rust? Use Go. Don't like Go? Use .NET 9. Seriously! It's open-source, supports AOT compilation, works on Linux just fine, and is within spitting distance of C++ for performance! [1] https://learn.microsoft.com/en-us/azure/azure-sql/database/network-access-controls-overview?view=azuresql https://learn.microsoft.com/en-us/azure/azure-sql/database/n... [2] https://learn.microsoft.com/en-us/azure/storage/files/files-smb-protocol?tabs=azure-portal https://learn.microsoft.com/en-us/azure/storage/files/files-...

gwd 2y ago

> At this point, I have to wonder what is even the point of missives like this. ...It's all just mindless blather unless you're actually talking about ways that software vendors will be held liable for bugs in their products. I think that liability for bugs is exactly where she's going with this. I'm not an expert, but it sounds from a few things I've heard on some Lawfare podcasts (e.g,. [1][2]) like the idea of software liability has been discussed for quite a while now in government policy circles. This sort of public statement may be laying the groundwork for building the political will to make it happen. [1] https://www.youtube.com/watch?v=9UneL5-Q98E&pp=ygUQbGF3ZmFyZSBzb2Z0d2FyZQ%3D%3D https://www.youtube.com/watch?v=9UneL5-Q98E&pp=ygUQbGF3ZmFyZ... [2] https://www.youtube.com/watch?v=zyNft-IZm_A&pp=ygUQbGF3ZmFyZSBzb2Z0d2FyZQ%3D%3D https://www.youtube.com/watch?v=zyNft-IZm_A&pp=ygUQbGF3ZmFyZ... EDIT: > Making truly secure software products is incredibly hard in this day and age, which is one reason why demanding software product liability is so scary. Loads of companies already are liable for bugs software that runs on their products: this includes cars, airplanes, and I would presume medical devices, and so on. The response has been what's called "safety certification": as an industry, you define a process which, if followed, you can in court say "we were reasonably careful", and then you hire an evaluator to confirm that you have followed that process. These processes don't prevent all bugs, naturally, but they certainly go a long way towards reducing them. Liability for companies who don't follow appropriate standard processes would essentially prevent cloud companies cutting security to get an edge in time-to-market or cost.

generic92034 2y ago
> The response has been what's called "safety certification": as an industry, you define a process which, if followed, you can in court say "we were reasonably careful", and then you hire an evaluator to confirm that you have followed that process. Can you still call it "liability" when all you have to do is performing some kind of compliance theater to get rid of it?
- telgareith 2y ago
  Ask Boeing.
- 2y ago

impossiblefork 2y ago

You can also choose to avoid complexity. Often a shorter computer program that is easy to understand can do exactly what a more complicated program can. We can simplify interfaces between systems and ensure that their specifications are short, readable and implementable without allocation, buffers or other things that can be implemented incorrectly. We can ensure that program code is stored separately from data. Now that LLMs are getting better we could probably have them go through all our code and introduce invariants, etc. to make sure it does exactly what it's supposed to, and if it can't then a human expert can intervene for the function for which the LLM can't find the proof. I think hardware manufacturers could help too. More isolation, Harvard architectures etc. would be quite appealing.

lelanthran 2y ago

> ways that software vendors will be held liable for bugs in their products. And if we do that then the state of software would grind to a halt, because there is no software that is completely bug-free.[1] Like it or not, the market has spoken on what it considers acceptable risk for general software. Software where human lives are at risk is already highly regulated, which is why so few human lives are lost to bugs, compared to lives lost to other engineering defects. [1] I feel we are already at a point where the market has, economically anyway, hit the point of diminishing returns for investment into reliability in software.

croes 2y ago

Do get laws to punish certain behavior that behavior must be considered as a bad thing in terms of being morally wrong. So it's a first step to liability.

pjmlp 2y ago

Making vendors understand that the time of EULAS waving responsibility is coming to pass, and like in any other grown up industry, liability is coming.

soerxpso 2y ago

> Professional structural engineers, for example, are used to taking liability for their designs and buildings. But with software security the complexity is nearly infinitely higher, and making it secure is much harder to guarantee. I'm not sure about your claim that structural engineering is less complex, but there's another (arguably much more significant) difference: structural safety is against an indifferent adversary (the weather, and physics); software security is against a malicious adversary. If someone with resources wants to take down a building (with exception for certain expensive military installations), no amount of structural engineering is going to stop them. Software that isn't vulnerable to cyberattacks should be compared to a bunker that isn't vulnerable to coordinated artillery strikes, not to your average building.

nradov 2y ago

If customers want their software vendors to take liability for software defects (including security vulnerabilities) then they can just negotiate that into licensing agreements. We don't need the federal government to get involved with new laws or regulations.

cratermoon 2y ago

3. Regulations. Yes I know the typical visitor to HN is deathly allergic to government regulations and regulatory bodies in general. That attitude in tech is how we got here.