3 ms·

There are 2.8 trillion lines of code in this world. If I’m an engineer hired to work on a preexisting project, like 95% of jobs are, do I want to take liability

There are 2.8 trillion lines of code in this world. If I’m an engineer hired to work on a preexisting project, like 95% of jobs are, do I want to take liability for code I didn’t write? Or for if I make a mistake when interacting with hundreds of thousands of lines of code I also didn’t write? No. What you’re suggesting is about as plausible as the Aesop fable about mice saying they should put a bell on the cat. Sounds great, completely impossible. So what about only new code then? In that case, does old code get grandfathered in? If so, Google gets to take tens of billions of lines with them for free, while startups face the audit burden which would be insurmountable to reach a similar scale. Heck, Google does not have enough skilled labor themselves to audit it all. Also completely unfeasible. And even if, even if, some country decided to audit all the code, and even if there was enough talent and labor in this world to get it done by the next decade, what does that mean? It means all research, development, and investment just moves to China and other countries that don’t require it. Also completely unfeasible. > “Their livelihoods should be on the line.” This fundamentally relies on the subject being so demonstrably knowable and predictable, that only someone guilty of negligence or malice could possibly make a mistake. This absolutely does not apply to software development, and for the reasons above, probably never will. The moment such a requirement comes into existence, any software developer who isn’t suicidal abandons the field.

whatever1 2y ago
Let’s say you are a civil engineer and your calculator had a problem spitting out wrong results the day you were calculating the amount of reinforcement for the school you were designing. If the school collapses on the kids, you are going to jail in most countries . It does not matter the calculator had an issue, you chose to use it and not verify the results.
- gjsman-1000 2y ago
  How many structural parts does a school have that need to be considered? How many iron beams? How many floors? Several thousand at most? Everything else on the BOM doesn’t matter - wallpaper isn’t a structural priority. In computer code, every last line is possibly structural. It also only takes a single missing = in the 1.2 million line codebase to kill. Comparing it to school engineering is an oversimplification. You should be comparing it to verifying the structural integrity of every skyscraper ever built; for each project.
  - aeonik 2y ago
    Chemical and elemental properties of the walls and wall paper can matter though. Leaded paint, Aresenic, flammability, weight (steel walls vs sheet rock). The complexity is still less than software though, and there are much better established standards of things that work together. Even if a screw is slightly wrong, it can still work. In software it's more like, every screw must have monocrystalline design, any grain boundary must be properly accounted for, and if not, that screw can take out an entire section of the building, possibly the whole thing, possibly the entire city.
  - whatever1 2y ago
    The claim was not that it will be easy, but since the stakes are high the buck has to stop somewhere. You cannot have Devin AI shipping crap and nobody picking up the phone when it hits the fan.
- yuvalr1 2y ago
  When handling critical software and hardware, for example automated cars, it should be the case that it's never ever the sole responsibility of a single individual. People make mistakes, and will always do. There should be a lot of safety mechanisms existing to ensure nothing critically bad ever happens due to a bug. If this is not the case, then management is to blame, and even the state, for not ensuring high quality of critical equipment. When something like that does happen, it is very hard to know the measure of responsibility every entity holds. This will most certainly be decided in court.
- jandrewrogers 2y ago
  That is because this is trivial to check and the systems are simple compared to software, so the cost imposition of the requirement to do so is minor. The software engineering equivalent would be a requirement to always check return codes. I don't think anyone believes that would move the needle in the case of software. There is a lot of literature on civil engineering failures. In fact, my civil engineering education was largely structured as a study of engineering failures. One of the most striking things about forensic analysis of civil engineering failures, a lesson the professors hammered on incessantly, is that they are almost always the result of really basic design flaws that every first-year civil engineering student can immediately recognize. There isn't some elaborate engineering discipline preventing failures in civil engineering. Exotic failures are exceedingly rare.
  - thfuran 2y ago
    So the defense for software is "I helped build a system so complex that even attempting to determine how it might fail was too hard, so it can't be my fault that it failed"?
    - gjsman-1000 2y ago
      Unless you want to go back to the steam age, it’s not a defense, but all we are humanly capable of. Never forget as well that it only takes a single cosmic ray to flip a bit. Even if you code perfectly, it can still fail, whether in this way or countless other black swans. 2.8 trillion lines of code aren’t going to rewrite themselves overnight. And as any software developer can tell you, a rewrite would almost certainly just make things worse.
      - thfuran 2y ago
        It would cost an untenable amount of money to rebuild all buildings, but that hasn't stopped us from creating and updating building codes.
        theamk 2y ago
- hn367621 2y ago
  More like your engineering organization gets sued into oblivion because it created design processes with a single point of failure. Things happen all the time. That’s why well run organizations have processes in place to catch and deal with them. In software, when people think it counts, they do too. The problem is not all people agree on “when it counts”
thfuran 2y ago
>do I want to take liability I don't suppose most doctors or civil engineers want to take liability either.
hulitu 2y ago
> do I want to take liability for code I didn’t write? That's the problem, isn't it ? Nobody wants to assume liability. They all want just the profits. Think about, if your car will come with a disclaimer like Microsoft's EULA.

whatever1 2y ago

Let’s say you are a civil engineer and your calculator had a problem spitting out wrong results the day you were calculating the amount of reinforcement for the school you were designing. If the school collapses on the kids, you are going to jail in most countries . It does not matter the calculator had an issue, you chose to use it and not verify the results.

gjsman-1000 2y ago
How many structural parts does a school have that need to be considered? How many iron beams? How many floors? Several thousand at most? Everything else on the BOM doesn’t matter - wallpaper isn’t a structural priority. In computer code, every last line is possibly structural. It also only takes a single missing = in the 1.2 million line codebase to kill. Comparing it to school engineering is an oversimplification. You should be comparing it to verifying the structural integrity of every skyscraper ever built; for each project.
- aeonik 2y ago
  Chemical and elemental properties of the walls and wall paper can matter though. Leaded paint, Aresenic, flammability, weight (steel walls vs sheet rock). The complexity is still less than software though, and there are much better established standards of things that work together. Even if a screw is slightly wrong, it can still work. In software it's more like, every screw must have monocrystalline design, any grain boundary must be properly accounted for, and if not, that screw can take out an entire section of the building, possibly the whole thing, possibly the entire city.
- whatever1 2y ago
  The claim was not that it will be easy, but since the stakes are high the buck has to stop somewhere. You cannot have Devin AI shipping crap and nobody picking up the phone when it hits the fan.
yuvalr1 2y ago
When handling critical software and hardware, for example automated cars, it should be the case that it's never ever the sole responsibility of a single individual. People make mistakes, and will always do. There should be a lot of safety mechanisms existing to ensure nothing critically bad ever happens due to a bug. If this is not the case, then management is to blame, and even the state, for not ensuring high quality of critical equipment. When something like that does happen, it is very hard to know the measure of responsibility every entity holds. This will most certainly be decided in court.
jandrewrogers 2y ago
That is because this is trivial to check and the systems are simple compared to software, so the cost imposition of the requirement to do so is minor. The software engineering equivalent would be a requirement to always check return codes. I don't think anyone believes that would move the needle in the case of software. There is a lot of literature on civil engineering failures. In fact, my civil engineering education was largely structured as a study of engineering failures. One of the most striking things about forensic analysis of civil engineering failures, a lesson the professors hammered on incessantly, is that they are almost always the result of really basic design flaws that every first-year civil engineering student can immediately recognize. There isn't some elaborate engineering discipline preventing failures in civil engineering. Exotic failures are exceedingly rare.
- thfuran 2y ago
  So the defense for software is "I helped build a system so complex that even attempting to determine how it might fail was too hard, so it can't be my fault that it failed"?
  - gjsman-1000 2y ago
    Unless you want to go back to the steam age, it’s not a defense, but all we are humanly capable of. Never forget as well that it only takes a single cosmic ray to flip a bit. Even if you code perfectly, it can still fail, whether in this way or countless other black swans. 2.8 trillion lines of code aren’t going to rewrite themselves overnight. And as any software developer can tell you, a rewrite would almost certainly just make things worse.
    - thfuran 2y ago
      It would cost an untenable amount of money to rebuild all buildings, but that hasn't stopped us from creating and updating building codes.
      - theamk 2y ago
hn367621 2y ago
More like your engineering organization gets sued into oblivion because it created design processes with a single point of failure. Things happen all the time. That’s why well run organizations have processes in place to catch and deal with them. In software, when people think it counts, they do too. The problem is not all people agree on “when it counts”

thfuran 2y ago

>do I want to take liability I don't suppose most doctors or civil engineers want to take liability either.

hulitu 2y ago

> do I want to take liability for code I didn’t write? That's the problem, isn't it ? Nobody wants to assume liability. They all want just the profits. Think about, if your car will come with a disclaimer like Microsoft's EULA.