5 ms·

> Standardized food safety practices Food safety practices only became standardized after regulation was enacted. > pre-approved and comparatively trivial rec

> Standardized food safety practices Food safety practices only became standardized after regulation was enacted. > pre-approved and comparatively trivial recipes That sounds like most software development. I think you are unwittingly making the case that software development is a lot like food production. Software development is only beginning to get regulated because it is only now reaching the level where it is hazardous to public safety, unlike food production which reached that a long time ago.

giantg2 3y ago
"Food safety practices only became standardized after regulation was enacted." Because you actually can standardize them. Software isn't so simple. "> pre-approved and comparatively trivial recipes That sounds like most software development." Lol no that does not. Why wouldn't high school graduates or drop outs work in software instead of at fast food? The number of languages, frameworks, patterns, etc are much more complex than basic sanitation and time/temp/acidity.
- kube-system 3y ago
  > Because you actually can standardize them. Software isn't so simple. It isn't simple due to choice, not due to the nature of software. Software is relatively simple compared to other meat-space engineering disciplines. Software engineering is an relatively immature engineering discipline, but it is implicated in enough safety critical systems these days that it is about time to start maturing. It will be painful but I welcome more software regulatory standards, because it is necessary for our trade to mature.
  - SOLAR_FIELDS 3y ago
    It’s probable to make the case that some forms of software are simple enough to regulate. How many Supabase style crud apps have been made in our lifetimes (not shading Supabase, they’re just automating the commonalities here)
    - kube-system 3y ago
      All software is simple enough to regulate. You don't have to micromanage every single line someone writes to regulate something. The way most professional regulations work is that someone writes down the safety practices that should be done, and then the law requires people to do those things. For example, one might require some software to undergo various degrees of planning, testing, analysis, support, documentation, etc. Right now, the amount of planning, testing, analysis, support, and documentation required by law is generally zero. This might be fine for someone's hobby project, but it is not okay for software that human lives depend on.
      - jocoda 3y ago
        > ...for software that human lives depend on. who decides, and how?
        kube-system 3y ago
        For every other technology regulated this way, this is determined at the end application. How does someone know that a particular application is something lives depend on? Either your lawyer, insurance company, or regulator explicitly tells you.
        davorak 3y ago
        > How does someone know that a particular application is something lives depend on? Either your lawyer, insurance company, or regulator explicitly tells you. To make an analogy to the physical world. We have a company, B, that makes bolts, they publishes the characteristics of that bolt but do not certify it for any particular use. Company C makes cars and decides to use bolts form company B. It turns out that is not a good choice since company B bolts do not have the characteristics that are need to use in a car. The CRA from the a simple reading used in the discussions here[1], holds company B responsible for company C using the bolts in a way where peoples lives depend on it. This sort of reuse can be much more common in software than it is bolts for example and just like company B did not control how company C used their product after buying it open source developers do not control how others use there software but CRA might make them liable for it. This does not make sense to me, company C should be liable for their choice of bolt, company B should be liable for any false or incorrect claims for the characteristics of their bolt. Company B should not be held liable for the misuse of their bolt by company C which is what the CRA seems to do. [1] https://news.ycombinator.com/item?id=38788919 https://news.ycombinator.com/item?id=38788919
  - jandrewrogers 3y ago
    > Software is relatively simple compared to other meat-space engineering disciplines. On what basis do you make this claim? If you do the same degree of optimization in software that is routinely applied in physical engineering disciplines that have some of most complex system dynamics problems, such as chemical engineering, the dynamics of software systems are qualitatively much more complex. We expect chemical engineering to design systems that asymptotically approach theoretically optimal efficiency along multiple dimensions. In software we rarely see anything approaching similar optimality except for small and exquisitely engineered components that are beyond the ken of most software engineers. In large software systems, the design problem is so complex that computational optimization to the degree we see in physical engineering is completely intractable, so similar approaches do not apply. In chemical engineering, the measure of system complexity is roughly the size of the system of differential equations the govern the total dynamics of the system. Computers then solve for the system, which can be computationally intensive. We do this routinely, with some caveats. An optimal design is not computable but we can get asymptotically close via approximation. In software engineering, the equivalent would be formal optimization and verification of the entire program. The complexity of doing this for non-trivial software is completely intractable. Software has so many degrees of freedom compared to physical systems that they aren’t even the same class of problem. It is arguable if it is even possible in theory to achieve similar degrees of design robustness and efficiency that we see in physical engineering systems. Unlike physical engineering, where a computer takes a set of equations and constraints, crunches numbers, and produces an approximately optimal design, no such thing is possible in software.
    - kube-system 3y ago
      I wasn't really thinking "complexity" in terms of formal academic problem scope, but more so "complexity" in the surface of how it interacts with the rest of the world, which is more along the lines of what would be relevant to a regulator. A regulator doesn't really care about the internal complexities of an LLM and whether or not that is more difficult than cracking petroleum. They care more about how those things interact with the rest of the world. Software is pretty limited in how it interacts with the rest of the world.
erik_seaberg 3y ago
Knuth’s code has bugs. NASA’s code has bugs. I would like to think that someday our profession might be able to achieve high enough quality to survive with liability, but today nobody is close to that at all.
- gavinhoward 3y ago
  I think that liability shouldn't require perfection, just close enough as long as the criteria is objective. I personally think that any criteria that SQLite and Curl can't pass is too strict.
  - erik_seaberg 3y ago
    The AMA doesn’t require perfection, yet a doctor has to pay six-figure liability insurance premiums for the risk of harming a small fraction of his patients. I don’t have faith that this would be run more practically.
    - gavinhoward 3y ago
      We have that problem in the medical world, but for some reason, we don't have it in the engineering world. Why? I don't know. Is the medical world just messed up? Or is there something wrong with licensure?
      - erik_seaberg 3y ago
        I think it’s because civil and mechanical engineering weren’t invented from scratch in living memory. We already have some safe, conservative materials and designs for them to reuse. Our profession is still in a very early stage, sort of like the era of barbers performing surgery.

giantg2 3y ago

"Food safety practices only became standardized after regulation was enacted." Because you actually can standardize them. Software isn't so simple. "> pre-approved and comparatively trivial recipes That sounds like most software development." Lol no that does not. Why wouldn't high school graduates or drop outs work in software instead of at fast food? The number of languages, frameworks, patterns, etc are much more complex than basic sanitation and time/temp/acidity.

kube-system 3y ago
> Because you actually can standardize them. Software isn't so simple. It isn't simple due to choice, not due to the nature of software. Software is relatively simple compared to other meat-space engineering disciplines. Software engineering is an relatively immature engineering discipline, but it is implicated in enough safety critical systems these days that it is about time to start maturing. It will be painful but I welcome more software regulatory standards, because it is necessary for our trade to mature.
- SOLAR_FIELDS 3y ago
  It’s probable to make the case that some forms of software are simple enough to regulate. How many Supabase style crud apps have been made in our lifetimes (not shading Supabase, they’re just automating the commonalities here)
  - kube-system 3y ago
    All software is simple enough to regulate. You don't have to micromanage every single line someone writes to regulate something. The way most professional regulations work is that someone writes down the safety practices that should be done, and then the law requires people to do those things. For example, one might require some software to undergo various degrees of planning, testing, analysis, support, documentation, etc. Right now, the amount of planning, testing, analysis, support, and documentation required by law is generally zero. This might be fine for someone's hobby project, but it is not okay for software that human lives depend on.
    - jocoda 3y ago
      > ...for software that human lives depend on. who decides, and how?
      - kube-system 3y ago
        For every other technology regulated this way, this is determined at the end application. How does someone know that a particular application is something lives depend on? Either your lawyer, insurance company, or regulator explicitly tells you.
        davorak 3y ago
        > How does someone know that a particular application is something lives depend on? Either your lawyer, insurance company, or regulator explicitly tells you. To make an analogy to the physical world. We have a company, B, that makes bolts, they publishes the characteristics of that bolt but do not certify it for any particular use. Company C makes cars and decides to use bolts form company B. It turns out that is not a good choice since company B bolts do not have the characteristics that are need to use in a car. The CRA from the a simple reading used in the discussions here[1], holds company B responsible for company C using the bolts in a way where peoples lives depend on it. This sort of reuse can be much more common in software than it is bolts for example and just like company B did not control how company C used their product after buying it open source developers do not control how others use there software but CRA might make them liable for it. This does not make sense to me, company C should be liable for their choice of bolt, company B should be liable for any false or incorrect claims for the characteristics of their bolt. Company B should not be held liable for the misuse of their bolt by company C which is what the CRA seems to do. [1] https://news.ycombinator.com/item?id=38788919 https://news.ycombinator.com/item?id=38788919
- jandrewrogers 3y ago
  > Software is relatively simple compared to other meat-space engineering disciplines. On what basis do you make this claim? If you do the same degree of optimization in software that is routinely applied in physical engineering disciplines that have some of most complex system dynamics problems, such as chemical engineering, the dynamics of software systems are qualitatively much more complex. We expect chemical engineering to design systems that asymptotically approach theoretically optimal efficiency along multiple dimensions. In software we rarely see anything approaching similar optimality except for small and exquisitely engineered components that are beyond the ken of most software engineers. In large software systems, the design problem is so complex that computational optimization to the degree we see in physical engineering is completely intractable, so similar approaches do not apply. In chemical engineering, the measure of system complexity is roughly the size of the system of differential equations the govern the total dynamics of the system. Computers then solve for the system, which can be computationally intensive. We do this routinely, with some caveats. An optimal design is not computable but we can get asymptotically close via approximation. In software engineering, the equivalent would be formal optimization and verification of the entire program. The complexity of doing this for non-trivial software is completely intractable. Software has so many degrees of freedom compared to physical systems that they aren’t even the same class of problem. It is arguable if it is even possible in theory to achieve similar degrees of design robustness and efficiency that we see in physical engineering systems. Unlike physical engineering, where a computer takes a set of equations and constraints, crunches numbers, and produces an approximately optimal design, no such thing is possible in software.
  - kube-system 3y ago
    I wasn't really thinking "complexity" in terms of formal academic problem scope, but more so "complexity" in the surface of how it interacts with the rest of the world, which is more along the lines of what would be relevant to a regulator. A regulator doesn't really care about the internal complexities of an LLM and whether or not that is more difficult than cracking petroleum. They care more about how those things interact with the rest of the world. Software is pretty limited in how it interacts with the rest of the world.

erik_seaberg 3y ago

Knuth’s code has bugs. NASA’s code has bugs. I would like to think that someday our profession might be able to achieve high enough quality to survive with liability, but today nobody is close to that at all.

gavinhoward 3y ago
I think that liability shouldn't require perfection, just close enough as long as the criteria is objective. I personally think that any criteria that SQLite and Curl can't pass is too strict.
- erik_seaberg 3y ago
  The AMA doesn’t require perfection, yet a doctor has to pay six-figure liability insurance premiums for the risk of harming a small fraction of his patients. I don’t have faith that this would be run more practically.
  - gavinhoward 3y ago
    We have that problem in the medical world, but for some reason, we don't have it in the engineering world. Why? I don't know. Is the medical world just messed up? Or is there something wrong with licensure?
    - erik_seaberg 3y ago
      I think it’s because civil and mechanical engineering weren’t invented from scratch in living memory. We already have some safe, conservative materials and designs for them to reuse. Our profession is still in a very early stage, sort of like the era of barbers performing surgery.