9 ms·

Software crisis: the next generation (2016)

71 points by eadmund 9y ago

informatimago 9y ago
And then, hardware is crap too...
tannhaeuser 9y ago
TFA mentions Google's Caja and Secure EcmaScript (SES). Does anybody know if that's still in development or under consideration by ECMA or others?
- cube00 9y ago
  Caja is only getting security fixes for now, one of the GitHub issues from March 2017 states: "New features for Caja are pretty much on hold for the time being. However, if you'd like to write a patch to add this one we'd be happy to review and incorporate it."
- abecedarius 9y ago
  I gather that ES6 is most of the way to SES, and the Frozen Realms proposal linked in the article is the last bit needed.
golergka 9y ago
We have some formal verification, it's called typed systems. Very limited, I know. But some people are still praising dynamically typed languages for their "speed" and "lack of compiler errors".
- NoGravitas 9y ago
  When you talk about static typing as providing some form of formal verification, you really can only talk about languages with advanced type systems like Haskell. Type systems like those in Java and C# give you very little formal verification (and I say this as someone who programs daily in C# for work and Python and Common Lisp for fun). What Java-like type systems give you is confidence in refactoring, which is not nothing, but it is not formal verification. (I'll also say that type inference and generic types in C# give me about 80% of what dynamic types in Python give me. The main thing I miss is heterogeneous tuples, which in C# have to be of type Object, or defined as structs/classes, which is too verbose for the job.)
  - golergka 9y ago
    I work with C# full-time and it's type system constantly helps me. Of course, it's not Haskell or Rust, but it is still so much better than my Javascript experience. Anonymous types, generic containers and interfaces are usually enough for my needs. Can you please go into some detail about heterogenous tuples in Python? I've never used it to build big long-lasting systems so I be not aware of some of it's type capabilities.
    - pka 9y ago
      Tuples are just two or more values with separate types lumped together, i.e. (Int, String). For example, in Haskell, to get the elements of a map, there's this function: assocs :: Map k v -> [(k, v)] which gives you an array of tuples, the first element of which is of type k (the map key type) and the second element being of type v (the map value type.)
      - golergka 9y ago
        I asked a question because I assumed that I might not know something about it, since C# has Tuple<T1, T2> generic class just for that and if it would be just that, Python would have nothing on C# in that regard. But your comment is exactly what I was thinking about. So, is there something else in Python that is not present in C#?
        pka 9y ago
        I doubt it, other than syntactic sugar, which makes tuples really easy to use, like destructuring and construction. I.e. in Python, if you have a function that returns a tuple: a, b = f(xxx) To create a tuple you just do (a, b). In Haskell, you get some other benefits, like tuples being functors (i.e. "things that can be mapped over"): map show ("tuple", 5) == ("tuple", "5") P.S. Not a Python expert, but I think tuples might be enumerable in Python, which works because Python is dynamically typed.
        klibertp 9y ago
- ajurna 9y ago
  Good to see the strict typing of C has saved it from most security issues. i think it it is quite facetious to put it down to type systems.
  - mpartel 9y ago
    C does not have particularly strict typing
  - golergka 9y ago
    There are different type systems (think C vs Rust), and having one is always better than having none. C still prevents some issues you can have with assembly.
  - naasking 9y ago
    An unsound type system doesn't prove anything about type systems. Doubly so if you don't actually use types and you're just exploiting that unsoundness with pervasive casting.
- flavio81 9y ago
  >We have some formal verification, it's called typed systems. Very limited, I know. Very limited. In my many years of experience doing software professionally, the serious bugs, that is, the ones that took us more than one day of debugging (after such a bug was able to be reproduced), were the ones that had nothing to do with types but with * bad understanding of the business rules * bad fundamental implementation of the problem domain * misuse / incorrect use of an API or libraries * API/library behaviour different from what the documentation says. No type system will save you from them. Type systems don't verify how the code does the correct thing. They only verify that the types moved between functions/modules satisfy certain conditions. Big deal. Where code reusability implies that your functions should apply to the widest type of circumstances (and thus increasing the scope in which they can be reused), type systems go fundamentally against to this goal, enforcing your functions to go very specific on what they're able to accept and return. Dynamically typed systems allow for faster coding, and also allow for interactive development which allows for faster, easier testing of individual components as they are developed. Also, the very best dynamically typed systems also allow for redefining / uploading updated function definitions while the code is running, which greatly increases development & testing speed (emphasis added.) This is not just a claim, this is my personal experience after about 23 years of programming where 90% of those years were spent using statically typed systems. As for speed, for highly optimized code Java, Haskell, and F# are basically in the same ballpark. Which is, very good speed. Well, Common Lisp (dynamically typed language) is not only in the same ballpark, but faster than them in some cases. You know what i'm going to choose. Or, different take on this: If i was going for absolute speed i'd be hacking in C, where no Hindley-Milner type checker gets in the way of clever tricks for gaining that extra bit of performance you need on that tight loop.
  - golergka 9y ago
    > In my many years of experience doing software professionally, the serious bugs... > This is not just a claim, this is my personal experience after about 23 years of programming where 90% of those years were spent using statically typed systems. If most of your experience was with statically typed systems, it's just logical you would have very little experience with bugs that are prevented by statically typed systems - because this prevention would be at work. Looks like an opposite of survivor's fallacy, really.
virgilp 9y ago
> Anybody who’s seen the systems inside a major tech company knows this is true. Or a minor tech company. Or the insides of any product with a software component. Anybody who's seen the sun rising east and setting west knows that this is true: the Sun rotates around the Earth. The other, less-obvious alternative, is that software is NOT crap, we just like to complain a lot. And it's just so easy & fun to blame everything on the software we work with. Most software that doesn't immediately die does its job fairly well. In fact, better than whatever alternative that existed before said software.
- sharemywin 9y ago
  I think the point of the article is something that does an 95% job for 10% the cost works for a lot of things, but somethings that trade off doesn't work.
  - virgilp 9y ago
    Maybe. And I could get behind a viewpoint that says, "some things are so dangerous, that a 99.999% job is not good enough". That's definitely true. But that is much less of a rant, and it requires the writer to do the hard work of defining those dangerous things, and arguing the benefits of adding the 4th, 5th, 6th nine against the cost. Saying "everything is crap" makes me just not take the article seriously, because the author only did half of the thinking job (not even half, maybe).
- AnIdiotOnTheNet 9y ago
  If our metric for "good" is based solely on what people choose to use, then one is forced to admit that we also have generally good security and privacy safeguards. Actually, I kinda agree with that assessment. "Good enough" is never good enough for academics, but practical reality is different.
  - virgilp 9y ago
    > then one is forced to admit that we also have generally good security and privacy safeguards. It's a tremendous conceit of the software community to actually believe these are purely-software problems that can be solved through software. Sure, software goes a long way, but I argue that it's a fool's errand to hope they can or should be solved in software, exclusively. Like - hardware will always be a factor, at least that should be obvious. But also culture/ society/ laws/ etc. have important practical implications. The very fact that we have lots of people scrutinizing these aspects suggests that the state of software security&privacy is really not "crap" (far from perfect, but definitely not "crap"). To you believe that article to be true, one must have a fairly black&white view of the world. 99% (probably much more) of the humans can't really bypass those "crap" security safeguards.... it's good that we have high standards (especially in areas like this), but come on. When in the human history was security(in general) better? What are we comparing it with, to postulate it's "crap"?
Annatar 9y ago
“The reason this stuff is crap is far more basic. It’s because better-than-crap costs a lot more, and crap is usually sufficient. ... ... ... Every dollar put into making software less crappy can’t be spent on other things we might also want, the list of which is basically endless.“ Software is crap because humans on the whole have an extremely difficult time reasoning through all the possible logic flows. Software is also crap because in order to design good software a lot of insight, experience and empathy is required, and as it turns out, people writing crap don’t have any of those prerequisites. Finally, most software is crap because most people writing it don’t care about getting it 100% correct, since it turns out that getting the last 5% correctly functioning is exponentially difficult, and those people would rather collect a paycheck than feel proud and content about what they wrote. All other industries manage to deliver complete, working products; we are the only industry which doesn’t; computers and software never work 100% correctly. What does that make us? It makes us jerk-offs, that’s what it makes us. I understand now why Keith Wesolowski ditched computers and went to work on a ranch. And damn it, he is right!
- mgaruccio 9y ago
  Do all other Industries manage it though? If they did we wouldn't have product recalls or any need for "consumer safety watchdogs". The problem is arguably worse in software but software can also be (relatively) easily patched later so a somewhat more lax attitude to quality on systems that are not safety critical makes some degree of sense.
- naasking 9y ago
  > Software is crap because humans on the whole have an extremely difficult time reasoning through all the possible logic flows. But we have tools and techniques that make this easier. Why don't we use them? Also, I don't think 100% correctness is the ultimate goal of this article. Software is probably nowhere even near that correctness level firstly, and secondly, software is also partly crap because it doesn't run any faster than it did 30 years ago despite tremendous performance advances in our hardware. > What does that make us? It makes us jerk-offs, that’s what it makes us. Let's be honest, software as it currently is written is considerably more complex than most other engineered systems that we take for granted. That said, we've had tools and techniques to mitigate a lot of this complexity, and to have computers double-check our work (eg. type systems and model checking), but we don't really use them as much as we should. What's worse, we even use tools that are intentionally intractable to any kind of work checking/validation (like dynamically typed languages). There's really no excuse for this.
zzzcpan 9y ago
More precise understanding of faults and how to deal with them can go a long way before suggesting formal verification. Bug is not a fault, but it may cause faults though. You can try to find this bug beforehand or you can deal with the faults at runtime. In practice not just bugs cause faults, but also hardware failures, natural disasters, human mistakes and so on, so you kind of have to deal with faults either way. > But we can certainly arrange to radically limit the scope of damage available to any particular piece of crap, which should vastly reduce systemic crappiness. Now, this is the big idea behind supervision trees. Where you split everything into the smallest possible isolated processes and supervisors that watch over them, so that in the event of any process failing it can just get restarted, limiting the scope of the problem to only that one tiny process for the shortest possible time. This idea might even reduce the cost of software development compared to some more popular software development practices. But it does require an easy to use actor model in the language.
aabbccddq 9y ago
Like anything else, you just don't see it. I thought you have a point until you mentioned Microsoft. The company that showed us it's perfectly fine and socially accepted to make top money from crap and not releasing security updates, and be proud of it. Thanks MS!
nickthemagicman 9y ago
This is cultural. The Walmart-ification of the worlds products. Planned Obsolescence. The short term profits to get the stock prices up by the end of the quarter mentality. The credit card mindset. How fast society moves, and how cutthroat capitalism is becoming...its all about short term gain because the future feels chaotic and unstable to most of the world. I personally think its the fact the money is getting more an more difficult to obtain because more and more is being hoarded in the upper strata of society. So short term mindsets on financial gain take over because financial opportunities don't come easily. This is reflected in software development.
tenpoundhammer 9y ago
>And still, this wouldn’t be so bad, if the crap wasn’t starting to seep into things that actually matter. >A leading indicator of what’s to come is the state of computer security. We’re seeing an alarming rise in big security breaches, each more epic than the next, to the point where they’re hardly news any more. Target has 70 million customers’ credit card and identity information stolen. Oh no! Security clearance and personal data for 21.5 million federal employees is taken from the Office of Personnel Management. ... I’m not a fan of this type of article, the article where we find a group of bad things and then go find something to blame for it. It’s easy to say all this stuff was caused because “we just don’t make stuff like we used to”, but I think it’s wrong-headed. Or more probably it’s used in the wrong sense. It’s true that we don’t make things like we used to. We have made giant leaps past anything we’ve ever done before. We have hundreds of thousands of companies online and millions of devices online, a level of interconnectedness that the planet has never seen before. We also have more accessible software development tools and many more people writing software than ever before. Increasing numbers of security breaches are a sign of our progress not a sign of our failures. It’s just a sign that there is a lot more stuff attached to the internet and that unfortunately includes people with the intention to commit crimes. Should we tolerate security breaches? No. Should we celebrate them? Of course not. But let’s see them for what they are and search for a solution instead of something to vilify.
- abecedarius 9y ago
  I didn’t get the impression this post meant to say we just don’t make stuff like we used to, but that the crap we’ve been taking as normal is going to have terrible consequences if we don’t mitigate it somehow soon. I agree that common practices are gradually getting better, but I don’t think that’s enough. Re: solutions, the author wrote “What are capabilities?” which got discussed here the other day.
program_whiz 9y ago
Well we would expect no different from medicine, or home construction, or automobiles without some basic guarantees: 1. High visibility of problems (people die, building collapse, crashes) 2. Regulation and inspection (some standards for evaluation and lines that should not be crossed) 3. Certification (you wouldn't let an uncertified random person do heart surgery on you, or build your house). But for this to be possible, we as a whole would have to agree these are priorities and impose these limits. We're still in the time of the "surgery without anasthesia" or "no building codes" or "just let the factories emit whatever they want". Understandable due to the short time that software has been really "important" in day-to-day life, but now its time to slow down and make sure we do it right.

71 points by eadmund 9y ago

And then, hardware is crap too...

TFA mentions Google's Caja and Secure EcmaScript (SES). Does anybody know if that's still in development or under consideration by ECMA or others?

cube00 9y ago
Caja is only getting security fixes for now, one of the GitHub issues from March 2017 states: "New features for Caja are pretty much on hold for the time being. However, if you'd like to write a patch to add this one we'd be happy to review and incorporate it."
abecedarius 9y ago
I gather that ES6 is most of the way to SES, and the Frozen Realms proposal linked in the article is the last bit needed.

golergka 9y ago

We have some formal verification, it's called typed systems. Very limited, I know. But some people are still praising dynamically typed languages for their "speed" and "lack of compiler errors".

NoGravitas 9y ago
When you talk about static typing as providing some form of formal verification, you really can only talk about languages with advanced type systems like Haskell. Type systems like those in Java and C# give you very little formal verification (and I say this as someone who programs daily in C# for work and Python and Common Lisp for fun). What Java-like type systems give you is confidence in refactoring, which is not nothing, but it is not formal verification. (I'll also say that type inference and generic types in C# give me about 80% of what dynamic types in Python give me. The main thing I miss is heterogeneous tuples, which in C# have to be of type Object, or defined as structs/classes, which is too verbose for the job.)
- golergka 9y ago
  I work with C# full-time and it's type system constantly helps me. Of course, it's not Haskell or Rust, but it is still so much better than my Javascript experience. Anonymous types, generic containers and interfaces are usually enough for my needs. Can you please go into some detail about heterogenous tuples in Python? I've never used it to build big long-lasting systems so I be not aware of some of it's type capabilities.
  - pka 9y ago
    Tuples are just two or more values with separate types lumped together, i.e. (Int, String). For example, in Haskell, to get the elements of a map, there's this function: assocs :: Map k v -> [(k, v)] which gives you an array of tuples, the first element of which is of type k (the map key type) and the second element being of type v (the map value type.)
    - golergka 9y ago
      I asked a question because I assumed that I might not know something about it, since C# has Tuple<T1, T2> generic class just for that and if it would be just that, Python would have nothing on C# in that regard. But your comment is exactly what I was thinking about. So, is there something else in Python that is not present in C#?
      - pka 9y ago
        I doubt it, other than syntactic sugar, which makes tuples really easy to use, like destructuring and construction. I.e. in Python, if you have a function that returns a tuple: a, b = f(xxx) To create a tuple you just do (a, b). In Haskell, you get some other benefits, like tuples being functors (i.e. "things that can be mapped over"): map show ("tuple", 5) == ("tuple", "5") P.S. Not a Python expert, but I think tuples might be enumerable in Python, which works because Python is dynamically typed.
        klibertp 9y ago
ajurna 9y ago
Good to see the strict typing of C has saved it from most security issues. i think it it is quite facetious to put it down to type systems.
- mpartel 9y ago
  C does not have particularly strict typing
- golergka 9y ago
  There are different type systems (think C vs Rust), and having one is always better than having none. C still prevents some issues you can have with assembly.
- naasking 9y ago
  An unsound type system doesn't prove anything about type systems. Doubly so if you don't actually use types and you're just exploiting that unsoundness with pervasive casting.
flavio81 9y ago
>We have some formal verification, it's called typed systems. Very limited, I know. Very limited. In my many years of experience doing software professionally, the serious bugs, that is, the ones that took us more than one day of debugging (after such a bug was able to be reproduced), were the ones that had nothing to do with types but with * bad understanding of the business rules * bad fundamental implementation of the problem domain * misuse / incorrect use of an API or libraries * API/library behaviour different from what the documentation says. No type system will save you from them. Type systems don't verify how the code does the correct thing. They only verify that the types moved between functions/modules satisfy certain conditions. Big deal. Where code reusability implies that your functions should apply to the widest type of circumstances (and thus increasing the scope in which they can be reused), type systems go fundamentally against to this goal, enforcing your functions to go very specific on what they're able to accept and return. Dynamically typed systems allow for faster coding, and also allow for interactive development which allows for faster, easier testing of individual components as they are developed. Also, the very best dynamically typed systems also allow for redefining / uploading updated function definitions while the code is running, which greatly increases development & testing speed (emphasis added.) This is not just a claim, this is my personal experience after about 23 years of programming where 90% of those years were spent using statically typed systems. As for speed, for highly optimized code Java, Haskell, and F# are basically in the same ballpark. Which is, very good speed. Well, Common Lisp (dynamically typed language) is not only in the same ballpark, but faster than them in some cases. You know what i'm going to choose. Or, different take on this: If i was going for absolute speed i'd be hacking in C, where no Hindley-Milner type checker gets in the way of clever tricks for gaining that extra bit of performance you need on that tight loop.
- golergka 9y ago
  > In my many years of experience doing software professionally, the serious bugs... > This is not just a claim, this is my personal experience after about 23 years of programming where 90% of those years were spent using statically typed systems. If most of your experience was with statically typed systems, it's just logical you would have very little experience with bugs that are prevented by statically typed systems - because this prevention would be at work. Looks like an opposite of survivor's fallacy, really.

virgilp 9y ago

> Anybody who’s seen the systems inside a major tech company knows this is true. Or a minor tech company. Or the insides of any product with a software component. Anybody who's seen the sun rising east and setting west knows that this is true: the Sun rotates around the Earth. The other, less-obvious alternative, is that software is NOT crap, we just like to complain a lot. And it's just so easy & fun to blame everything on the software we work with. Most software that doesn't immediately die does its job fairly well. In fact, better than whatever alternative that existed before said software.

sharemywin 9y ago
I think the point of the article is something that does an 95% job for 10% the cost works for a lot of things, but somethings that trade off doesn't work.
- virgilp 9y ago
  Maybe. And I could get behind a viewpoint that says, "some things are so dangerous, that a 99.999% job is not good enough". That's definitely true. But that is much less of a rant, and it requires the writer to do the hard work of defining those dangerous things, and arguing the benefits of adding the 4th, 5th, 6th nine against the cost. Saying "everything is crap" makes me just not take the article seriously, because the author only did half of the thinking job (not even half, maybe).
AnIdiotOnTheNet 9y ago
If our metric for "good" is based solely on what people choose to use, then one is forced to admit that we also have generally good security and privacy safeguards. Actually, I kinda agree with that assessment. "Good enough" is never good enough for academics, but practical reality is different.
- virgilp 9y ago
  > then one is forced to admit that we also have generally good security and privacy safeguards. It's a tremendous conceit of the software community to actually believe these are purely-software problems that can be solved through software. Sure, software goes a long way, but I argue that it's a fool's errand to hope they can or should be solved in software, exclusively. Like - hardware will always be a factor, at least that should be obvious. But also culture/ society/ laws/ etc. have important practical implications. The very fact that we have lots of people scrutinizing these aspects suggests that the state of software security&privacy is really not "crap" (far from perfect, but definitely not "crap"). To you believe that article to be true, one must have a fairly black&white view of the world. 99% (probably much more) of the humans can't really bypass those "crap" security safeguards.... it's good that we have high standards (especially in areas like this), but come on. When in the human history was security(in general) better? What are we comparing it with, to postulate it's "crap"?

Annatar 9y ago

“The reason this stuff is crap is far more basic. It’s because better-than-crap costs a lot more, and crap is usually sufficient. ... ... ... Every dollar put into making software less crappy can’t be spent on other things we might also want, the list of which is basically endless.“ Software is crap because humans on the whole have an extremely difficult time reasoning through all the possible logic flows. Software is also crap because in order to design good software a lot of insight, experience and empathy is required, and as it turns out, people writing crap don’t have any of those prerequisites. Finally, most software is crap because most people writing it don’t care about getting it 100% correct, since it turns out that getting the last 5% correctly functioning is exponentially difficult, and those people would rather collect a paycheck than feel proud and content about what they wrote. All other industries manage to deliver complete, working products; we are the only industry which doesn’t; computers and software never work 100% correctly. What does that make us? It makes us jerk-offs, that’s what it makes us. I understand now why Keith Wesolowski ditched computers and went to work on a ranch. And damn it, he is right!

mgaruccio 9y ago
Do all other Industries manage it though? If they did we wouldn't have product recalls or any need for "consumer safety watchdogs". The problem is arguably worse in software but software can also be (relatively) easily patched later so a somewhat more lax attitude to quality on systems that are not safety critical makes some degree of sense.
naasking 9y ago
> Software is crap because humans on the whole have an extremely difficult time reasoning through all the possible logic flows. But we have tools and techniques that make this easier. Why don't we use them? Also, I don't think 100% correctness is the ultimate goal of this article. Software is probably nowhere even near that correctness level firstly, and secondly, software is also partly crap because it doesn't run any faster than it did 30 years ago despite tremendous performance advances in our hardware. > What does that make us? It makes us jerk-offs, that’s what it makes us. Let's be honest, software as it currently is written is considerably more complex than most other engineered systems that we take for granted. That said, we've had tools and techniques to mitigate a lot of this complexity, and to have computers double-check our work (eg. type systems and model checking), but we don't really use them as much as we should. What's worse, we even use tools that are intentionally intractable to any kind of work checking/validation (like dynamically typed languages). There's really no excuse for this.

zzzcpan 9y ago

More precise understanding of faults and how to deal with them can go a long way before suggesting formal verification. Bug is not a fault, but it may cause faults though. You can try to find this bug beforehand or you can deal with the faults at runtime. In practice not just bugs cause faults, but also hardware failures, natural disasters, human mistakes and so on, so you kind of have to deal with faults either way. > But we can certainly arrange to radically limit the scope of damage available to any particular piece of crap, which should vastly reduce systemic crappiness. Now, this is the big idea behind supervision trees. Where you split everything into the smallest possible isolated processes and supervisors that watch over them, so that in the event of any process failing it can just get restarted, limiting the scope of the problem to only that one tiny process for the shortest possible time. This idea might even reduce the cost of software development compared to some more popular software development practices. But it does require an easy to use actor model in the language.

aabbccddq 9y ago

Like anything else, you just don't see it. I thought you have a point until you mentioned Microsoft. The company that showed us it's perfectly fine and socially accepted to make top money from crap and not releasing security updates, and be proud of it. Thanks MS!

nickthemagicman 9y ago

This is cultural. The Walmart-ification of the worlds products. Planned Obsolescence. The short term profits to get the stock prices up by the end of the quarter mentality. The credit card mindset. How fast society moves, and how cutthroat capitalism is becoming...its all about short term gain because the future feels chaotic and unstable to most of the world. I personally think its the fact the money is getting more an more difficult to obtain because more and more is being hoarded in the upper strata of society. So short term mindsets on financial gain take over because financial opportunities don't come easily. This is reflected in software development.

tenpoundhammer 9y ago

>And still, this wouldn’t be so bad, if the crap wasn’t starting to seep into things that actually matter. >A leading indicator of what’s to come is the state of computer security. We’re seeing an alarming rise in big security breaches, each more epic than the next, to the point where they’re hardly news any more. Target has 70 million customers’ credit card and identity information stolen. Oh no! Security clearance and personal data for 21.5 million federal employees is taken from the Office of Personnel Management. ... I’m not a fan of this type of article, the article where we find a group of bad things and then go find something to blame for it. It’s easy to say all this stuff was caused because “we just don’t make stuff like we used to”, but I think it’s wrong-headed. Or more probably it’s used in the wrong sense. It’s true that we don’t make things like we used to. We have made giant leaps past anything we’ve ever done before. We have hundreds of thousands of companies online and millions of devices online, a level of interconnectedness that the planet has never seen before. We also have more accessible software development tools and many more people writing software than ever before. Increasing numbers of security breaches are a sign of our progress not a sign of our failures. It’s just a sign that there is a lot more stuff attached to the internet and that unfortunately includes people with the intention to commit crimes. Should we tolerate security breaches? No. Should we celebrate them? Of course not. But let’s see them for what they are and search for a solution instead of something to vilify.

abecedarius 9y ago
I didn’t get the impression this post meant to say we just don’t make stuff like we used to, but that the crap we’ve been taking as normal is going to have terrible consequences if we don’t mitigate it somehow soon. I agree that common practices are gradually getting better, but I don’t think that’s enough. Re: solutions, the author wrote “What are capabilities?” which got discussed here the other day.

program_whiz 9y ago

Well we would expect no different from medicine, or home construction, or automobiles without some basic guarantees: 1. High visibility of problems (people die, building collapse, crashes) 2. Regulation and inspection (some standards for evaluation and lines that should not be crossed) 3. Certification (you wouldn't let an uncertified random person do heart surgery on you, or build your house). But for this to be possible, we as a whole would have to agree these are priorities and impose these limits. We're still in the time of the "surgery without anasthesia" or "no building codes" or "just let the factories emit whatever they want". Understandable due to the short time that software has been really "important" in day-to-day life, but now its time to slow down and make sure we do it right.