9 ms·

I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. Software dev is generally a s

I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. That's an unsatisfying answer, but if we try to hide from the implications just because we don't like it, we'll end up creating companies that fail to startups without these constraints. It's also self-evident that in most situations, correctness and reliability aren't a concern. The counterexamples account for maybe 1% of the field. 99% of the time, if your program breaks, you can pay someone to go fix it and no serious harm was done. Even Github outages, which affect almost all of us, hardly matter.

jolux 9y ago
>I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. I don't think this is self-evident whatsoever. One of the problems in software engineering is that we never know when a solution is "right," we only know when it's vaguely not-wrong, and even then almost everything we create is still subtly wrong in a way we forgot to think about. Math is solid because there is certainty in the correctness of certain proofs, and these can be used as building blocks for further results. Philosophically speaking this is the closest we're going to get to "real" engineering practice in software, and we're pretty dang far from it if you know what that looks like. >Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. But this is a symptom of any pre-scientific field in engineering, it's not specifically endemic to software engineering. The main difference is that in software engineering we're applying math directly, and in other engineering fields we're applying science. In a certain sense, the natural sciences are purely empirical (modulo advanced physics of course, which overlaps with pure math and philosophy these days) while math and consequently computer science are purely rational. So, what are the solid elements in math? Proofs. Proofs are also how you ensure that the field actually moves forward and you're not stuck recreating solid results that already exist. Sound familiar? It happens constantly in software, but imperative and poorly-typed languages inhibit composability and reuse because they lend themselves easily to extremely specific solutions to general problems. >It's also self-evident that in most situations, correctness and reliability aren't a concern. The counterexamples account for maybe 1% of the field. 99% of the time, if your program breaks, you can pay someone to go fix it and no serious harm was done. Even Github outages, which affect almost all of us, hardly matter. This is preposterous. Just because lives are not in danger does not mean that it's not worth doing right by the solution, and frankly I think a lot more lives are waiting to be put in danger by crappy software with bad security than you realize. It's already become almost dogma that all software engineers need to have a deep understanding of security, and I don't understand why correctness can't fall under that rubric as well. If we're going to be having Geohot or even Google or whoever building self-driving cars for possibly billions of people, enormous critical infrastructure projects coming under computer control, etc, the whole industry from education on up is going to have to have a serious attitude adjustment to keep up with the demands of safety, security, and reliability. It might not matter 99% of the time, (I think it's a lot more than that of course) but we need to make sure that we can as a discipline deliver that 1% when it is absolutely critical, and as of right now it doesn't seem like we can.
- sillysaurus3 9y ago
  It might not matter 99% of the time, (I think it's a lot more than that of course) but we need to make sure that we can as a discipline deliver that 1% when it is absolutely critical, and as of right now it doesn't seem like we can. FWIW, we agree on this point. But it seems worth treating this 1% case as a separate discipline rather than trying to lump it together with software dev. No one would claim that NASA's software engineering is the same type of work as, say, writing a new HN feature. It's already become almost dogma that all software engineers need to have a deep understanding of security The most secure programs are those that undergo frequent penetration tests and have bug bounty programs. Speaking as a pentester, I think there's not much chance of regular software devs being any good at security. There's just too much to know. I want to agree with your other points, because in principle it's the correct thing. Unfortunately experience has taught us that the correct thing usually loses in the real world. Being first to market mattered way more for Ethereum than the fact that the DAO had a bug in their smart contract, for example. But there are hundreds or even thousands of examples of this type. If we pretend that a teenager hacking in their bedroom is doing something fundamentally different than what most developers do each day at their jobs, then we lose out on the ability of that teenager to innovate. We become an exclusionary clique rather than an inclusive group. Luckily market forces still prevent us from becoming that insular, but in the era of walled gardens it's easy to imagine we're not too far off from that fate. The main issue is that if we try to restrict the free market e.g. with legislation, then the important work will simply move overseas to areas without those restrictions. And unless you're proposing legal restrictions on the software dev trade, it's unclear how to enforce any of the proposals upthread.
  - jolux 9y ago
    >No one would claim that NASA's software engineering is the same type of work as, say, writing a new HN feature. Well no, but the same underlying principles are still operating. The only difference is how much you care about heeding them. You don't need an aerospace engineering degree to make a paper airplane or a short bridge or a raft or whatever, but that doesn't mean that your knowledge of how to do so reliably and correctly wouldn't be improved by such a degree, or that if you're going to sell a product that you hope people will pay you for and subsequently depend on that you shouldn't bother trying to make it functional (in the "it works" sense) to the best of your abilities. >If we pretend that a teenager hacking in their bedroom is doing something fundamentally different than what most developers do each day at their jobs, then we lose out on the ability of that teenager to innovate. Ok, that teenager can hack, sure, just the same as they can build a two-stroke engine or an electronic alarm for their door or play around with nuclear fusion or whatever. But if they're going to sell those things and make claims about their safety and reliability, the validity of those claims should be enforced by an industry guild accreditation program or legal regulations or whatever. There's more than one way to skin this cat, but it really needs skinning. >The main issue is that if we try to restrict the free market e.g. with legislation, then the important work will simply move overseas to areas without those restrictions. The goalposts are being moved here, though, near as I can tell. I told you what I wanted done and why I thought it would work, and now you're telling me I have to figure out how to do it specifically in such a way that it can't be circumvented. I haven't thought up a specific solution to this question you pose, and as such I would point you in the direction of how liability works in other disciplines for similar cases and so on. I do actually think a lot of the same systems could work for enforcement, the biggest difficulty is actually figuring out what the principles should be. If the biggest problem with enforcement is that teenage hackers can't innovate anymore, I'm not really all that concerned. Romanticizing that image does nothing to change the hard realities of the industry.
nautilus12 9y ago
I havent thought of this before but i was a mathematician that does software engineering, and Im starting to see some stark similarities between a mathematical proof and a "test" as in a unit test. I imagine if you went down this road you would end up with something very similar if not identical to TDD. What i was hoping this thread would discuss is how to turn software design patterns like the adaptor or factory patterns into a science in the same way that differential topology turns differential equations from a haphazard collection of methods into a rigorous science. I dont know why more people havent focused on formulating this more practical side of software engineering. Why is it that SOLID works so well, etc
- Denzel 9y ago
  Yes, this is of extreme interest to me as well. The why behind SOLID. I think you could formulate more rigorous support based upon information and graph theory, maybe neuroscience, and other areas I'm less familiar with. For example, the Single Responsibility Principle (SRP) is primarily concerned with making software more manageable on both an individual and team basis. How? By minimizing: 1. Communication overhead between teams/modules 2. Information overload in an individual You could look at #1 from a graph-theoretic and information basis. I'm sure there are many interesting things to prove there. Like Amdahl's and the opposite of Metcalfe's Law [1][2] applied to team/communication/module dependencies. Just as a basic example, if one class has 10 responsibilities shared by 10 engineers with no boundaries specified, then the probability of conflict and unintended consequences rises. Thus the rate of development slows. As for #2, applying more rigor and proof to the question of how these principles help a human understand quicker. I don't have the necessary background right now to explore this in more detail, but like you, I'm very interested in any possible formulations. [1]: https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law [2]: https://en.wikipedia.org/wiki/Metcalfe%27s_law https://en.wikipedia.org/wiki/Metcalfe%27s_law
  - hobls 9y ago
    The problem is that even things like "how many people touch this class" and "how many external teams do we work with" aren't simple at all. Some external teams are much easier to work with than others. Sometimes you want your whole team to intimately understand an area of your code. Making software is an intensely human process, which is why managing software teams is almost entirely about considering the people on your team, and what makes each of them different.
Denzel 9y ago
> scattershot haphazard endeavor that involves trying dozens of angles until one of them works Sounds a lot like engineering/design/art. There's a large amount of overlap between the three. I had the pleasure of speaking to the principal electrical engineer whose firm was responsible for designing the entire electrical system for One World Trade Center in Manhattan and the Citi field as well. He described his work a lot like a software developer would describe their own. And then, in his words, when his company was done they "passed the plans over to the electricians to build it". It just so happens, in our profession, we've automated the part that electricians are responsible for in his project. Our electricians are called compilers. They dutifully carry out the plan and from time to time they surface warnings/errors back to the engineer for input/correction. People conflate the meaning of software development because one person shares so many different responsibilities, that are usually handled by separate people in other professions: architecture, design, analysis, implementation, maintenance, etc. What a civil/electrical/mechanical engineer does isn't so different from what a software engineer does. It's a "scattershot haphazard endeavor that involves trying dozens of angles [based upon guiding principles] until one of them works" usually run through loads of simulation and analysis. Now, the difference is, when these engineers want to bring their idea into the world it takes physical labor, unlike software development where the feedback loop is near instantaneous. We bring our programs into the world with compilers and can run them immediately. I mean, there are plenty of examples where civil/electrical/mechanical engineers failed in their design and thus created a bug in their project. See [1] or any contemporary CPU from AMD/Intel or automobile recalls or spacecraft failures, etc. There are certain principles of software engineering that lead to more effective software. That's a fact. And the companies that understand this fact will not fail to startups, in fact they exhibit a severe technological advantage. Google and Facebook are two companies that understand software engineering. Talk to the YouTube/Instagram teams and see whether they felt better off -- technologically -- before or after acquisition. [1]: https://en.wikipedia.org/wiki/Citigroup_Center#Engineering_crisis_of_1978 https://en.wikipedia.org/wiki/Citigroup_Center#Engineering_c...
- mdpopescu 9y ago
  Agreed. Reading [1] was a fantastic eye-opener for me. The code is the design, and design is always "trying until it works". [1] http://www.developerdotstar.com/mag/articles/reeves_design.html http://www.developerdotstar.com/mag/articles/reeves_design.h...
mannykannot 9y ago
> Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. So software engineering isn't engineering, either? I would imagine any civil engineer who actually knows what it takes to build a bridge would tend to agree, at least on the basis of the points you have put forward so far.
- chii 9y ago
  if you look at civil engineering at it's childhood stages that is, back when pre-historic man tried to build huts and lean-tos, you'd find that they too, didn't use any solid engineering practises, but simply tried different things and found out what worked. Being constrained physically makes it easier to determine what works, but the essense isn't any different to what software engineering is today. I bet if we fast forward 1000 years, you'd find that software engineering will have become as rigorous as what we have today in civil engineering.
  - concede_pluto 9y ago
    This. Someday software developers will look back on us the way doctors look back on barbers' bloodletting.
    - seanmcdirmid 9y ago
      Now that we brought out the doctor analogy, I have news for you....doctors deal with an unknown system, gain personal and institutional experience through trial and error, are basically detectives, are rarely working with full information, and have, despite their heroic efforts, a significant task failure rate (the patient dies). How is that not like software engineering?
      - chii 9y ago
        i dont think doctors fail as often as software projects. and they only fail on the fringes of medicine - like treating cancer, or rare diseases. You don't often see doctors failing to treat the cold or a broken angle badly that they kill their patients! But in the software world, it seems the equivalent of the cold is a CRUD app, but still fails so often that it's newsworthily talked about when such a software project succeed!
xigency 9y ago
Not really familiar with RTOS/embedded/security/military/medical software then, are we. If those examples are 1% of software, they take disproportionately more funding than 1%. And I'm sure customers of software, from personal computers, to business apps, to corporate websites would appreciate more consistent results as well. Reckless software engineering has damaged the reputation of the field as a whole.
Turing_Machine 9y ago
>I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. Yes. Writing software is not math. Does it use math concepts? Of course, but so does just about everything else. While we're at it, writing software is also not engineering, even though there are engineering concepts that can be applied. Writing software is its own thing. Pretending that it's something else (like math or engineering) invariably leads to category errors. Think about food recipes. They certainly use measurements and timing, and engineering principles can certainly be applied (especially when executing them on an industrial scale) but they are not math (or engineering) either. Examining the measurements, timing, and the production chain doesn't tell you anything about whether the recipe is delicious or inedible. Arguing that a piece of software should be "proven correct" makes about as much sense as arguing that a recipe should be "proven correct". You might as well judge the recipe by the standards of poetry ("Does it have evocative imagery? Does it rhyme or alliterate well?"). People have been chasing the unicorn of software correctness proofs for 60 years, with a notable lack of generalizable success (there are plenty of toy examples, of course). What usually happens is that the "programming is math" people come up with some bizarre academic language that no real-world programmer would use unless forced to do so at gunpoint (followed by the new language sinking without a trace). Alternatively, the "programming is engineering" people come up with some baroque formal process that requires you to write a 500 page document and get six committees to sign off on it before you can write "Hello, world". I'm old enough that I've seen these things happen multiple times. >It's also self-evident that in most situations, correctness and reliability aren't a concern. I wouldn't say they aren't a concern at all, but if you're wasting six months screwing around with formal proofs, UML diagrams, or things of that nature, while in the meantime your competitor is iterating three or four times, that is definitely a concern. Operate that way and your milkshake is going to be drunk, yo.
- jolux 9y ago
  >Writing software is not math. You're wrong, and I've explained this upthread. >Think about food recipes. Do you have experience with non-imperative programming paradigms? I'm sorry to say that the comparison to recipes in this context seems fairly naive. >People have been chasing the unicorn of software correctness proofs for 60 years, with a notable lack of generalizable success (there are plenty of toy examples, of course). Static type systems are arguably a product of this, especially advanced ones like Haskell's. >What usually happens is that the "programming is math" people come up with some bizarre academic language that no real-world programmer would use unless forced to do so at gunpoint (followed by the new language sinking without a trace). Now you're just being anti-intellectual. The whole point is that the real-world programmers are stumbling into all this crap constantly without realizing it. It's completely fucking unavoidable and very much tied up with the fact that programming is math. The only question is what you choose to do about that: learn the math, or stay ignorant. I don't mean to be condescending but I find comparing UML diagrams and proof assistants a little offensive (among other things), and it suggests you don't know what you're talking about. Modern proof assistants and other formal techniques like dependent typing happen while you're programming (Idris/Agda) or generate programs themselves (Coq), they aren't some sort of Waterfall-ish thing where you have to deal with all the ceremony before you start to get shit done. On the contrary, you get shit done, and it works better when you're done with it too.
mhandb 9y ago
> Software dev ... involves trying dozen angles until one of them works It is a way to solve mathematical problems, See "Guess and Check" from this book https://en.m.wikipedia.org/wiki/How_to_Solve_It https://en.m.wikipedia.org/wiki/How_to_Solve_It
lmm 9y ago
> I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. But the actual practice of doing mathematics (i.e. coming up with novel proofs) works exactly like this as well. The differences are in how much confidence we have in the results, and how much we're able to reuse them. > It's also self-evident that in most situations, correctness and reliability aren't a concern. The counterexamples account for maybe 1% of the field. 99% of the time, if your program breaks, you can pay someone to go fix it and no serious harm was done. Even Github outages, which affect almost all of us, hardly matter. I agree that we collectively don't currently care as much about correctness as we pretend we do. I believe software correctness is becoming more important (e.g. the rise of ransomware) and is going to become much much more important, but that's due to what I accept is a non-mainstream view of the future.
skgoa 9y ago
> Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. Maybe in an agile web startup. SW engineering in a big industry project is a very deliberate process. You start by a requirements engineer writing very precise requirements. Then a SW engineer turns this into a module design and a test specification. These are turned into source code (for which we use code generators to an ever higher degree) and test cases by yet other SW engineers. In the end everything gets reviewed and/or tested. Trying things until they work is just not done. That would be way too dangerous for SW that controls cars, airplanes, nuclear power plants, rockets etc.
- noir_lord 9y ago
  Safety critical software is a small subset of the software that gets written in "big industry" projects. There are hugely more systems not written to that standard.
  - skgoa 9y ago
    Yes, but those systems are engineered in the exact same processes. Just with less testing and fewer documents.

jolux 9y ago

>I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. I don't think this is self-evident whatsoever. One of the problems in software engineering is that we never know when a solution is "right," we only know when it's vaguely not-wrong, and even then almost everything we create is still subtly wrong in a way we forgot to think about. Math is solid because there is certainty in the correctness of certain proofs, and these can be used as building blocks for further results. Philosophically speaking this is the closest we're going to get to "real" engineering practice in software, and we're pretty dang far from it if you know what that looks like. >Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. But this is a symptom of any pre-scientific field in engineering, it's not specifically endemic to software engineering. The main difference is that in software engineering we're applying math directly, and in other engineering fields we're applying science. In a certain sense, the natural sciences are purely empirical (modulo advanced physics of course, which overlaps with pure math and philosophy these days) while math and consequently computer science are purely rational. So, what are the solid elements in math? Proofs. Proofs are also how you ensure that the field actually moves forward and you're not stuck recreating solid results that already exist. Sound familiar? It happens constantly in software, but imperative and poorly-typed languages inhibit composability and reuse because they lend themselves easily to extremely specific solutions to general problems. >It's also self-evident that in most situations, correctness and reliability aren't a concern. The counterexamples account for maybe 1% of the field. 99% of the time, if your program breaks, you can pay someone to go fix it and no serious harm was done. Even Github outages, which affect almost all of us, hardly matter. This is preposterous. Just because lives are not in danger does not mean that it's not worth doing right by the solution, and frankly I think a lot more lives are waiting to be put in danger by crappy software with bad security than you realize. It's already become almost dogma that all software engineers need to have a deep understanding of security, and I don't understand why correctness can't fall under that rubric as well. If we're going to be having Geohot or even Google or whoever building self-driving cars for possibly billions of people, enormous critical infrastructure projects coming under computer control, etc, the whole industry from education on up is going to have to have a serious attitude adjustment to keep up with the demands of safety, security, and reliability. It might not matter 99% of the time, (I think it's a lot more than that of course) but we need to make sure that we can as a discipline deliver that 1% when it is absolutely critical, and as of right now it doesn't seem like we can.

sillysaurus3 9y ago
It might not matter 99% of the time, (I think it's a lot more than that of course) but we need to make sure that we can as a discipline deliver that 1% when it is absolutely critical, and as of right now it doesn't seem like we can. FWIW, we agree on this point. But it seems worth treating this 1% case as a separate discipline rather than trying to lump it together with software dev. No one would claim that NASA's software engineering is the same type of work as, say, writing a new HN feature. It's already become almost dogma that all software engineers need to have a deep understanding of security The most secure programs are those that undergo frequent penetration tests and have bug bounty programs. Speaking as a pentester, I think there's not much chance of regular software devs being any good at security. There's just too much to know. I want to agree with your other points, because in principle it's the correct thing. Unfortunately experience has taught us that the correct thing usually loses in the real world. Being first to market mattered way more for Ethereum than the fact that the DAO had a bug in their smart contract, for example. But there are hundreds or even thousands of examples of this type. If we pretend that a teenager hacking in their bedroom is doing something fundamentally different than what most developers do each day at their jobs, then we lose out on the ability of that teenager to innovate. We become an exclusionary clique rather than an inclusive group. Luckily market forces still prevent us from becoming that insular, but in the era of walled gardens it's easy to imagine we're not too far off from that fate. The main issue is that if we try to restrict the free market e.g. with legislation, then the important work will simply move overseas to areas without those restrictions. And unless you're proposing legal restrictions on the software dev trade, it's unclear how to enforce any of the proposals upthread.
- jolux 9y ago
  >No one would claim that NASA's software engineering is the same type of work as, say, writing a new HN feature. Well no, but the same underlying principles are still operating. The only difference is how much you care about heeding them. You don't need an aerospace engineering degree to make a paper airplane or a short bridge or a raft or whatever, but that doesn't mean that your knowledge of how to do so reliably and correctly wouldn't be improved by such a degree, or that if you're going to sell a product that you hope people will pay you for and subsequently depend on that you shouldn't bother trying to make it functional (in the "it works" sense) to the best of your abilities. >If we pretend that a teenager hacking in their bedroom is doing something fundamentally different than what most developers do each day at their jobs, then we lose out on the ability of that teenager to innovate. Ok, that teenager can hack, sure, just the same as they can build a two-stroke engine or an electronic alarm for their door or play around with nuclear fusion or whatever. But if they're going to sell those things and make claims about their safety and reliability, the validity of those claims should be enforced by an industry guild accreditation program or legal regulations or whatever. There's more than one way to skin this cat, but it really needs skinning. >The main issue is that if we try to restrict the free market e.g. with legislation, then the important work will simply move overseas to areas without those restrictions. The goalposts are being moved here, though, near as I can tell. I told you what I wanted done and why I thought it would work, and now you're telling me I have to figure out how to do it specifically in such a way that it can't be circumvented. I haven't thought up a specific solution to this question you pose, and as such I would point you in the direction of how liability works in other disciplines for similar cases and so on. I do actually think a lot of the same systems could work for enforcement, the biggest difficulty is actually figuring out what the principles should be. If the biggest problem with enforcement is that teenage hackers can't innovate anymore, I'm not really all that concerned. Romanticizing that image does nothing to change the hard realities of the industry.

nautilus12 9y ago

I havent thought of this before but i was a mathematician that does software engineering, and Im starting to see some stark similarities between a mathematical proof and a "test" as in a unit test. I imagine if you went down this road you would end up with something very similar if not identical to TDD. What i was hoping this thread would discuss is how to turn software design patterns like the adaptor or factory patterns into a science in the same way that differential topology turns differential equations from a haphazard collection of methods into a rigorous science. I dont know why more people havent focused on formulating this more practical side of software engineering. Why is it that SOLID works so well, etc

Denzel 9y ago
Yes, this is of extreme interest to me as well. The why behind SOLID. I think you could formulate more rigorous support based upon information and graph theory, maybe neuroscience, and other areas I'm less familiar with. For example, the Single Responsibility Principle (SRP) is primarily concerned with making software more manageable on both an individual and team basis. How? By minimizing: 1. Communication overhead between teams/modules 2. Information overload in an individual You could look at #1 from a graph-theoretic and information basis. I'm sure there are many interesting things to prove there. Like Amdahl's and the opposite of Metcalfe's Law [1][2] applied to team/communication/module dependencies. Just as a basic example, if one class has 10 responsibilities shared by 10 engineers with no boundaries specified, then the probability of conflict and unintended consequences rises. Thus the rate of development slows. As for #2, applying more rigor and proof to the question of how these principles help a human understand quicker. I don't have the necessary background right now to explore this in more detail, but like you, I'm very interested in any possible formulations. [1]: https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law [2]: https://en.wikipedia.org/wiki/Metcalfe%27s_law https://en.wikipedia.org/wiki/Metcalfe%27s_law
- hobls 9y ago
  The problem is that even things like "how many people touch this class" and "how many external teams do we work with" aren't simple at all. Some external teams are much easier to work with than others. Sometimes you want your whole team to intimately understand an area of your code. Making software is an intensely human process, which is why managing software teams is almost entirely about considering the people on your team, and what makes each of them different.

Denzel 9y ago

> scattershot haphazard endeavor that involves trying dozens of angles until one of them works Sounds a lot like engineering/design/art. There's a large amount of overlap between the three. I had the pleasure of speaking to the principal electrical engineer whose firm was responsible for designing the entire electrical system for One World Trade Center in Manhattan and the Citi field as well. He described his work a lot like a software developer would describe their own. And then, in his words, when his company was done they "passed the plans over to the electricians to build it". It just so happens, in our profession, we've automated the part that electricians are responsible for in his project. Our electricians are called compilers. They dutifully carry out the plan and from time to time they surface warnings/errors back to the engineer for input/correction. People conflate the meaning of software development because one person shares so many different responsibilities, that are usually handled by separate people in other professions: architecture, design, analysis, implementation, maintenance, etc. What a civil/electrical/mechanical engineer does isn't so different from what a software engineer does. It's a "scattershot haphazard endeavor that involves trying dozens of angles [based upon guiding principles] until one of them works" usually run through loads of simulation and analysis. Now, the difference is, when these engineers want to bring their idea into the world it takes physical labor, unlike software development where the feedback loop is near instantaneous. We bring our programs into the world with compilers and can run them immediately. I mean, there are plenty of examples where civil/electrical/mechanical engineers failed in their design and thus created a bug in their project. See [1] or any contemporary CPU from AMD/Intel or automobile recalls or spacecraft failures, etc. There are certain principles of software engineering that lead to more effective software. That's a fact. And the companies that understand this fact will not fail to startups, in fact they exhibit a severe technological advantage. Google and Facebook are two companies that understand software engineering. Talk to the YouTube/Instagram teams and see whether they felt better off -- technologically -- before or after acquisition. [1]: https://en.wikipedia.org/wiki/Citigroup_Center#Engineering_crisis_of_1978 https://en.wikipedia.org/wiki/Citigroup_Center#Engineering_c...

mdpopescu 9y ago
Agreed. Reading [1] was a fantastic eye-opener for me. The code is the design, and design is always "trying until it works". [1] http://www.developerdotstar.com/mag/articles/reeves_design.html http://www.developerdotstar.com/mag/articles/reeves_design.h...

mannykannot 9y ago

> Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. So software engineering isn't engineering, either? I would imagine any civil engineer who actually knows what it takes to build a bridge would tend to agree, at least on the basis of the points you have put forward so far.

chii 9y ago
if you look at civil engineering at it's childhood stages that is, back when pre-historic man tried to build huts and lean-tos, you'd find that they too, didn't use any solid engineering practises, but simply tried different things and found out what worked. Being constrained physically makes it easier to determine what works, but the essense isn't any different to what software engineering is today. I bet if we fast forward 1000 years, you'd find that software engineering will have become as rigorous as what we have today in civil engineering.
- concede_pluto 9y ago
  This. Someday software developers will look back on us the way doctors look back on barbers' bloodletting.
  - seanmcdirmid 9y ago
    Now that we brought out the doctor analogy, I have news for you....doctors deal with an unknown system, gain personal and institutional experience through trial and error, are basically detectives, are rarely working with full information, and have, despite their heroic efforts, a significant task failure rate (the patient dies). How is that not like software engineering?
    - chii 9y ago
      i dont think doctors fail as often as software projects. and they only fail on the fringes of medicine - like treating cancer, or rare diseases. You don't often see doctors failing to treat the cold or a broken angle badly that they kill their patients! But in the software world, it seems the equivalent of the cold is a CRUD app, but still fails so often that it's newsworthily talked about when such a software project succeed!

xigency 9y ago

Not really familiar with RTOS/embedded/security/military/medical software then, are we. If those examples are 1% of software, they take disproportionately more funding than 1%. And I'm sure customers of software, from personal computers, to business apps, to corporate websites would appreciate more consistent results as well. Reckless software engineering has damaged the reputation of the field as a whole.

Turing_Machine 9y ago

>I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. Yes. Writing software is not math. Does it use math concepts? Of course, but so does just about everything else. While we're at it, writing software is also not engineering, even though there are engineering concepts that can be applied. Writing software is its own thing. Pretending that it's something else (like math or engineering) invariably leads to category errors. Think about food recipes. They certainly use measurements and timing, and engineering principles can certainly be applied (especially when executing them on an industrial scale) but they are not math (or engineering) either. Examining the measurements, timing, and the production chain doesn't tell you anything about whether the recipe is delicious or inedible. Arguing that a piece of software should be "proven correct" makes about as much sense as arguing that a recipe should be "proven correct". You might as well judge the recipe by the standards of poetry ("Does it have evocative imagery? Does it rhyme or alliterate well?"). People have been chasing the unicorn of software correctness proofs for 60 years, with a notable lack of generalizable success (there are plenty of toy examples, of course). What usually happens is that the "programming is math" people come up with some bizarre academic language that no real-world programmer would use unless forced to do so at gunpoint (followed by the new language sinking without a trace). Alternatively, the "programming is engineering" people come up with some baroque formal process that requires you to write a 500 page document and get six committees to sign off on it before you can write "Hello, world". I'm old enough that I've seen these things happen multiple times. >It's also self-evident that in most situations, correctness and reliability aren't a concern. I wouldn't say they aren't a concern at all, but if you're wasting six months screwing around with formal proofs, UML diagrams, or things of that nature, while in the meantime your competitor is iterating three or four times, that is definitely a concern. Operate that way and your milkshake is going to be drunk, yo.

jolux 9y ago
>Writing software is not math. You're wrong, and I've explained this upthread. >Think about food recipes. Do you have experience with non-imperative programming paradigms? I'm sorry to say that the comparison to recipes in this context seems fairly naive. >People have been chasing the unicorn of software correctness proofs for 60 years, with a notable lack of generalizable success (there are plenty of toy examples, of course). Static type systems are arguably a product of this, especially advanced ones like Haskell's. >What usually happens is that the "programming is math" people come up with some bizarre academic language that no real-world programmer would use unless forced to do so at gunpoint (followed by the new language sinking without a trace). Now you're just being anti-intellectual. The whole point is that the real-world programmers are stumbling into all this crap constantly without realizing it. It's completely fucking unavoidable and very much tied up with the fact that programming is math. The only question is what you choose to do about that: learn the math, or stay ignorant. I don't mean to be condescending but I find comparing UML diagrams and proof assistants a little offensive (among other things), and it suggests you don't know what you're talking about. Modern proof assistants and other formal techniques like dependent typing happen while you're programming (Idris/Agda) or generate programs themselves (Coq), they aren't some sort of Waterfall-ish thing where you have to deal with all the ceremony before you start to get shit done. On the contrary, you get shit done, and it works better when you're done with it too.

mhandb 9y ago

> Software dev ... involves trying dozen angles until one of them works It is a way to solve mathematical problems, See "Guess and Check" from this book https://en.m.wikipedia.org/wiki/How_to_Solve_It https://en.m.wikipedia.org/wiki/How_to_Solve_It

lmm 9y ago

> I think it's self-evident that there is no theoretical basis for software development in the way that would satisfy your criteria. Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. But the actual practice of doing mathematics (i.e. coming up with novel proofs) works exactly like this as well. The differences are in how much confidence we have in the results, and how much we're able to reuse them. > It's also self-evident that in most situations, correctness and reliability aren't a concern. The counterexamples account for maybe 1% of the field. 99% of the time, if your program breaks, you can pay someone to go fix it and no serious harm was done. Even Github outages, which affect almost all of us, hardly matter. I agree that we collectively don't currently care as much about correctness as we pretend we do. I believe software correctness is becoming more important (e.g. the rise of ransomware) and is going to become much much more important, but that's due to what I accept is a non-mainstream view of the future.

skgoa 9y ago

> Software dev is generally a scattershot haphazard endeavor that involves trying dozens of angles until one of them works. Maybe in an agile web startup. SW engineering in a big industry project is a very deliberate process. You start by a requirements engineer writing very precise requirements. Then a SW engineer turns this into a module design and a test specification. These are turned into source code (for which we use code generators to an ever higher degree) and test cases by yet other SW engineers. In the end everything gets reviewed and/or tested. Trying things until they work is just not done. That would be way too dangerous for SW that controls cars, airplanes, nuclear power plants, rockets etc.

noir_lord 9y ago
Safety critical software is a small subset of the software that gets written in "big industry" projects. There are hugely more systems not written to that standard.
- skgoa 9y ago
  Yes, but those systems are engineered in the exact same processes. Just with less testing and fewer documents.