4 ms·
I think you're mainly looking at the failures that happened for mostly political or management reasons rather than where real attempts were made. Software engin
by nickpsecurity 11y ago
I think you're mainly looking at the failures that happened for mostly political or management reasons rather than where real attempts were made. Software engineering research gave us great stuff whose quality was largely unmatched by "programming, motherfucker" types who just threw shit together and hacked on top of it. Programmers making high quality stuff are rare. So much OpenBSD crowd differentiates based on how they apply a few proven methods to boost their quality. So, let's look at some that succeeded to see if software engineering had any value.
First was probably Dijkstra's THE operating system in 1968. Careful use of abstraction, interface checks, systematic testing of individual components, and careful composition led to a system that performed nearly flawlessly. All high assurance systems used a similar model in safety- and security-oriented systems with near flawless results despite much analysis, pentesting, field use, etc. Still same concepts used to this day same results. QED already. However, high assurance is just too limiting, slow-moving, and heavyweight in skill set. Best to focus it on premium components (eg TCB, key protocols). So, let's keep going to see what software engineering might have given average programmers.
By early 70's, Fagan at IBM implemented Inspection Process that formalized software review to try to counter all these bugs they caught. That it was systematic, regular, measured, and adaptive meant it caught tons of errors in the studies. Code reviews have only become popular among "programming, motherfucker" types in the recent decade with companies still trying to lure more in with HN articles despite it being empirically proven back in the 70's. Software engineering R&D also developed tons of tools to help but most programmers don't use them outside maybe testing or build systems.
Note: Around this time period, Wirth's languages and work built on them provide strong type safety, memory safety, interface checks, and one was immune to concurrency errors w/ language + architecture choices. Only a small niche adopted them and programmers were mostly not using them. All the buffer overflows, race conditions, etc followed logically.
By the 80's, Harlin Mills tried to stick it to hardware people by inventing the first, true engineering practice for software: Cleanroom methodology. Combined a formal notation, structured programming, decomposition, safe subset of language, by-eye verification conditions, usage-based testing, and statistical certification of quality. IBM tested it to find their defect rate was at shockingly low levels on first run with it improving from that point on. CASE tools were developed to semi-automate it. Small businesses differentiating on quality issued warranties (WTF!?) for their software at specific defect rates. Got mentioned on Dr Dobbs with a chance to reach programmers but "professional programmers" don't engineer software regardless of evidence. Cleanroom faded, typical programming continued, and basic defects are all over the place in software which is barely comprehensible.
From the 80's to now, research continued on building tools to automate as much as possible. Some prevented issues, some detected them during development, some caught them at runtime, some just recovered. In some situations, the whole problem could be solved via a synthesis and with reliable results. Programmers totally ignored this line of software engineering research. Unfamiliarity with languages like Wirth's, Ada, ML's, and Eiffel meant that would happen to a degree. However, programmers continued to ignore results even when applied to C, Java, etc that they used. A niche market of companies recognized the value and started applying such tools. CPU vendors, in EDA market, poured tons of money into it due to the cost of defects and got a series of incredible results going to this day. It's why hardware has a tiny defect rate despite tens of millions to ten billion logic functions running concurrenty.
Now, in recent events, some lessons of software engineering research and innovations have ended up in some mainstream stuff. Go, Rust, Scala, and Clojure come to mind immediately. The articles on them are pushing the stuff software engineers, especially in academia, had been pushing all along. Imagine how much better our code & tools would be had programmers listened to software engineers idk maybe 40 years ago.
Meanwhile, there's still companies and R&D leading the way. Altran/Praxis's Correct by Construction process has been in the lead with mathematical specs of behavior, proper error handling, SPARK implementations proven free of common bugs, and field results showing among lowest defect rate in existence. Galois is applying software engineering to low-level stuff combining Haskell, DSL's like Ivory, and C with good results. The Design-by-Contract crowd has been catching errors others missed. DO-178B scene is applying every tech and trick they can to reduce defect count. Embedded scene, like Esterel's SCADE, is generating whole apps from specs that work the first time. One academic combined Cleanroom and Python to get an amazing productivity-to-defect ratio. And so on and so forth with every independent evaluation and case study showing these methods deliver results "programming, motherfucker" types still believe don't exist. And predictably. And repeatedly. And even mathematically if you want. Kind of like engineering.
So, software engineering exists. The term got abused by the wrong types of people pushing the wrong stuff. The good stuff got ignored by them and most programmers. However, there's a string of tools, papers, companies, and so on applying that stuff. Their results are better than programmers almost every time sometimes with less work. Programmers are even adopting some of their stuff after decades of mocking or ignoring it. Those programmers own stuff is getting better. So, software engineering is a thing, it's contributions to our capabilities are enormous, and we'd all benefit if programmers used more of it.
My slogan: "Engineering, motherfuckers, do you speak it!?"
"No? We'll, that explains the results..."
- kragen 11y agoYou don't get to "programming, motherfucker" by "throwing shit together and hacking on top of it"; that's what happens when you don't know how to program, maybe because you wasted your time studying UML and bug triage instead of learning how to design stuff. You don't get high-assurance systems by improving your metrics collection and reducing the standard deviation of your effort estimates; you get them with, as you say, careful code review (by people who know how to program) as practiced by "programming, motherfucker" types like Gosper and Beck and Fowler for decades, effective abstraction, and code that is correct by construction and formal proof rather than merely by testing — which you also need. Dijkstra spilled gallons of ink in the 1970s and 80s railing against the hucksters who appropriated the "software engineering" term, trying to redirect people's attention back towards writing software that could be logically proven correct. It's taken decades of effort to start to realize his vision (and Hoare's, et al.), but that research isn't happening in "software engineering"; it's happening in computer science. Hardware has been able to take advantage of formal methods for much longer than software, because hardware has never been more than a thousandth as complex as software since at least the 1950s. Cleanroom (btw, you misspelled "Harlan" as "Harlin"; the poor guy deserves better) is great as far as it goes, but it doesn't focus on proper abstraction design, and its testing methodology basically assumes you're programming a finite state machine. But if you're doing that, there are much better ways to verify it, including rigorous formal verification. Formal verification comes from logicians and computer scientists, not the kind of "software engineers" who write System Definition Documents. Golang is from the Bell Labs tema that gave us Unix. Rust is an application of affine typing, a variant of Girard's linear typing. Clojure is an application of research into FP-persistent data structures, using the syntax of Lisp, the classic hacker's language. These are not the kinds of things the original article is directing your attention to. It's not talking about affine type systems and Hindley-Milner type inference. It's talking about management research masquerading as research on how to engineer software. And that's a fraud.
- nickpsecurity 11y ago"These are not the kinds of things the original article is directing your attention to. It's not talking about affine type systems and Hindley-Milner type inference. It's talking about management research masquerading as research on how to engineer software. And that's a fraud." I totally agree with you there. I think a lot of the dispute in my post and yours centers on the definition of software engineering. You're applying it as the approach strictly about process, paperwork, drowning in numbers, etc with nothing produced. I'm applying it to mean any software development activities that apply engineering principles to software with engineering-like results. Computer Science is where a lot of what I'm citing came from, although CompSci != engineering software in general case. So, I have to have another term and "software engineer" literally means a person that engineers software. Think I need a solution to this problem as it's recurring. I've clearly illustrated most programmers, even good ones, don't apply engineering fully and leverage what's proven. It's always a subset, if at all, often with informal approach driving the whole thing. There are groups doing the whole thing like engineering but they're rare & not paperwork pushers. If the phrase is irreparably tarnished, I might need to ditch "software engineer" for a phrase that describes the difference between how a group like Praxis does things and how a decent programming shop does it. The difference is how systematic it's designed, assured, and documented with every available, proven method put to use plus careful tradeoffs. Need a new terms I'm thinking because old one caused a huge chunk of your counter-points which apply to the crowd we both can't stand but not really the niche I'm promoting. "Hardware has been able to take advantage of formal methods for much longer than software, because hardware has never been more than a thousandth as complex as software since at least the 1950s." Partly and often developers are reason. Getting from hand-wired hardware to synthesis, formal methods, simulation, and testing took hundreds of millions in investment with many bright minds. The reason it worked was they noticed common patterns in their designs/implementations, started reusing them in new ones, built verification techniques for them, focused heavily on integration/reuse, and just went from there. Cleanroom was one of many methods that did something similar (but semi-formal) for software in structuring, verification, and integration. Formal verification teams did it again in so many isolated cases covering so many properties and types of software. The Cleanroom, FV, and model-based synthesis results show software development could get as much verification as hardware for sure. Just would take a sizeable labor investment in tooling and libraries plus agreement to simplify implementation and interfaces to facilitate analysis. Not saying that will mainstream but it's achievable and can be incremental. At least we're getting type systems and tooling, though, from the niche (esp CompSci) that cares about the stuff. "btw, you misspelled "Harlan" as "Harlin"; the poor guy deserves better" Damn haha. He really does. Yes, Harlan's methods could be improved. No argument here. An example was how the graph-based models were doing better than hierarchical. Another is executing your code to test assumptions on its dependencies. Point about Cleanroom was that it worked, its style/results were much like engineering, demonstrated many good practices, had empirical evidence backing it, and was a logical starting point that was entirely ignored by developers. And it wasn't the only one. Far as FSM's: you nailed that one. Almost all high assurance systems used an Interacting State Machine model where FSM primitives were analysed and composed functionally (often CSP style). It was the only proven model in terms of getting results that survived all analysis and pen testing. You can do rigorous formal verification but that's high assurance: something you and other readers reject automatically in these discussions. So, I left that off. Plenty of research & products working on different models but they're still getting proven out over time. Gotta use what's proven for now if I want get the best results. "Formal verification comes from logicians and computer scientists" All true and good examples: I listed some in another comment. Then they or the engineers I refer to put them into practice. Most developers don't. Still need a term for the former that won't tie into the other crowd you've been describing that stole and tarnished the phrase.