5 ms·

All good points. "software engineering" is how we take raw code and turn it into reliable systems. The worst code you'll ever see written is by computer scien

by seiji 11y ago

All good points. "software engineering" is how we take raw code and turn it into reliable systems. The worst code you'll ever see written is by computer science PhDs who just have to get something done for their research paper experiments. It'll be 6,000 lines of matlab with no loops, no functions, and entire 30 line sections just get copy/pasted when they need to repeat an action. That's code without software engineering. That's "coding" and not "developing." In a way, software engineering is a moving goalpost in the same way as "if a computer can do it now, it's not AI." Many of our standard practices these days used to be "must be considered separately" software engineering practices 10 years ago (revision control, proper documentation, creating libraries, etc). Open Source projects run by communities with a focus towards testing, documentation, and maintainability are the best thing to happen to software engineering. But, there are still crazy people running around writing code just for themselves and refusing to realize software is more than code code code these days. We pretty much need two divisions here: "software engineering (social)" and "software engineering (technical)". Social software engineering is defined by well maintained, long-lived, multi-contributor software projects. Technical software engineering is the implementation of the testing systems and contribution systems behind those long lived sustainable projects (e.g. are you calculating the branch coverage of your unit tests? do you even have unit tests? (no, testing your live servers aren't "unit" tests, those are integration tests) are you testing all combinations of acceptable and invalid inputs (null, proper value, invalid value, out of range value positive, out of range value negative, ...)? are your systems even testable and not bound to side-effect laden global states? do you have proper mock objects to compensate for your side-effect driven development? do you allow outside contributions so you don't have to maintain all 60 of these detailed testing and failure scenarios yourself?). (there's also the entire other branch of "how to be a better software writer through requirements gathering and agile/lean/XP/monkeypants practices," but that's out of scope here.)

nickpsecurity 11y ago
"All good points. "software engineering" is how we take raw code and turn it into reliable systems." No, that's software development. It comes in many forms. Your academic example is a good one of a failed approach. Software engineering is applying engineering practices to software to produce The Right Thing. Examples of it are Cleanroom, Praxis' Correct-by-Construction, Green Hill's PHASE, and so on. These connect requirements to clear specs (semi-formal or mathematical) to code. The systems are usually abstracted, modular, and layered in a way to allow testing of individual components and how they connect. Both successful and failure states are documented with fail-safe methodology. Design/code reviews, testing, static analysis, automated build processes, and so on are common here. The results are consistent, effective, and with low defects. Much like engineering. Then, there's software development and programming in general. They're selectively applying some of this stuff but no way looks like engineering. The results are likewise all over the place. "Open Source projects run by communities with a focus towards testing, documentation, and maintainability are the best thing to happen to software engineering." All of that was happening in software engineering back in the 1960's-1980's for companies that tried to do the real thing. Just formal, code reviews and testing got incredible reliability gains (see OpenVMS). High assurance systems did much more. An article, about both security & software engineering, described the very best of security in 1970's with specific practices and results they had. Most software development hasn't caught up to that level despite the fact that every aspect of it was exceeded in academia and professional communities focused on quality/security. What I do agree with is that OSS projects using such methods have gotten them mainstream adoption in many OSS and proprietary projects. The benefits were real. However, the stuff already existed for decades, was even mandated by some standards, and the mainstream is a pale imitation of all that. Improving, for sure, but it's actually them that still have much to learn. If only the will. "We pretty much need two divisions here: "software engineering (social)" and "software engineering (technical)". " This I'm less sure about as now we're on an open-ended topic. :) There's definitely social and technical factors. I'd say the technical side is designing, implementing, and verifying any aspect of it. You need to understand the tech to varying degrees to do each. At same time, there's a social component because everyone is working together and one can affect another. seL4 teams, development and formal methods, were an example where constant communication solved issues where one side's abstraction/implementation was causing severe issues on the other. So, it's all technical and social minus maybe project manager or owner(s) making the money off it. ;)
kragen 11y ago
Revision control, proper documentation, creating libraries, use of high-level languages, were common practice at Bell Labs ("programming, motherfucker") in the 1970s, and alien to "software engineering"-driven groups like Computer Sciences Corporation (CMM Level 5!) and Harlan Mills's group at IBM. What we usually call "unit tests" these days, and certainly "mock objects", came from Extreme Programming, which is a reaction against the "software engineering" managerialism cult. I agree with you that those things are how you take raw code and turn it into reliable systems, and that you don't get there by copying and pasting code and writing 6000-line functions. But this article is not promoting those things. It's promoting formalized process engineering. (However, in my book, using very-high-level languages like Matlab (or, better, Octave or Numpy), and high-level operations like matrix-multiply instead of explicit loops, are best practices.) I do agree that social practices around effectively collaborating with other people are very important and require a lot of effort. You can have the world's most effective hackers on your project, but if they're wasting their time because you don't have a budget for a test server or because they're not allowed to fix the bugs they find, your productivity is still going to be low.
- nickpsecurity 11y ago
  I totally agree about the process junkies ignoring good stuff and pushing their religion under banner of software engineering. They're the posers who weakened the term. The Bell Labs people were real, software engineers both experimenting and building on proven methods. Also... "What we usually call "unit tests" these days, and certainly "mock objects"" That's not true. Like other assurance activities, Extreme Programming just did what was already done, gave it a new name, mixed it with bullshit (esp pair programming), and got it some needed attention. The Orange Book testing criteria, in 1983, required as much testing as resources allowed of primitives, their compositions, all interfaces, and common failure conditions. That's on top of specifications, covert channel analysis, pen testing, using only proven components, build system automation, etc that far exceed XP's quality regiment. Funny you mention Cleanroom because it was first thing I thought of when I heard the "new" Agile approach. Iterative development, code reviews, usage-based testing for acceptance, verifying each module... Cleanroom had all that. At least they eventually figured it out and it seems competitors in Agile identified XP's problems as well. Although old, I'm glad the concepts are mainstreaming because it gave us something important: solid, well-maintained, constantly-updated TOOLS to support the efforts. :)
azhenley 11y ago
What you're referring to is known as "production bias." A CS PhD student isn't evaluated on his code, but rather the result. Same for any "end user programmer," they just want the result and not good quality code, even if it would benefit them more in the long run to write good code (this is known as the "paradox of the active user").

by seiji 11y ago

nickpsecurity 11y ago

"All good points. "software engineering" is how we take raw code and turn it into reliable systems." No, that's software development. It comes in many forms. Your academic example is a good one of a failed approach. Software engineering is applying engineering practices to software to produce The Right Thing. Examples of it are Cleanroom, Praxis' Correct-by-Construction, Green Hill's PHASE, and so on. These connect requirements to clear specs (semi-formal or mathematical) to code. The systems are usually abstracted, modular, and layered in a way to allow testing of individual components and how they connect. Both successful and failure states are documented with fail-safe methodology. Design/code reviews, testing, static analysis, automated build processes, and so on are common here. The results are consistent, effective, and with low defects. Much like engineering. Then, there's software development and programming in general. They're selectively applying some of this stuff but no way looks like engineering. The results are likewise all over the place. "Open Source projects run by communities with a focus towards testing, documentation, and maintainability are the best thing to happen to software engineering." All of that was happening in software engineering back in the 1960's-1980's for companies that tried to do the real thing. Just formal, code reviews and testing got incredible reliability gains (see OpenVMS). High assurance systems did much more. An article, about both security & software engineering, described the very best of security in 1970's with specific practices and results they had. Most software development hasn't caught up to that level despite the fact that every aspect of it was exceeded in academia and professional communities focused on quality/security. What I do agree with is that OSS projects using such methods have gotten them mainstream adoption in many OSS and proprietary projects. The benefits were real. However, the stuff already existed for decades, was even mandated by some standards, and the mainstream is a pale imitation of all that. Improving, for sure, but it's actually them that still have much to learn. If only the will. "We pretty much need two divisions here: "software engineering (social)" and "software engineering (technical)". " This I'm less sure about as now we're on an open-ended topic. :) There's definitely social and technical factors. I'd say the technical side is designing, implementing, and verifying any aspect of it. You need to understand the tech to varying degrees to do each. At same time, there's a social component because everyone is working together and one can affect another. seL4 teams, development and formal methods, were an example where constant communication solved issues where one side's abstraction/implementation was causing severe issues on the other. So, it's all technical and social minus maybe project manager or owner(s) making the money off it. ;)

kragen 11y ago

Revision control, proper documentation, creating libraries, use of high-level languages, were common practice at Bell Labs ("programming, motherfucker") in the 1970s, and alien to "software engineering"-driven groups like Computer Sciences Corporation (CMM Level 5!) and Harlan Mills's group at IBM. What we usually call "unit tests" these days, and certainly "mock objects", came from Extreme Programming, which is a reaction against the "software engineering" managerialism cult. I agree with you that those things are how you take raw code and turn it into reliable systems, and that you don't get there by copying and pasting code and writing 6000-line functions. But this article is not promoting those things. It's promoting formalized process engineering. (However, in my book, using very-high-level languages like Matlab (or, better, Octave or Numpy), and high-level operations like matrix-multiply instead of explicit loops, are best practices.) I do agree that social practices around effectively collaborating with other people are very important and require a lot of effort. You can have the world's most effective hackers on your project, but if they're wasting their time because you don't have a budget for a test server or because they're not allowed to fix the bugs they find, your productivity is still going to be low.

nickpsecurity 11y ago
I totally agree about the process junkies ignoring good stuff and pushing their religion under banner of software engineering. They're the posers who weakened the term. The Bell Labs people were real, software engineers both experimenting and building on proven methods. Also... "What we usually call "unit tests" these days, and certainly "mock objects"" That's not true. Like other assurance activities, Extreme Programming just did what was already done, gave it a new name, mixed it with bullshit (esp pair programming), and got it some needed attention. The Orange Book testing criteria, in 1983, required as much testing as resources allowed of primitives, their compositions, all interfaces, and common failure conditions. That's on top of specifications, covert channel analysis, pen testing, using only proven components, build system automation, etc that far exceed XP's quality regiment. Funny you mention Cleanroom because it was first thing I thought of when I heard the "new" Agile approach. Iterative development, code reviews, usage-based testing for acceptance, verifying each module... Cleanroom had all that. At least they eventually figured it out and it seems competitors in Agile identified XP's problems as well. Although old, I'm glad the concepts are mainstreaming because it gave us something important: solid, well-maintained, constantly-updated TOOLS to support the efforts. :)

azhenley 11y ago

What you're referring to is known as "production bias." A CS PhD student isn't evaluated on his code, but rather the result. Same for any "end user programmer," they just want the result and not good quality code, even if it would benefit them more in the long run to write good code (this is known as the "paradox of the active user").