5 ms·
Edsger W. Dijkstra: Answers to questions from students of Software Engineering
- RiderOfGiraffes 17y agoDijkstra has been mentioned before on HN: http://searchyc.com/ewd http://searchyc.com/ewd
- hvs 17y agoI'm a huge fan of Dijkstra, for obvious reasons, and I won't trot out the usual argument that "he's living in an ivory tower" or that "formal analysis is impossible in large applications", but his attitude towards "Software Engineering" does become grating after reading many of his articles. One of the primary reasons that formal analysis is not widely used in software development today (besides the fact that it is often very difficult) is not due to the stupidity or laziness of "software engineers". It is due to the radical failing of computer scientists to both effectively teach the concept, and to develop tools to make it practical. On top of that, the widely used languages in the industry today are just now beginning to allow for the type of high-level formal analysis that Dijkstra promotes. Functional languages are finally starting to make inroads into the industry in the form of Haskell, F#, Erlang; and even imperative languages such as C# are adopting functional attributes. But we are still a long way from the regular use of formal analysis (if it ever actually occurs) because the universities are not providing much, if any, guidance in this area to the industry. Formal analysis is a very hard -- maybe impossible -- problem for anything but toy applications. And I think it is an worthy goal to attempt to achieve, but it will not be well-served by denigrating the work of the many dedicated and intelligent "Software Engineers".
- spitfire 17y agoFormal analysis is a fact of life in many industries. Aviation, rail/transportation, medical systems and some power systems. All these industries have one thing in common - a lot of money to throw around. So the technology and techniques are there, they just have to be brought down to the masses. There are signs of that happening, it just may take more time than you expect.
- hvs 17y agoI don't think it'll take more time than I expect, but I take your point. Like I said, I think formal analysis of software development is a laudable goal that should be pursued. But in many ways, software is so much more complex than other engineering fields that it becomes increasingly difficult if the development is not carefully controlled (something that Dijkstra is a big believer in). I know that Dijkstra recognizes that it is not an easy problem to solve, and that he feels that computer scientists have not adequately researched this area. My main point was more about his general attitude to computer engineers than to his actual point. He addresses this well in this EWD Manuscript: http://www.cs.utexas.edu/~EWD/transcriptions/EWD13xx/EWD1304.html http://www.cs.utexas.edu/~EWD/transcriptions/EWD13xx/EWD1304...
- spitfire 17y agoI agree with you entirely. A huge problem is in mindset and culture. I've mentioned formal methods here on HN before and been essentially shouted down. Which seems odd considering the hacker-above-the-commoners ethos HN wears on its sleeve. But I digress, I think in the next few years as functional languages become more mainstream we'll see a huge uptake in formal methods. Not end to end mind you, but of the low level proover type tools. Formal project planning and design languages still take too much training to be widely accepted. Can you imaging a ruby hacker analyzing and writing up his code in Z language before they code? Neither can I. But any amount of progress on this front is a good thing.
- 10ren 17y ago
- 10ren 17y agoI love Dijkstra. I share his valuing of things that make sense, and the dream that all things can be understood - and his humble acceptance that we have to break down problems to fit inside his, mine and your human brains. But you can see where Alan Kay's "nano-Dijkstras" joke comes from, e.g. "the horrendous mess produced by a group of incompetent, unorganized programmers", referring to your complex project. Industry - and you - and me - have accepted that things don't always work perfectly. Industry pays people to fix things, rather than make them perfect in the first place. Partly it's an acceptance of human error; though it's also an acceptance that you'll need to change it even if it's perfect, because the world keeps changing. "Iterative development" elevates this to a dogma. Here's a thought experiment: imagine you need a library (e.g. for network protocols; or a fancy GUI; or to parse a EDI transactions). You have a choice of two libraries. Both have been used extensively by others, and both work well when you try them out. One is proven (in Dijsktra's sense) and one is not. 1. How much more will you pay for the proven one? 10%? x2? x10? x100? 2. Now consider how much extra work was required to rigorously build the proven one. (Of course, if the builder of the component also had access to proven sub-components, this would be easier). [You can imagine doing this by building it yourself if you like - using dollars just makes it clearer.]
- hvs 17y agoI'm not sure why you got downvoted for this as I think it is a fair assessment. I think it assumes that the current state of development will continue on its current path, though. The increasing complexity of applications demands that we come up with better models for development that allow us to think about problems more abstractly. One could make the argument that this is only truly possible with formally verified applications.
- 10ren 17y agoHistorically, we SOTSOG - using abstractions, so complexity is pushed down to a lower layer (language, libraries, OS). I think this will continue to be how we cope with complexity - one specific case at a time, rather than a new general way to address all complexity issues (sounds like magic). If the rate of complexity increase itself increases (i.e. accelerates), then we might need a a different methodology altogether, such formal verification as you suggest. However, we have had the need for better ways to handle complexity since before Fred's Mythical Man-month (IBM 360), and we've had the tool of formal verification techniques to do it for a long time. It hasn't happened yet, suggesting there is some problem with it. In my admittedly very limited experience of formal verification, it makes problems more complex, rather than less (This is why Dijkstra says that you can't prove software the way it is currently written - you have to write it in a simpler way, with clearer interactions between modules and so on). Its benefit is that you know it's 100% correct (assuming your assumptions, understanding of the real-world problem, understanding of interactions with other components are all correct, your proof is correct, and any proof automation tools you use are also 100% correct). OK, it gives you a higher probability that it is correct. Ah, this looks like the opposite cause and effect from your suggestion (that formal methods are a way to handle complexity), and instead that trying to formally verify applications forces us to write them in a simpler way, which is... simpler. Or, we could just write them in a simpler way, and not worry about the formal verification part. It still leaves the problem: what is this way of writing apps in a simpler way? Isn't the only way to do this by dividing it into abstractions with well-defined behaviours and clear interfaces? In other words: Historically, we SOTSOG - using abstractions, so complexity is pushed down to a lower layer (language, libraries, OS). PS: I would love it if there was a new general way to approach complexity. But as Fred said, "no silver bullet". "better models for development that allow us to think about problems more abstractly" That's possible I guess; but I don't think we lack in our ability to think about problems abstractly. It's finding the right abstraction that is tricky, and this amounts to a search through the space of abstractions. That sounds straightforward to automate, except that he search space increases exponentially with the complexity of the problem. So we need a heuristic to guide that search. So far, the best heuristic is us. If we can automate this guide through the search space - the ability to make connections, have intuitions and insights - it would be a huge leap towards Strong AI. My Masters was an approach to this, and the start of my PhD. Quite possible, I think, but hard.
- stefano 17y agoMy favourite one: "It is not the task of the University to offer what society asks for, but to give what society needs. [The things society asks for are generally understood, and you don't need a University for that; the university has to offer what no one else can provide.]" The sad thing is that CS courses tend to go towards the opposite direction.
- abalashov 17y agoNo, I'm afraid that computer science has suffered from the popularity of the Internet. It has attracted an increasing —not to say: overwhelming!— number of students with very little scientific inclination and in research it has only strengthened the prevailing (and somewhat vulgar) obsession with speed and capacity. Hallelujah, old friend - speak the word.
- tpyo 17y agoWhat does the computer industry need saving from?