5 ms·

No Silver Bullet: Essence and Accidents of Software Engineering (1987)

41 points by sturza 6y ago

ncmncm 6y ago
Dan Luu just wrote a blog post about this. He says BL is fundamentally wrong in his analysis, and that the overwhelming majority of time spent programming in 1987 turns out to be what BL calls "accident", and has been radically reduced by progress since. There is still no "silver bullet", but there are many, many bullets with cumulative effect, so that many programming tasks that would have taken prohibitively long in 1987 can now be done on a whim. https://danluu.com/essential-complexity/ https://danluu.com/essential-complexity/ Probably the biggest bullet is greater accessibility and power of libraries, coupled with languages better able to express and use them. When libraries are more useful, there is more motivation to make and publish them, and so there are more available to use.
- Twisol 6y ago
  The 2006 paper "Out of the Tar Pit" also disagrees with Brooks' estimate of total accidental complexity, explaining their position in terms of {essential,accidental} {state,control}. One of their arguments is that accidental (derived) state has a magnifying effect on complexity, so a first-order goal of any software architecture should be to reduce accidental state as far as possible -- and arguably, centralize the remaining essential state as much as possible. https://github.com/papers-we-love/papers-we-love/blob/master/design/out-of-the-tar-pit.pdf https://github.com/papers-we-love/papers-we-love/blob/master... It somewhat puts the old saying in another light: "There are two hard problems in computer science: cache invalidation and naming things." Cached state is, effectively, another name for derived state.
  - ezekiel68 6y ago
    You neglected to mention the third of the two hard problems: "off-by-one errors" :D
- dang 6y ago
  Discussed a month ago: https://news.ycombinator.com/item?id=25569148 https://news.ycombinator.com/item?id=25569148
- andreskytt 6y ago
  The basic principle of practical systems theory states, that you can’t remove complexity, you can merely transform it into a form of complexity you are more or less capable of handling. Libraries are hugely complex nowadays. Did a test a while back: from zero to a rudimentary web service with Spring on took ~2 days, with go (lacking any previous experience with the language) it was a few hours.
dang 6y ago
If curious see also 2019 https://news.ycombinator.com/item?id=20818537 https://news.ycombinator.com/item?id=20818537 2017 https://news.ycombinator.com/item?id=15476733 https://news.ycombinator.com/item?id=15476733 2015 https://news.ycombinator.com/item?id=10306335 https://news.ycombinator.com/item?id=10306335 2011 (1 comment) https://news.ycombinator.com/item?id=3068513 https://news.ycombinator.com/item?id=3068513 Related from 2008 (by Brad Cox): https://news.ycombinator.com/item?id=239323 https://news.ycombinator.com/item?id=239323
ezekiel68 6y ago
"These gains flow from the transformation of computer manufacture from an assembly industry into a process industry." [from the posted article] Fred Brooks is a true genius to percolate a generation's worth of transformation into such a statement.
pbiggar 6y ago
This was a formative essay for me. When I look around the software industry, it's completely full of accidental complexity. Consider Kubernetes and the entire Cloud Native world - how much of that is actual business logic that your customers need? None of it. The main areas of complexity I perceive today in building software are: - "infrastructure complexity" (see above) - "deployment complexity": continuous delivery and getting code into production - API complexity: using 3rd party tools used to be function calls, now they're complex distributed systems - code complexity: git, pull requests, setting up your dev environment, etc I wrote more about this here [1], but I honestly can't believe how complex it is to create software today. [1] https://blog.darklang.com/what-is-dark/ https://blog.darklang.com/what-is-dark/
- rramadass 6y ago
  Very right! The frustrating thing is that it doesn't have to be this way. The "tower of babel" of languages/tools/technologies used in Web/Cloud/DevOps programming is daunting and i don't feel like it is really worth my time learning any of it unless and until i actually need it for a project.
lincpa 6y ago
"The grand unified programming theory" is the only way to solve this problem. https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow

41 points by sturza 6y ago

Dan Luu just wrote a blog post about this. He says BL is fundamentally wrong in his analysis, and that the overwhelming majority of time spent programming in 1987 turns out to be what BL calls "accident", and has been radically reduced by progress since. There is still no "silver bullet", but there are many, many bullets with cumulative effect, so that many programming tasks that would have taken prohibitively long in 1987 can now be done on a whim. https://danluu.com/essential-complexity/ https://danluu.com/essential-complexity/ Probably the biggest bullet is greater accessibility and power of libraries, coupled with languages better able to express and use them. When libraries are more useful, there is more motivation to make and publish them, and so there are more available to use.

Twisol 6y ago
The 2006 paper "Out of the Tar Pit" also disagrees with Brooks' estimate of total accidental complexity, explaining their position in terms of {essential,accidental} {state,control}. One of their arguments is that accidental (derived) state has a magnifying effect on complexity, so a first-order goal of any software architecture should be to reduce accidental state as far as possible -- and arguably, centralize the remaining essential state as much as possible. https://github.com/papers-we-love/papers-we-love/blob/master/design/out-of-the-tar-pit.pdf https://github.com/papers-we-love/papers-we-love/blob/master... It somewhat puts the old saying in another light: "There are two hard problems in computer science: cache invalidation and naming things." Cached state is, effectively, another name for derived state.
- ezekiel68 6y ago
  You neglected to mention the third of the two hard problems: "off-by-one errors" :D
dang 6y ago
Discussed a month ago: https://news.ycombinator.com/item?id=25569148 https://news.ycombinator.com/item?id=25569148
andreskytt 6y ago
The basic principle of practical systems theory states, that you can’t remove complexity, you can merely transform it into a form of complexity you are more or less capable of handling. Libraries are hugely complex nowadays. Did a test a while back: from zero to a rudimentary web service with Spring on took ~2 days, with go (lacking any previous experience with the language) it was a few hours.

dang 6y ago

If curious see also 2019 https://news.ycombinator.com/item?id=20818537 https://news.ycombinator.com/item?id=20818537 2017 https://news.ycombinator.com/item?id=15476733 https://news.ycombinator.com/item?id=15476733 2015 https://news.ycombinator.com/item?id=10306335 https://news.ycombinator.com/item?id=10306335 2011 (1 comment) https://news.ycombinator.com/item?id=3068513 https://news.ycombinator.com/item?id=3068513 Related from 2008 (by Brad Cox): https://news.ycombinator.com/item?id=239323 https://news.ycombinator.com/item?id=239323

ezekiel68 6y ago

"These gains flow from the transformation of computer manufacture from an assembly industry into a process industry." [from the posted article] Fred Brooks is a true genius to percolate a generation's worth of transformation into such a statement.

pbiggar 6y ago

This was a formative essay for me. When I look around the software industry, it's completely full of accidental complexity. Consider Kubernetes and the entire Cloud Native world - how much of that is actual business logic that your customers need? None of it. The main areas of complexity I perceive today in building software are: - "infrastructure complexity" (see above) - "deployment complexity": continuous delivery and getting code into production - API complexity: using 3rd party tools used to be function calls, now they're complex distributed systems - code complexity: git, pull requests, setting up your dev environment, etc I wrote more about this here [1], but I honestly can't believe how complex it is to create software today. [1] https://blog.darklang.com/what-is-dark/ https://blog.darklang.com/what-is-dark/

rramadass 6y ago
Very right! The frustrating thing is that it doesn't have to be this way. The "tower of babel" of languages/tools/technologies used in Web/Cloud/DevOps programming is daunting and i don't feel like it is really worth my time learning any of it unless and until i actually need it for a project.

lincpa 6y ago

"The grand unified programming theory" is the only way to solve this problem. https://github.com/linpengcheng/PurefunctionPipelineDataflow https://github.com/linpengcheng/PurefunctionPipelineDataflow