5 ms·

Friction: why we can't build software as fast as we think we can

40 points by dools 13y ago

thruflo 13y ago
The article moves away from its interesting proposition to just restate something bland about keeping scopes tight. We can't build software as fast as we can think because we don't have the tools. What those tools should be is an interesting and important thought experiment.
- wvenable 13y ago
  Unless that tool is a time machine, how will any other tool help determine all the possible unknown issues a project might or might not have.
- avmich 13y ago
  See http://www.jsoftware.com/papers/tot.htm http://www.jsoftware.com/papers/tot.htm .
  - gtani 13y ago
    Thanks, will read later (me: former 15 hour/day APL programmer at Merrill Lynch)
    - avmich 13y ago
      Then you probably know that already. The link is rather for those who seeks tools "to work with the speed of thought". Different languages claim achievement of different grades of "if you can state the problem in the language, you're done", but APL family does that with its own flavor of elegance.
- danmaz74 13y ago
  >We can't build software as fast as we can think because we don't have the tools Actually, the real "problem" is that, as soon as we have the tools to easily solve something that before was difficult, we move the target a step farther. But that's also why programming is so interesting, and programmers are in such high demand, so I wouldn't really call it a "problem".
  - lttlrck 13y ago
    The right tools move the goalposts. If this wasn't the case things would stagnate pretty fast.
- dools 13y ago
  I just made a comment further up the thread which is relevant to this one: https://news.ycombinator.com/item?id=5810623 https://news.ycombinator.com/item?id=5810623 The original title of the article is the "bland" one, not the "interesting" one ;) However I do think that tools have the ability to solve the bland problem, too.
wpietri 13y ago
This is a good start, but I wish the author had taken it deeper. E.g.: * Can a team discover patterns in the things they underestimate? * Can a team do better by not estimating? * Can a business do better by not estimating? * What are the axes along which scope can be cut? * What emotional needs keep stakeholders from cutting scope? I also have found a lot of value in trying to stop using terms like "launch"; I think they confuse things more than they help.
- Spearchucker 13y ago
  Can a team/business do better by not estimating? No, you can't (or rather, I don't see how). As he said, the most reliable way around it is having someone on your team that's been through it before. Most don't, because the experience that implies is not easily obtainable (if it were this problem wouldn't be as endemic). An estimate makes the person (ideally team) think the solution through, which should (assuming the estimate has any semblance of realism) bring out at least a conceptual understanding of the scope (which bounds the idea). Estimation, unfortunately, comes with baggage. In the self-managing agile world we have today, the inexperienced devs facing this problem are the same ones that dislike design up front, and consequently a coherent estimate.
  - wpietri 13y ago
    I think it depends on the context. At my last company, we dropped mostly estimating because it wasn't getting us much in the way of value. We had other things to make us think the solution through. It also let us give up some illusions: That there was a single right solution. That we could know the idea's bounds in advance. That we wouldn't learn anything important by releasing early and often. Of course, we were a startup. In that context, it's easier to see that neither the problem nor the solution is understood. But looking back on other projects, I think 90% of the time I've spent estimating was wasted. In the future I expect I'd only do estimates if a) there were a question about being able to meet a hard external deadline (as opposed to a made-up bullshit deadline, which is most of them), or b) we needed to choose between alternative paths that were large enough that handwave relative estimates wouldn't cover it.
tudorconstantin 13y ago
The only estimation I give on tasks is "as fast as possible"
nickthemagicman 13y ago
I think the problem with programming is fricking programmers. We write so much obtuse code with no documentation, adhering to no standards, and with no tutorials that you can literally get side tracked for days on some esoteric dependency that has you bashing your head against the wall. (getting mcrypt working with php I'm looking at you) You should keep other developers in mind when writing software.
- deleted 13y ago
  [deleted]
- dools 13y ago
  A couple of people in this thread have mentioned "tools" and I think this is mostly because some friendly admin at YC decided to editorialise the title of my post so that it differs from the article (and subtly changes the meaning of it), so they're talking about tools that help us build software as fast as we can think... which is different from the original title of the article which says "as fast as we think we can". Anyway. I think that where tools can help is by making a wider variety of different projects more predictable because domain experts create easy to use tools that abstract away alot of the friction for us. One notable example is jQuery: it has allowed a huge variety of non-javascript experts to very reliably produce work that, in the old days, would have been wildly unpredictable due to skull crushing differences in browser behaviour and the intricacies of JavaScript as a language.
jconley 13y ago
Reminds me of a great Steve McConnell book, Software Estimation: Demystifying the Black Art. http://www.amazon.com/gp/aw/d/0735605351/ref=redir_mdp_mobile http://www.amazon.com/gp/aw/d/0735605351/ref=redir_mdp_mobil...
- deleted 13y ago
  [deleted]
- jacques_chester 13y ago
  I agree with this recommendation. McConnell has done his usual trick of distilling lots of material into a concise, readable account.
macca321 13y ago
AKA You Ain't Gonna Need It, sensible advice indeed. To me though, Friction in software development is something slightly different: https://www.google.co.uk/search?q=%22zero+friction%22+ayende https://www.google.co.uk/search?q=%22zero+friction%22+ayende
- dools 13y ago
  YAGNI to me is something different from scope creep. YAGNI, to me, is more about programmers tendency to want to "solve each problem only once" which leans to premature abstraction and generalisation, more often than not before they truly understand the scope of the problem. Being ruthless about what features you include is more about being able to say, at the end of the project when it's overdue and everyone's hassled out of their brain "well, at least we know we got it done as fast as we possibly could have". I'm always reminded of the HHGTTG bit (can't remember which book it's in) about the ship waiting until a new society arises that is capable of manufacturing lemon scented napkins so they can take off[1]. I think we spend a lot of time in software building lemon scented napkins :) [1]http://thelostcityofcarcosa.com/2012/11/12/lemon-soaked-napkins/ http://thelostcityofcarcosa.com/2012/11/12/lemon-soaked-napk...
swalsh 13y ago
Here's the thing, estimating software development time is essentially equivalent to estimating time to think + overhead. Overhead includes actually typing the code, booting the computer, dealing SVN issues, etc. The overhead is relatively consistent, and can be minimized and predicted. Time to think is highly variable. Previously I used to work for a consulting company. After about a year of working there the company started to settle in a particular niche. We built libraries to do things, then we just started to churn out work. Predicting development time became an exact science, because all we had to do was estimate overhead. There was very little new things, so time to think was minimized to nearly nothing. I hated working in a code factory, so I left for greener pastures. The new place has a lot of new things, trying to fit it together is a real challenge, and requires thinking. My estimates are no where near where they were at my old company. My main point is this, estimating software time is hard because you don't know what you don't know. If you can solve that problem, you solved one of humanity's greatest challenges.
mckoss 13y ago
There are two different questions here: - Why can't we build software as fast as we think? - Why can't we build software as fast as we think we can? The author probably meant the latter (which is more associated with estimation and prediction). But I find the former question MUCH more interesting. We've all had the experience of envisioning software solutions that would be useful to us - but then the actual task of developing that software seems much more difficult than the task SHOULD be. I attribute much of this to IMPEDANCE rather than FRICTION. We are often not using the best tools for the job, and hence suffer great amount of superfluous work in solving any particular problem. General purpose programming languages may always have this problem; being able to describe any solution means that they are not particularly well suiting to expressing solutions to particular classes of problems). I've always experienced myself being much more efficient when I have domain-specific languages specifically tailored to the class of problems I want to solve. I think we are moving (slowly) in the direction of developing more targeted languages (e.g. Angular or Dart from Google for client-side web development).
- dools 13y ago
  Actually the title of the article is "Friction: why you can't build software as fast as you think you can" (emphasis here only). Some admin was obviously in an editorial mood and has updated it to confuse everyone.
- jacques_chester 13y ago
  Fred Brooks pointed out in his famous No Silver Bullet essay [1] that the complexity bound on software development is set by the problem domain, not the solution domain. No matter how much better we get at our jobs, the world remains complex and our performance will be bound by that complexity. [1] http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBullet.html http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBulle...
jacques_chester 13y ago
In their doctrinal manual, Warfighting, the US Marine Corps use the same term and with a similar meaning. The original analogy comes from von Clausewitz. Friction is the force that resists all action and saps energy. It makes the simple difficult and the difficult seemingly impossible. Friction may be mental, as in indecision over a course of action. It may be physical, as in effective enemy fire or a terrain obstacle that must be overcome. Friction may be external, imposed by enemy action, the terrain, weather, or mere chance. Friction may be self-induced, caused by such factors as lack of a clearly defined goal, lack of coordination, unclear or complicated plans, complex task organizations or command relationships, or complicated technologies. Whatever form it takes, because war is a human enterprise, friction will always have a psychological as well as a physical impact.

40 points by dools 13y ago

The article moves away from its interesting proposition to just restate something bland about keeping scopes tight. We can't build software as fast as we can think because we don't have the tools. What those tools should be is an interesting and important thought experiment.

wvenable 13y ago
Unless that tool is a time machine, how will any other tool help determine all the possible unknown issues a project might or might not have.
avmich 13y ago
See http://www.jsoftware.com/papers/tot.htm http://www.jsoftware.com/papers/tot.htm .
- gtani 13y ago
  Thanks, will read later (me: former 15 hour/day APL programmer at Merrill Lynch)
  - avmich 13y ago
    Then you probably know that already. The link is rather for those who seeks tools "to work with the speed of thought". Different languages claim achievement of different grades of "if you can state the problem in the language, you're done", but APL family does that with its own flavor of elegance.
danmaz74 13y ago
>We can't build software as fast as we can think because we don't have the tools Actually, the real "problem" is that, as soon as we have the tools to easily solve something that before was difficult, we move the target a step farther. But that's also why programming is so interesting, and programmers are in such high demand, so I wouldn't really call it a "problem".
- lttlrck 13y ago
  The right tools move the goalposts. If this wasn't the case things would stagnate pretty fast.
dools 13y ago
I just made a comment further up the thread which is relevant to this one: https://news.ycombinator.com/item?id=5810623 https://news.ycombinator.com/item?id=5810623 The original title of the article is the "bland" one, not the "interesting" one ;) However I do think that tools have the ability to solve the bland problem, too.

wpietri 13y ago

This is a good start, but I wish the author had taken it deeper. E.g.: * Can a team discover patterns in the things they underestimate? * Can a team do better by not estimating? * Can a business do better by not estimating? * What are the axes along which scope can be cut? * What emotional needs keep stakeholders from cutting scope? I also have found a lot of value in trying to stop using terms like "launch"; I think they confuse things more than they help.

Spearchucker 13y ago
Can a team/business do better by not estimating? No, you can't (or rather, I don't see how). As he said, the most reliable way around it is having someone on your team that's been through it before. Most don't, because the experience that implies is not easily obtainable (if it were this problem wouldn't be as endemic). An estimate makes the person (ideally team) think the solution through, which should (assuming the estimate has any semblance of realism) bring out at least a conceptual understanding of the scope (which bounds the idea). Estimation, unfortunately, comes with baggage. In the self-managing agile world we have today, the inexperienced devs facing this problem are the same ones that dislike design up front, and consequently a coherent estimate.
- wpietri 13y ago
  I think it depends on the context. At my last company, we dropped mostly estimating because it wasn't getting us much in the way of value. We had other things to make us think the solution through. It also let us give up some illusions: That there was a single right solution. That we could know the idea's bounds in advance. That we wouldn't learn anything important by releasing early and often. Of course, we were a startup. In that context, it's easier to see that neither the problem nor the solution is understood. But looking back on other projects, I think 90% of the time I've spent estimating was wasted. In the future I expect I'd only do estimates if a) there were a question about being able to meet a hard external deadline (as opposed to a made-up bullshit deadline, which is most of them), or b) we needed to choose between alternative paths that were large enough that handwave relative estimates wouldn't cover it.

tudorconstantin 13y ago

The only estimation I give on tasks is "as fast as possible"

nickthemagicman 13y ago

I think the problem with programming is fricking programmers. We write so much obtuse code with no documentation, adhering to no standards, and with no tutorials that you can literally get side tracked for days on some esoteric dependency that has you bashing your head against the wall. (getting mcrypt working with php I'm looking at you) You should keep other developers in mind when writing software.

deleted 13y ago
[deleted]
dools 13y ago
A couple of people in this thread have mentioned "tools" and I think this is mostly because some friendly admin at YC decided to editorialise the title of my post so that it differs from the article (and subtly changes the meaning of it), so they're talking about tools that help us build software as fast as we can think... which is different from the original title of the article which says "as fast as we think we can". Anyway. I think that where tools can help is by making a wider variety of different projects more predictable because domain experts create easy to use tools that abstract away alot of the friction for us. One notable example is jQuery: it has allowed a huge variety of non-javascript experts to very reliably produce work that, in the old days, would have been wildly unpredictable due to skull crushing differences in browser behaviour and the intricacies of JavaScript as a language.

jconley 13y ago

Reminds me of a great Steve McConnell book, Software Estimation: Demystifying the Black Art. http://www.amazon.com/gp/aw/d/0735605351/ref=redir_mdp_mobile http://www.amazon.com/gp/aw/d/0735605351/ref=redir_mdp_mobil...

deleted 13y ago
[deleted]
jacques_chester 13y ago
I agree with this recommendation. McConnell has done his usual trick of distilling lots of material into a concise, readable account.

macca321 13y ago

AKA You Ain't Gonna Need It, sensible advice indeed. To me though, Friction in software development is something slightly different: https://www.google.co.uk/search?q=%22zero+friction%22+ayende https://www.google.co.uk/search?q=%22zero+friction%22+ayende

dools 13y ago
YAGNI to me is something different from scope creep. YAGNI, to me, is more about programmers tendency to want to "solve each problem only once" which leans to premature abstraction and generalisation, more often than not before they truly understand the scope of the problem. Being ruthless about what features you include is more about being able to say, at the end of the project when it's overdue and everyone's hassled out of their brain "well, at least we know we got it done as fast as we possibly could have". I'm always reminded of the HHGTTG bit (can't remember which book it's in) about the ship waiting until a new society arises that is capable of manufacturing lemon scented napkins so they can take off[1]. I think we spend a lot of time in software building lemon scented napkins :) [1]http://thelostcityofcarcosa.com/2012/11/12/lemon-soaked-napkins/ http://thelostcityofcarcosa.com/2012/11/12/lemon-soaked-napk...

swalsh 13y ago

Here's the thing, estimating software development time is essentially equivalent to estimating time to think + overhead. Overhead includes actually typing the code, booting the computer, dealing SVN issues, etc. The overhead is relatively consistent, and can be minimized and predicted. Time to think is highly variable. Previously I used to work for a consulting company. After about a year of working there the company started to settle in a particular niche. We built libraries to do things, then we just started to churn out work. Predicting development time became an exact science, because all we had to do was estimate overhead. There was very little new things, so time to think was minimized to nearly nothing. I hated working in a code factory, so I left for greener pastures. The new place has a lot of new things, trying to fit it together is a real challenge, and requires thinking. My estimates are no where near where they were at my old company. My main point is this, estimating software time is hard because you don't know what you don't know. If you can solve that problem, you solved one of humanity's greatest challenges.

mckoss 13y ago

There are two different questions here: - Why can't we build software as fast as we think? - Why can't we build software as fast as we think we can? The author probably meant the latter (which is more associated with estimation and prediction). But I find the former question MUCH more interesting. We've all had the experience of envisioning software solutions that would be useful to us - but then the actual task of developing that software seems much more difficult than the task SHOULD be. I attribute much of this to IMPEDANCE rather than FRICTION. We are often not using the best tools for the job, and hence suffer great amount of superfluous work in solving any particular problem. General purpose programming languages may always have this problem; being able to describe any solution means that they are not particularly well suiting to expressing solutions to particular classes of problems). I've always experienced myself being much more efficient when I have domain-specific languages specifically tailored to the class of problems I want to solve. I think we are moving (slowly) in the direction of developing more targeted languages (e.g. Angular or Dart from Google for client-side web development).

dools 13y ago
Actually the title of the article is "Friction: why you can't build software as fast as you think you can" (emphasis here only). Some admin was obviously in an editorial mood and has updated it to confuse everyone.
jacques_chester 13y ago
Fred Brooks pointed out in his famous No Silver Bullet essay [1] that the complexity bound on software development is set by the problem domain, not the solution domain. No matter how much better we get at our jobs, the world remains complex and our performance will be bound by that complexity. [1] http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBullet.html http://www.cs.nott.ac.uk/~cah/G51ISS/Documents/NoSilverBulle...

jacques_chester 13y ago

In their doctrinal manual, Warfighting, the US Marine Corps use the same term and with a similar meaning. The original analogy comes from von Clausewitz. Friction is the force that resists all action and saps energy. It makes the simple difficult and the difficult seemingly impossible. Friction may be mental, as in indecision over a course of action. It may be physical, as in effective enemy fire or a terrain obstacle that must be overcome. Friction may be external, imposed by enemy action, the terrain, weather, or mere chance. Friction may be self-induced, caused by such factors as lack of a clearly defined goal, lack of coordination, unclear or complicated plans, complex task organizations or command relationships, or complicated technologies. Whatever form it takes, because war is a human enterprise, friction will always have a psychological as well as a physical impact.