8 ms·

What Do We Know About Time Pressure in Software Development?

150 points by chrisaycock 5y ago

cjblomqvist 5y ago
Maybe this will be hugely impopular, but in addition to the three causes mentioned, isn't it possible that one cause actually comes from developers simply not performing? If comparing to sports, in a team, is it always the coach's fault? Or is it possible that one player is not performing as expected? Is it then the coach's issue that he/she didn't expect that?
rising-sky 5y ago
> [In this article, time pressure refers to both budget pressure, e.g., eight hours (budget) to do a task, and schedule pressure, e.g., a task that must be done by tomorrow.] Aren't these the same thing? the latter being somewhere around 24hours give or take
- flerchin 5y ago
  Not if you can have an army of people work the problem.
  - alex_anglin 5y ago
    Then you too can enjoy The Mythical Man Month.
    - ejb999 5y ago
      in case you don't think you have time to read it all, buy 3 or 4 copies of it to speed things up (there, I summarized it for you :>) ).
- lmilcin 5y ago
  No, they are not. I may want something to be done by the end of next week and at the same time I would like somebody to spend less than 4 hours on the task. Spending entire two weeks of effort would not be acceptable in this case even if the task was delivered on time. An example would be a task that is not on the critical path of the project but still needs to be done. There is a deadline by which it must be delivered or it becomes part of critical path and starts delaying entire project. Assuming the task is performed by one person using the fraction of the time available before the deadline, I would like to be able to use the rest of that time (resource) as either a buffer in the project or for completion of another task. Having the person deliver the task before deadline but after two weeks of continuous work where I assume only 4 hours will be necessary means that the project was deprived of almost two weeks of buffer or there are now some other uncompleted tasks. When we are talking about project management, there is an important, clear distinction between time and cost.
  - lifeisstillgood 5y ago
    >>> An example would be a task that is not on the critical path of the project but still needs to be done. There is a deadline by which it must be delivered or it becomes part of critical path and starts delaying entire project. I know it is picky but, if a task "needs to be done" then it is on the critical path. Just because "build the bridge" is not needed yet, it is on the critical path even before you arrive at the waters edge. I am of the opinion that pretty much every project management function can be replaced in software.
    - lmilcin 5y ago
      > I know it is picky but, if a task "needs to be done" then it is on the critical path. You are wrong. Critical path is the longest path from start to end of the project. It dictates the length of the project. The idea is that if any of these tasks takes longer, entire project takes longer. https://en.wikipedia.org/wiki/Critical_path_method https://en.wikipedia.org/wiki/Critical_path_method Has nothing to do with whether the task is important in itself. But the task on critical path is important from the point of view that it has power to affect entire project. You can still have tasks that need to be done that are not on critical path. This usually means the task can be done in parallel of some other critical task and you have more time available to complete it than the time slot where you can work on it. Imagine you are manager with only two employees. One employee can only write code, the other can only write documentation. Your project requires delivering to the client completed application with documentation. It takes 1 month to complete development and it takes 1 week to complete documentation. Moreover both employees can start immediately because they both know what needs to be done. Which is critical path? If the documentation task slips by one day, the project will not take more time. On the other hand if development slips it will. From that point of view, development is on a critical path. It does not matter which task is more important. If you had bright coder that took one week for development and slow writer that took a month to prepare fantastic documentation, the situation would revers and now it would be documentation that is on the critical path and where you will focus as a project manager to deliver it on time.
AnimalMuppet 5y ago
"If you want it bad, that's how you're going to get it." That is, time pressure kills quality.
- MaysonL 5y ago
  "Soon, cheap, and good. Pick two."
  - commandlinefan 5y ago
    I don't know - what are you going to do when they say "ok, soon and good". Are you going to demand a raise before you deliver?
    - AnimalMuppet 5y ago
      No, you're going to demand that they burn huge amounts of money on several highly skilled contractors.
      - ngc248 5y ago
        On Budget, On Time, On scope ... Pick any two <--- this one is more clearer
  - throwhauser 5y ago
    Pick one, actually.
    - quickthrower2 5y ago
      One is the maximum most of the time, but zero is more than possible
disambiguation 5y ago
Well written and thorough article, but I can't help notice if this isn't just reverse engineering a standard MBA curriculum. > Time pressure is also reported to be the most frequent external cause of unhappiness among developers. > The inverted U-shaped relationship is one typical example of this, which means that time pressure increases efficiency up to a certain point, but after that increases in time pressure cause a decrease in efficiency A good manager knows how to sustain just the right amount of pressure.
- awb 5y ago
  > A good manager knows how to sustain just the right amount of pressure. And they know about Parkinson’s Law: https://en.m.wikipedia.org/wiki/Parkinson%27s_law https://en.m.wikipedia.org/wiki/Parkinson%27s_law > Parkinson's law is the adage that "work expands so as to fill the time available for its completion."
commandlinefan 5y ago
This article implies, but doesn't come right out and say, something that I strongly suspect: that producing an accurate (or even close-to accurate) software project estimate is a time-consuming, unpredictable task. The question nobody seems to be examining is whether or not the cost of doing a large, expensive up-front analysis that might take an arbitrary amount of time is less than the cost of just doing it concurrently with the software development itself.
- nonameiguess 5y ago
  While this is likely true, the problem is W2 labor tends to be compensated on a more or less fixed price per unit of time basis, and to cover that cost, you need to bill clients roughly in proportion to total labor time spent, and most clients aren't going to purchase something for an unknown price to be named later. So you need to estimate to give them a price. From the perspective of the seller, these estimates don't even need to be "accurate" as long as you're making many of them and the errors are symmetrically distributed. There's even math theory behind this called Fermi estimation. The problem is, from the client perspective, they can't average out the errors over many estimates if they're only buying one thing, and from the seller perspective, I don't think our errors are symmetrically distributed. Instead, we consistently underestimate how long something will take. It's not even unique to software. There's a huge housing boom going on where I live, with condo developments all over the place, and not a single one is actually ready to open by the season their sign out front claims they'll be open. Defense acquisition. Highway construction. Everything takes longer than budget forecasts claim.
  - kqr 5y ago
    Part of the problem, especially when it comes construction and defense, is awarding contracts to the lowest bidder with effectively no punishment for exceeding the initial estimate. This pretty much guarantees underestimation, both for statistical and behavioural reasons.
    - halpert 5y ago
      I believe when awarding construction contracts, some of the deciders look at past projects, which could discount underbidding. Often, other engineers are deciding on the contracts, so they also have a general idea of how much it should cost.
e2e4 5y ago
https://dilbert.com/strip/2022-01-02 https://dilbert.com/strip/2022-01-02
- quickthrower2 5y ago
  The project takes as long as it needs, while people tell each other fancy stories.
int0x2e 5y ago
About 10 years ago, I worked on a large, multi team (multi-org really) project that was run using TOC (Theory Of Constraints). Put very very very simply, every team was told to give honest, buffer-less estimates for their primary work areas, and then a global project buffer was added as a factor of the sum of estimates. At any point in time, we worked as if every single thing would line up perfectly, so upcoming milestone targets would often feel crazy or unlikely. But people being eager to please, and mostly unwillingness to be the one team blocking the entire project - meant that folks worked hard to meet some of these targets. It was extremely hard. It was a bit stressful. It felt crazy at times. But it got done in half the time I would have initially expected such a complex project to take. To me, the most impressive thing about it all was that the project leader drove us all quite hard but was super cool - he explained that he wanted everyone to be open and honest about blockers so they could be moved with the full force he was given, and that actually, his way of measuring project progress was the rate at which schedules slipped.
- midrus 5y ago
  Sure, you can do this for a couple weeks. Do this as the normal way of working and you'll end up alone. Burnout is a thing despite we agree or not on the reasons.
- lumost 5y ago
  This sounds like a system where those who don’t buffer get stuck working long hours. Those who add a buffer don’t get stuck on long hours.
  - throwaway2331 5y ago
    This is how it is in finance. Your life is your work, but the ability to own 5 houses is a nice balm.
    - deleted 5y ago
      [deleted]
- code_biologist 5y ago
chrisaycock 5y ago
Treating every project as if it's in emergency mode is a great way to burnout the team and get shoddy work results. Another idiotic practice is multitasking since context switching introduces tons of overhead. And tying it all together, the importance of a project determines its resources. A truly critical task needs 100% focus. If a manager isn't willing to assign a full-time employee to a task, then that project isn't critical.
wombatmobile 5y ago
> The Yerkes–Dodson law refers to an inverted U-shaped relationship between performance and arousal, where performance increases with arousal, but only up to a certain point. The quest to discover and apply "laws" like this courts disaster or disruption because people are neurodiverse. ADHD people respond to deadlines in a completely different way to non ADHD - sometimes inversely, sometimes obversely, and so require different management. Same but different with autistic spectrum people, introverts, extroverts, bipolar, insomnia, sleep apneates, diabetics and long covids. It may be tempting to dismiss neurodiversity as a fringe consideration and just assume most people are "normal", but their circumstances, both in the workplace and beyond it, may complicate Yerkes-Dodson as much as neurodiversity whilst evading detection by managers and systems.
f3rnando 5y ago
Where we are going, we dont need no time. :P https://www.youtube.com/watch?v=26aUmDvDROE https://www.youtube.com/watch?v=26aUmDvDROE
bayesian_horse 5y ago
Yes.
artellectual 5y ago
At the beginning of my current job (I joined as CTO) I had to inherit a lot of misunderstandings that the management team had. They had hired people who knew nothing about building software (before me joining), most likely those people have never really built and shipped anything. Everyone on the management was under the impression that software is easy and everything needs to be delivered on time and if it's delayed we need to throw more money at the problem and that would solve the problem. They would continuously ask for "timelines" and never understood the word "estimate". They knew the definition of the word but could never accept the reality of the definition. I was tasked with building a brand new core banking engine and a kyc engine. I talked to the existing tech team that was there and none had ever done anything complex. Mostly they just knew how to write some basic PHP pages to do some simple tasks like rendering a CSV table for a specific report etc... So they needed to first be trained. To cut a long story short I was the only one on the team to be able to write productive code to build such a complex system. I hired someone who could write code to help train the existing team. However given the complexity of the problem and the tight timelines I had to solve. I basically did the design and implemented about 70% of the code myself while my other teammates were being trained. This also included building a KYC engine along the way. I remembered suffering immensely building the engine and hiring / building the team. I would focus on working with the team during the day and coded at night, since that was the only time I could get un-interrupted hours to code. I worked til 2AM almost every night for 7 days a week for over a year (6 months to get it to launch and then refinements / continuous improvements after). To complete the engine and get it to serve hundreds of thousands of transactions every month. I was never paid for overtime. Today the company stands strong with handsome profits. Trust me. I can tell you, the problem is not the developers / process / agile / kanban / etc..., the problem is management. It's always management. I was able to get management to understand the true scope of the task by stopping working overtime and standing my ground (when the company had recovered and was able to produce a stable income to support itself). The overtime I did hid away all the 'complexities' and the 'debt', and prevented the company from learning. I basically ended up paying for the company's mismanagement / toxic culture with my time. Fortunately a lot of toxic people left the company, a growing company will generally push out the non-performers organically. It took a lot of effort on my part personally to resolve issues plaguing the company, establish an engineering culture, build a strong tech team that can withstand pressure and deliver continuously and can build on top of the framework I provided. The lesson has always been. It's never the software team (at least never the team I manage) whom are not productive or doing their best. It has always been management / company culture / never truly understanding the scope / requirement enough to understand how huge of an undertaking a given project is. I can tell you without a doubt this article pretty much hit the nail on the head. I'm living proof and have the battle scars to prove it. Now I tell management about time estimates as it is. "No fixed timeline, the team estimate, the team builds, it's done when it's done, live with it or fire me." I've fortunately afford myself the position to be able to deliver the truth serum to management and they have no choice but to accept it. It didn't come cheap, I'm still scarred.
anonimamente 5y ago
TL;DR, literally, I didn't bother to read the whole thing. Time pressure will never go away. It only takes common sense to realize that those who want something, i.e. are paying for it, want it as soon as possible. The difference from making physical things is the perception that software is soft: it seems to be infinitely malleable and all it takes is hand waving magic or sheer willpower to manifest it. This is of course the lie that those building software understand, and those that do not, will not.

150 points by chrisaycock 5y ago

Maybe this will be hugely impopular, but in addition to the three causes mentioned, isn't it possible that one cause actually comes from developers simply not performing? If comparing to sports, in a team, is it always the coach's fault? Or is it possible that one player is not performing as expected? Is it then the coach's issue that he/she didn't expect that?

rising-sky 5y ago

> [In this article, time pressure refers to both budget pressure, e.g., eight hours (budget) to do a task, and schedule pressure, e.g., a task that must be done by tomorrow.] Aren't these the same thing? the latter being somewhere around 24hours give or take

flerchin 5y ago
Not if you can have an army of people work the problem.
- alex_anglin 5y ago
  Then you too can enjoy The Mythical Man Month.
  - ejb999 5y ago
    in case you don't think you have time to read it all, buy 3 or 4 copies of it to speed things up (there, I summarized it for you :>) ).
lmilcin 5y ago
No, they are not. I may want something to be done by the end of next week and at the same time I would like somebody to spend less than 4 hours on the task. Spending entire two weeks of effort would not be acceptable in this case even if the task was delivered on time. An example would be a task that is not on the critical path of the project but still needs to be done. There is a deadline by which it must be delivered or it becomes part of critical path and starts delaying entire project. Assuming the task is performed by one person using the fraction of the time available before the deadline, I would like to be able to use the rest of that time (resource) as either a buffer in the project or for completion of another task. Having the person deliver the task before deadline but after two weeks of continuous work where I assume only 4 hours will be necessary means that the project was deprived of almost two weeks of buffer or there are now some other uncompleted tasks. When we are talking about project management, there is an important, clear distinction between time and cost.
- lifeisstillgood 5y ago
  >>> An example would be a task that is not on the critical path of the project but still needs to be done. There is a deadline by which it must be delivered or it becomes part of critical path and starts delaying entire project. I know it is picky but, if a task "needs to be done" then it is on the critical path. Just because "build the bridge" is not needed yet, it is on the critical path even before you arrive at the waters edge. I am of the opinion that pretty much every project management function can be replaced in software.
  - lmilcin 5y ago
    > I know it is picky but, if a task "needs to be done" then it is on the critical path. You are wrong. Critical path is the longest path from start to end of the project. It dictates the length of the project. The idea is that if any of these tasks takes longer, entire project takes longer. https://en.wikipedia.org/wiki/Critical_path_method https://en.wikipedia.org/wiki/Critical_path_method Has nothing to do with whether the task is important in itself. But the task on critical path is important from the point of view that it has power to affect entire project. You can still have tasks that need to be done that are not on critical path. This usually means the task can be done in parallel of some other critical task and you have more time available to complete it than the time slot where you can work on it. Imagine you are manager with only two employees. One employee can only write code, the other can only write documentation. Your project requires delivering to the client completed application with documentation. It takes 1 month to complete development and it takes 1 week to complete documentation. Moreover both employees can start immediately because they both know what needs to be done. Which is critical path? If the documentation task slips by one day, the project will not take more time. On the other hand if development slips it will. From that point of view, development is on a critical path. It does not matter which task is more important. If you had bright coder that took one week for development and slow writer that took a month to prepare fantastic documentation, the situation would revers and now it would be documentation that is on the critical path and where you will focus as a project manager to deliver it on time.

AnimalMuppet 5y ago

"If you want it bad, that's how you're going to get it." That is, time pressure kills quality.

MaysonL 5y ago
"Soon, cheap, and good. Pick two."
- commandlinefan 5y ago
  I don't know - what are you going to do when they say "ok, soon and good". Are you going to demand a raise before you deliver?
  - AnimalMuppet 5y ago
    No, you're going to demand that they burn huge amounts of money on several highly skilled contractors.
    - ngc248 5y ago
      On Budget, On Time, On scope ... Pick any two <--- this one is more clearer
- throwhauser 5y ago
  Pick one, actually.
  - quickthrower2 5y ago
    One is the maximum most of the time, but zero is more than possible

disambiguation 5y ago

Well written and thorough article, but I can't help notice if this isn't just reverse engineering a standard MBA curriculum. > Time pressure is also reported to be the most frequent external cause of unhappiness among developers. > The inverted U-shaped relationship is one typical example of this, which means that time pressure increases efficiency up to a certain point, but after that increases in time pressure cause a decrease in efficiency A good manager knows how to sustain just the right amount of pressure.

awb 5y ago
> A good manager knows how to sustain just the right amount of pressure. And they know about Parkinson’s Law: https://en.m.wikipedia.org/wiki/Parkinson%27s_law https://en.m.wikipedia.org/wiki/Parkinson%27s_law > Parkinson's law is the adage that "work expands so as to fill the time available for its completion."

commandlinefan 5y ago

This article implies, but doesn't come right out and say, something that I strongly suspect: that producing an accurate (or even close-to accurate) software project estimate is a time-consuming, unpredictable task. The question nobody seems to be examining is whether or not the cost of doing a large, expensive up-front analysis that might take an arbitrary amount of time is less than the cost of just doing it concurrently with the software development itself.

nonameiguess 5y ago
While this is likely true, the problem is W2 labor tends to be compensated on a more or less fixed price per unit of time basis, and to cover that cost, you need to bill clients roughly in proportion to total labor time spent, and most clients aren't going to purchase something for an unknown price to be named later. So you need to estimate to give them a price. From the perspective of the seller, these estimates don't even need to be "accurate" as long as you're making many of them and the errors are symmetrically distributed. There's even math theory behind this called Fermi estimation. The problem is, from the client perspective, they can't average out the errors over many estimates if they're only buying one thing, and from the seller perspective, I don't think our errors are symmetrically distributed. Instead, we consistently underestimate how long something will take. It's not even unique to software. There's a huge housing boom going on where I live, with condo developments all over the place, and not a single one is actually ready to open by the season their sign out front claims they'll be open. Defense acquisition. Highway construction. Everything takes longer than budget forecasts claim.
- kqr 5y ago
  Part of the problem, especially when it comes construction and defense, is awarding contracts to the lowest bidder with effectively no punishment for exceeding the initial estimate. This pretty much guarantees underestimation, both for statistical and behavioural reasons.
  - halpert 5y ago
    I believe when awarding construction contracts, some of the deciders look at past projects, which could discount underbidding. Often, other engineers are deciding on the contracts, so they also have a general idea of how much it should cost.

e2e4 5y ago

https://dilbert.com/strip/2022-01-02 https://dilbert.com/strip/2022-01-02

quickthrower2 5y ago
The project takes as long as it needs, while people tell each other fancy stories.

int0x2e 5y ago

About 10 years ago, I worked on a large, multi team (multi-org really) project that was run using TOC (Theory Of Constraints). Put very very very simply, every team was told to give honest, buffer-less estimates for their primary work areas, and then a global project buffer was added as a factor of the sum of estimates. At any point in time, we worked as if every single thing would line up perfectly, so upcoming milestone targets would often feel crazy or unlikely. But people being eager to please, and mostly unwillingness to be the one team blocking the entire project - meant that folks worked hard to meet some of these targets. It was extremely hard. It was a bit stressful. It felt crazy at times. But it got done in half the time I would have initially expected such a complex project to take. To me, the most impressive thing about it all was that the project leader drove us all quite hard but was super cool - he explained that he wanted everyone to be open and honest about blockers so they could be moved with the full force he was given, and that actually, his way of measuring project progress was the rate at which schedules slipped.

midrus 5y ago
Sure, you can do this for a couple weeks. Do this as the normal way of working and you'll end up alone. Burnout is a thing despite we agree or not on the reasons.
lumost 5y ago
This sounds like a system where those who don’t buffer get stuck working long hours. Those who add a buffer don’t get stuck on long hours.
- throwaway2331 5y ago
  This is how it is in finance. Your life is your work, but the ability to own 5 houses is a nice balm.
  - deleted 5y ago
    [deleted]
code_biologist 5y ago

chrisaycock 5y ago

Treating every project as if it's in emergency mode is a great way to burnout the team and get shoddy work results. Another idiotic practice is multitasking since context switching introduces tons of overhead. And tying it all together, the importance of a project determines its resources. A truly critical task needs 100% focus. If a manager isn't willing to assign a full-time employee to a task, then that project isn't critical.

wombatmobile 5y ago

> The Yerkes–Dodson law refers to an inverted U-shaped relationship between performance and arousal, where performance increases with arousal, but only up to a certain point. The quest to discover and apply "laws" like this courts disaster or disruption because people are neurodiverse. ADHD people respond to deadlines in a completely different way to non ADHD - sometimes inversely, sometimes obversely, and so require different management. Same but different with autistic spectrum people, introverts, extroverts, bipolar, insomnia, sleep apneates, diabetics and long covids. It may be tempting to dismiss neurodiversity as a fringe consideration and just assume most people are "normal", but their circumstances, both in the workplace and beyond it, may complicate Yerkes-Dodson as much as neurodiversity whilst evading detection by managers and systems.

f3rnando 5y ago

Where we are going, we dont need no time. :P https://www.youtube.com/watch?v=26aUmDvDROE https://www.youtube.com/watch?v=26aUmDvDROE

bayesian_horse 5y ago

Yes.

artellectual 5y ago

At the beginning of my current job (I joined as CTO) I had to inherit a lot of misunderstandings that the management team had. They had hired people who knew nothing about building software (before me joining), most likely those people have never really built and shipped anything. Everyone on the management was under the impression that software is easy and everything needs to be delivered on time and if it's delayed we need to throw more money at the problem and that would solve the problem. They would continuously ask for "timelines" and never understood the word "estimate". They knew the definition of the word but could never accept the reality of the definition. I was tasked with building a brand new core banking engine and a kyc engine. I talked to the existing tech team that was there and none had ever done anything complex. Mostly they just knew how to write some basic PHP pages to do some simple tasks like rendering a CSV table for a specific report etc... So they needed to first be trained. To cut a long story short I was the only one on the team to be able to write productive code to build such a complex system. I hired someone who could write code to help train the existing team. However given the complexity of the problem and the tight timelines I had to solve. I basically did the design and implemented about 70% of the code myself while my other teammates were being trained. This also included building a KYC engine along the way. I remembered suffering immensely building the engine and hiring / building the team. I would focus on working with the team during the day and coded at night, since that was the only time I could get un-interrupted hours to code. I worked til 2AM almost every night for 7 days a week for over a year (6 months to get it to launch and then refinements / continuous improvements after). To complete the engine and get it to serve hundreds of thousands of transactions every month. I was never paid for overtime. Today the company stands strong with handsome profits. Trust me. I can tell you, the problem is not the developers / process / agile / kanban / etc..., the problem is management. It's always management. I was able to get management to understand the true scope of the task by stopping working overtime and standing my ground (when the company had recovered and was able to produce a stable income to support itself). The overtime I did hid away all the 'complexities' and the 'debt', and prevented the company from learning. I basically ended up paying for the company's mismanagement / toxic culture with my time. Fortunately a lot of toxic people left the company, a growing company will generally push out the non-performers organically. It took a lot of effort on my part personally to resolve issues plaguing the company, establish an engineering culture, build a strong tech team that can withstand pressure and deliver continuously and can build on top of the framework I provided. The lesson has always been. It's never the software team (at least never the team I manage) whom are not productive or doing their best. It has always been management / company culture / never truly understanding the scope / requirement enough to understand how huge of an undertaking a given project is. I can tell you without a doubt this article pretty much hit the nail on the head. I'm living proof and have the battle scars to prove it. Now I tell management about time estimates as it is. "No fixed timeline, the team estimate, the team builds, it's done when it's done, live with it or fire me." I've fortunately afford myself the position to be able to deliver the truth serum to management and they have no choice but to accept it. It didn't come cheap, I'm still scarred.

anonimamente 5y ago

TL;DR, literally, I didn't bother to read the whole thing. Time pressure will never go away. It only takes common sense to realize that those who want something, i.e. are paying for it, want it as soon as possible. The difference from making physical things is the perception that software is soft: it seems to be infinitely malleable and all it takes is hand waving magic or sheer willpower to manifest it. This is of course the lie that those building software understand, and those that do not, will not.