33 ms·

Planning and estimating large-scale software projects

328 points by kalleth 5y ago

kalleth 5y ago
Author here. I've been lucky enough in my career to hold some senior positions, and I thought I'd give a step-by-step on an approach I took with an "enterprise-scale" software project, and how I stole some techniques from university project management courses to meet with some success. Happy to answer any questions :)
- mzarate06 5y ago
  I'm curious that you mention project management courses from university. How much benefit have you found them to provide in practice? Asking b/c I've taken two courses in dev process or project management in my academic career, and neither provided substantial value or benefit to how I've lead projects professionally.
  - kalleth 5y ago
    In the early stages of my career, or in startup life? Absolutely not. Very little relevance; XP/SCRUM were both covered together in a single 50 minute lecture, the rest of the PM aspects were tackling paperwork-generation methodologies like the "Rational Unified Process" and "Dynamic Systems Development Model", both of which I feel like would be _hell_ if I actually had to work within. However, there were techniques (like critical path analysis) that as I've got more senior, started working at larger companies, and started stepping down the senior leadership path, I've started to see some applicability to. Not direct applications - they still need taking with a massive pinch of salt, and modifying for modern learnings in industry, but they do start to provide some value, even if it's just learning what the grey-hairs in the exec are used to seeing :)
    - mtippett 5y ago
      Fully agree. There are huge nuggets of wisdom in old-school engineering. For example PERT is something that most people don't know about. But it's simple to apply in the real world. Get three estimates - optimistic, pessimistic and realistic. double the realistic, add the optimistic and pessimistic and divide by four. It's basically a centerweighted estimate - but it forces you to think about the the corner-cases and what could go wrong. Lots of other old school stuff that we "just turning grey" people need to translate for the new kids.
- pphysch 5y ago
  What tools and methods do you use for creating, sharing, and modifying project charts?
  - kalleth 5y ago
    Depends how much of a perfectionist I'm feeling. For the initial development, a sharpie, index cards, and a whiteboard wall - or Lucidchart, because it's basically an in-browser whiteboard/drawing tool. Once the project is ongoing and you'll need to account for changes, I've either done it manually (which takes an age) or handed it off to PM's to oversee using either MS Project, airtable with a custom-authored set of actions/etc, or PrimaVera.
- ksec 5y ago
  Do you think there is a different in how US and UK tech companies in project planning?
  - kalleth 5y ago
    I've never worked for a US tech company, so I wouldn't know, sorry!
- justin_oaks 5y ago
  How are deadline dates assigned? Is the deadline exactly the same as the estimated completion date? Realistic estimates aren't padded, but they still have significant probability of being inaccurate. After all, they're estimates, not information from the future transmitted to the past.
  - kalleth 5y ago
    This is where the ideal meets the annoying reality of The Enterprise (tm). I can't talk in too much detail, but in general, the deadline date was fixed through commercial contracts signed at a high enough level that engineering didn't have sight of them. The concept and commercial case was sound, but the implementation hadn't been worked out yet, when a date was set. My strong preference would be for estimation to come first, of course, before a deadline is picked (and even then, only picked if it is really a necessary deadline), which is then based on reality, and also include some slack for unintended discoveries.
    - hu3 5y ago
      I had this "Enterprise sets the deadlines" experience just yesterday. I delivered a spreadsheet with detailed information of how long it would take to develop each feature of the new module they wanted. It totalled around 3 months of development. Fancy suit folks told me 3 months wouldn't do because sales promised it would be ready in 2 months. Then I was asked where could shortcuts be taken. In the end we had to cut features that will certainly upset our client given their initial expectation.
- PUSH_AX 5y ago
  Our paths crossed on the engineering team of a certain letterbox flower company, for a short time at least. Good to see you doing well Tom, nice article.
  - kalleth 5y ago
    Thanks! And I couldn't possibly comment on which company that could be... Hope you're doing well too!
- sarks_nz 5y ago
  Thanks for the article. Curious how you ensure dev outputs match up with other (external) deadlines, for example, the sales/marketing teams firing up a promotion of your new functionality. How do you incentivise devs to hit those timeframes?
hnbad 5y ago
I have to admit I popped a monocle when I saw the estimate for what is described as a rudimentary ecommerce website without even basic features like authentication and payment come down to 43 team-weeks, or in other words 12,040 team-hours. Even at one person per team being billed as $20/hr this works out to almost a quarter million dollars. I understand the numbers are given as an example but I think the problem is that the scale of the task hardly justifies the complexity of the planning and using "teams" and "weeks" as sizes instead of "developers" and "hours/days".
- kalleth 5y ago
  I think I would have popped a monocle too, had that been the real example and estimate from a team! I tried to think up an accessible example that didn't require too much context on the part of the reader, so obviously, as you correctly point out, all the numbers are made up, and I'm trying to use it solely to demonstrate the workflow :)
  - commandlinefan 5y ago
    > all the numbers are made up Well... have you actually applied this process successfully? If so, wouldn't you have some actual numbers to point to from a past project? Names and details changed a bit to protect the innocent, of course.
    - kalleth 5y ago
      I have, yes - or I'd feel a bit like a charlatan writing about it! The problem with the real world examples is the business domain, which was hyper complex and the specific "pieces" of work I described wouldn't have been easy to grasp for most not familiar with the esoteric side of fintech that the project took place in. So I went with a simpler, albeit contrived and more accessible example.
      - commandlinefan 5y ago
        Were they monocle-popping results?
agentultra 5y ago
This is a great breakdown of your process. I've seen many quite like it and have also been asked to make these kinds of estimates myself on projects for many of the same reasons: someone made a promise to another person, signed a contract, planned a release date for something they just made up, etc. What I find frustrating about the whole situation is that no matter what process you use for making these estimates you have a roughly 30-some-odd percent chance of being right. It almost never has anything to do with the process you used when it does go well. If it did estimating software projects would be trivial, wouldn't it? Everyone would use this process and we wouldn't have 60-some-odd-percent of large enterprise software projects going over time and budget. In reality people have used this very process, I'm sure, and have been in the 60-some-odd-precent. People have been studying this phenomenon since before I was a nerdy kid hacking on my Amiga. Having a roadmap or a plan to get from A to B is good. It will need to be readjusted as you explore the problem space and navigate the waters so to speak. But the only real guarantee we can make as engineers is that we'll make progress in my experience. I'm only giving really rough estimates in the beginning and those estimates improve as we get closer to our end goal. I only start talking about actual release dates when we get close to being finished and are mostly polishing out the rough corners and have already done a few iterations internally. If someone makes a promise they can't keep or have no business making -- in my books -- that's their mistake and they've made it a problem for everyone else.
- RandomLensman 5y ago
  Estimate is but one part. The other is planning for obstacles and changes, i.e. have escalation paths and responsibilities in place: Clear procedures to remove impediments, decision makers in the loop to ok scope/feature changes, resourcing agreed upfront, user engagement for testing locked-down, senior leadership aligned and kept informed regularly. Someone running the administrative side of the project, keeping people on target, etc. (could be a double hat, of course). Sounds all rather "menial", but high degrees of organization really make a difference in delivery on larger projects.
- AnimalMuppet 5y ago
  Yeah. I'm a big fan of making the person who made the promise fix the mess. You dreamed up something and promised it to the customer? You get to go back to the customer and eat the crow. Maybe that will teach you not to do it next time. Anything else is just enabling (or even rewarding) bad behavior. If you do, expect to get more of it. Note well: I have never been a manager, especially not an upper-level manager over both sales and engineering. I don't know how well my recommendation will fly in the real world. (Hey, I guess that makes me the guy who just sold something without knowing if it can work...)
ColinHayhurst 5y ago
> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on the really hard problems. Of course, it meant getting to know those developers and a prior (which we set at x2, for new starters). Individuals were remarkably consistent in terms of their actual performance; so a x1 developer would almost always be a x1; a x1.5 would almost always be x1.5.
- ragebol 5y ago
  How much learning time did you typically need to arrive at that multiplier?
  - ColinHayhurst 5y ago
    At the end of two completed projects from each new starter
- LegitShady 5y ago
  The multiplier was the same way I dealt with estimating time budget for homework in university. I would estimate the amount of time I thought it should take and multiply by 3 and it would usually take between 2-3x at the end. It's a little harder when the work is more nebulous and I don't have to do it for others but it's all based on personal estimations of productivity as well as understanding of the problem. It's not easy to do.
- killjoywashere 5y ago
  A Jira plug-in to use some sort of Bayesian system to adjust developer priors at the task and sprint level. That would be amazing.
  - msluyter 5y ago
    Yeah I've always felt like the missing part of the loop (in the context of pointing stories) is looking at point estimates and then tracking how accurate they were, _by developer_. E.g., I throw 2 points, but it turns out the story takes something along the lines of 13 points, then my estimation was pretty low and my personal "estimation multiplier" is increased. Subsequently, actual point estimations could take the estimation multiplier into effect. Of course, this is mostly a toy idea that's fun to think about, but seems pretty untenable given that it requires keeping track of time, which most developers hate. That, and there'd no doubt be incentive to game the system by padding estimates or other shenanigans.
antondd 5y ago
This brings back memories from the days of my early career (an ex-PM survivor here). I would be curious to see some data, even anecdotal, on the success of this approach. Here's some interesting statistics from the industry (not specific to software, but you can extrapolate): http://apepm.co.uk/project-management-statistics/ http://apepm.co.uk/project-management-statistics/ In my view, traditional software project management is ineffective. I would put it somewhere between the Myers-Briggs personality test and modern day astrology.
- kalleth 5y ago
  Insightful, thank you! The entire project management industry doesn't have that great a "hit rate" -- consider the budget overruns for the last few Olympics, or for Crossrail. I'm just not sure why software projects are "special" -- if you can avoid it being a project and instead make it ongoing OpEx like, for example, GDS managed for the UK in 2016, then great, you've sidestepped that, but until the entire PM industry discovers how to improve overall project management techniques, I don't see why we'd consider our industry "above" them.
  - some_chap 5y ago
    I'd say that software is "special" because it's ephemeral, meaning there's incredibly few limitations on the possibilities (or changes to requirements mid-project) when compared to projects involving physical items. It /can/ be managed like a physical engineering project, but the cost and time ramp up so severely that it's not practical for most situations.
  - tonyedgecombe 5y ago
    You can't really do that with the Olympics though as there is a hard deadline. Once you have a fixed date then either quality or cost are going to have to give.
  - marcus_holmes 5y ago
    > instead make it ongoing OpEx Every non-tech startup founder who's approached me with "how long will it take to build an MVP for my startup idea?", I've answered with this. Development is an ongoing cost, a process, not a once-off capex cost. I recommend to them going the other way. Start with "how much can you afford to pay a dev team sustainably?" then work out how many devs that works out to, then work out how long your MVP will take to build based on their estimates (and estimates are not deadlines). Not quite the same as #no_estimates (which I also try to argue for whenever possible), but close.
christophergs 5y ago
Great post. A key point you don't bring up is the aftermath, even if you do deliver. Especially in non-tech companies there still remains the tendency to view these projects as "done" after the end of the project/MVP etc., with no understanding that sites need ongoing maintenance and improvements. And that this work is still considerable.
- gonzo41 5y ago
  That reminds me of that scene in the series Chernobyl where the main scientist briefs everyone on the cleanup effort and ends with something like "The first battle is won and now begins the long war" with everyone being suitably cold in their response. I get the same response sometimes when I talk about the long tail of maintenance at work.
  - christophergs 5y ago
    Haha, love that show. There's definitely a parody in there somewhere > Now that I know what software estimation is, I no longer need you.
- noptd 5y ago
  Agreed, especially for MVPs and "phase 1" projects. At my current company, it's reached a point where we flat out reject product proposals for features or changes that would need to be hacked together for an MVP without a time commitment from all necessary stakeholders on how it will properly be implemented for phase two (iff phase one is a success). It's amazing how quickly "critical" features become irrelevant to product when they understand even half the amount of work required to properly implement them.
  - epalm 5y ago
    Knowing when to say ‘No’ is an important (and probably underrated) skill.
deleted 5y ago
[deleted]
ngrilly 5y ago
> It also tells us how many team-weeks this fictional, idealised project would require [...] by adding all the estimates together. I would be wary with just "adding all the estimates together". That's because we tend to estimate the median or the mode of the task duration, and not the average. Means can be added together, but not medians.
- marcosdumay 5y ago
  > Means can be added together Is the error distribution of task size estimations normally distributed? Because I do really expect it to have a fat tail, and if it does, you can't add means either.
  - ngrilly 5y ago
    I think most of us in software engineering assume the probability distribution has a fat tail. I've seen some authors name this the "blowup factor". For instance, your most likely estimation is 10 days, best case is 5 days, and worst case is 30 days. I think adding means is still meaningful (see central limit theorem and law of large numbers).
    - marcosdumay 5y ago
      It's exactly the central limit theorem that breaks for fat tailed distribution. Some fat tail distributions also break the law of large numbers, but I don't think task size estimation is this flawed.
  - jdlshore 5y ago
    There’s a variety of analyses out there and they very consistently show a log-normal distribution for release predictions. I’ve analyzed Star Citizen’s publicly available data and found the same for their task estimates. It’s very reliable. You do see truncated log-normals, though, when the estimates are padded.
dls2016 5y ago
Project management is hard. I wrote and won an SBIR award this year. A much different scale than the article, but I spent a lot of time writing the plan and budget and sourcing components and estimating software tasks. Two months in and a big chunk of that is out the window… haha. Finding connectors and other components has been a big source of pain, especially if purchasing in small quantities. Another example: I bought a consumable product which then immediately became unavailable… so do I try to make do with what I have on hand or make the decision to switch to a replacement? Stressful, but I try to have fun. And extremely satisfying when things work out!
28304283409234 5y ago
I've started estimations by their true name: Assumptions.
- heresie-dabord 5y ago
  For those who are new to software development, the following terminology may prove helpful: Software Project Estimumption (or Assumptimation, the professional community is split) Software Requirements GatherWhims Software Requirements Analysthetics Unit Test Coveroverage
henning 5y ago
Bullshit. You can make guesses about the future, but since that is going to inevitably change, it's a guess and can never be anything but a guess. At most companies, the scope is going to be radically expanded and looking too far in the future is a complete waste of time. Other people in other fields are held accountable for deadlines because their work does not completely change and is not severely under-specified. If it is, then they are also just guessing.
- crazy1van 5y ago
  > it's a guess and can never be anything but a guess Of course it's a guess. The question is how to make more accurate guesses.
  - dragonwriter 5y ago
    > The question is how to make more accurate guesses. I think the question should be about how to maximize the net return on time dedicated to development, which may mean spending less time on (and investing less reliance on) estimates rather than expending unbounded effort improving the quality of estimates.
    - satyrnein 5y ago
      What's the value to Sales/Marketing/etc of knowing what's coming when? How do you estimate that? It's estimates all the way down!
- NikolaNovak 5y ago
  I think author addressed that explicitly as valid and true software engineer perspective, right at beginning of the article; and explained that such software engineers, and products they build, then usually slot into larger company's ecosystem (as by strict math, most IT team members will work at a large company as opposed to a startup), full of people and leaders and departments and teams and project who have plans and deadlines and dependencies, which are also valid and true. I've had either luck or misfortune to "flip a switch" from two decades of being a techie/architect, to being a mid-manager, on basically a specific date as opposed to over the years, due to project's needs; and it's like that B&W picture of two faces and one vase in middle. Both perspectives are true, even if opposing and contradictory. Good business lead will understand software engineer's perspective - even if it's not their primary view of the world, they can squint and catch a glimpse as needed. Likewise for good software leads. Obstinate unwillingness to see or ascribe merit to other perspectives puts a ceiling on everybody's progress.
pphysch 5y ago
There are arguably two major phases here, and its only the second one that most folks here find controversial. * Steps 0-2: determining "what" the project "is" (design, architecture, ontology) * Steps 3-6: procuring and allocating resources to complete it (economics, management, politics) It is tempting to decouple the two phases, and as a technologist focusing solely on architecture while leaving the economics up to leadership. However, social factors (the real people involved with the project) are an integral part of actually getting anything done, so I agree with the author's premise that the whole process should be viewed holistically (and ideally run by one technologist).
- kalleth 5y ago
  Well phrased, thank you! I didn't think of it in this way, but yes, that kind of phasing makes sense.
dpweb 5y ago
I found it really fluctuates based on the team members. We definitely had 3x 4x differences in productivity between the worst and best on the team. Our estimates had to be good as it determined the size of the sales deal. Most everyone on the team was there for years, so we knew each other well. We could give a solid estimate but with a new team member is more challenging.
mumblemumble 5y ago
> you should expect to be held accountable if your estimates are too far off the mark because you failed to do your due diligence when coming up with them. In a perfect world, I agree. In the real world, which has a remarkable knack for failing to live up to expectations, what I find is that companies are rarely willing to allow the development team adequate time to do their due diligence. Answers, in and of themselves, are cheap. I can give you those all day. If you want to be able to hold me accountable for their accuracy, though, then you need to be looking at my correct answer rate sheet. For me, the magic of #noestimates is the magic of open, honest cynicism. If my realistic options are silence and blowing smoke up my boss's ass, I'd really prefer it if they would allow me to choose silence. That way we can both keep our dignity.
lifeisstillgood 5y ago
- Projects only become official and tracked once someone has hacked together enough of a prototype to prove it works. - at this point all project management is pretending it takes 100 managers to land something one girl / guy got flying. - stop project managing, stop estimating, and just start treating companies as VC firms. Hire good devs, make them care about your mission, invest in those that take off. Don't take the control away from the original devs
sarks_nz 5y ago
How to make developers want to hit their deadlines with quality? Startup land. There was no feedback loop that rewarded developers to meet the estimates. Stock options weren't an option, and I didn't want them to do a sloppy job just to hit the 'deadline'.
- ed312 5y ago
  Bonus and/or equity grants via review feedback are a blunt but effective tool. You could probably reduce this to "My team is not highly engaged/motivated, and I think it would better meet my company's needs if they were. How can I improve that?"
- skohan 5y ago
  Honestly it's not developers' job to hit their estimates. If we're talking about a long-term estimate, as in "this project will be finished in 6 months", it's your job in management to find a way to do this. You've got to break the goal down into achievable sub-goals, and monitor progress along the way. Long term estimates will be wrong. Software projects generally take longer than expected, so it's up to you as a manager to anticipate this and communicate to stakeholders with the correct degree of uncertainty. If it's an external deadline which must be met, firstly you should engineer enough extra time into the timeline to handle inevitable delays. And if at any point you feel like the timeline is unachievable, it's up to you to renegotiate with stakeholders, or adjust the scope to make it achievable. And if you have the feeling your team is slacking off and not getting work done, honestly this sounds like a lack of leadership skills. It's up to you to have the kind of relationship with your developers so that they are motivated to meet the team's collective goals and take responsibility. That's basically all that being a manager is.
  - sarks_nz 5y ago
    Thanks for that. I'm talking about the difference between hitting the deadline on Monday versus Friday. How to incentivise that? ie, do half an hour more work for a week, or skip the table-tennis when someone asks, etc. As a developer, I was into making sure I hit my goals, and at work to work. As a manager I do struggle with how to emphasise that ownership of product, quality, time. Why should developers care about hitting Monday with effort, instead of coasting to Friday?
galaxyLogic 5y ago
Doesn't this assume you have the "spec" before you start implementing it? But then how do you estimate how long it will take to come up with that spec? If you have a very good detailed spec you have already done much of to work to make the implementation easy. If the spec is high-level and "fuzzy" it leaves the work of "resolving the spec" to the programmer. So trying to estimate the time it takes to code a system depends on the quality of the spec, and therefore is difficult if there is no standard on how detailed the spec is to be.
- kalleth 5y ago
  If you don't have the spec, then you likely don't have a deadline, or a "project", really -- and something like this would be the wrong choice for an approach to follow. I'd say leaving the overall roadmap (which is all this produces, at the end of the day, if you ignore the estimation piece) fuzzy and allowing the team to work that out with users/subject matter experts is the right approach, imo.
tikhonj 5y ago
Seems like there's a structural issue at play here: the team is accountable for building a specific set of features at a specific time. High-level functionality is fixed and the timeline is fixed. If the estimate is off or something unexpected comes up, what's going to give? Answer: everything else. Anything that the planners and executives didn't consider ahead of time. Other aspects of the work: code quality, product design, accessibility, performance, robustness, edge-case-handling... Team learning, culture and satisfaction. At the limit, the team will compromise on anything that you don't need to claim a feature is "done" with a reasonably straight face. It's simply not a good system even when the estimates work out reasonably well—and, empirically, they usually don't. To be fair, this isn't entirely the fault of estimates in general or this estimation approach in particular; I believe those contribute, but it's primarily a reflection of how the company and culture are structured. If you're already in a system like that and you can't change it, trying to do estimates well might be the best option forwards, but only because you're already in a corner.
iamgopal 5y ago
I would divide the projected total task in to 5 equal part and again each part in 5 equal part, and just worry about first of the first, after every part completion we can revise the target
igouy 5y ago
https://rightingsoftware.org/ https://rightingsoftware.org/ "Based on first principles in software engineering and a comprehensive set of matching tools and techniques, Löwy’s methodology integrates system design and project design. First, he describes the primary area where many software architects fail and shows how to decompose a system into smaller building blocks or services, based on volatility. Next, he shows how to flow an effective project design from the system design; how to accurately calculate the project duration, cost, and risk; and how to devise multiple execution options."
meesles 5y ago
The more projects like these I plan and execute, the more I find the need to account for external parties affecting timelines. In the case of the author's payment system, say they go with a vendor. At that scale, there may be weeks of contract negotiations for fees, rates, minimums, etc. What if some piece of documentation is flawed and you need to have a back-and-forth with their support? What if they need time to onboard you into their systems? It makes me appreciate Apple's supply chain mastery that allows them to deliver exactly on time because they own their whole process inside and out and demand similar rigor from any vendors that supply them. If we could imitate that in software, we could eliminate a huge source of uncertainty in many projects.
- beckingz 5y ago
  Apple also cuts scope and delays committing to a release until they know they can deliver...
Aeolun 5y ago
> Engineering does not work in a vacuum, and there are commercial contracts that will be signed, at a high level, without your involvement. Deals that will have been agreed before you joined the organization. Indeed. And they’ll have been built on lies, damned lies and wishes.
grayclhn 5y ago
Call me crazy, but if we're going to call these goals, "estimates," they should actually be based on real data. Otherwise we're just making up numbers. If the relevant data aren't tracked or stored anywhere, that kind of tells you how serious the org is about making accurate estimates.
mtippett 5y ago
I like how he says little-a agile - what most teams use. big-a Agile I reserve for specific Agile methodologies. Most companies aren't willing to invest in the rigor and effort for big-A Agile, and so you end up with this weird-ass hybrid.
dave1999x 5y ago
No seriously, no. Dive straight into debunking #noestimates but don't get to the fact that no one ever really knows what they want.
jussy 5y ago
From what I've seen the biggest problem is that the estimates are done as part of a sales cycle or to get approval to spend money of some sort. Which is a different motivator to actually getting it done.

328 points by kalleth 5y ago

Author here. I've been lucky enough in my career to hold some senior positions, and I thought I'd give a step-by-step on an approach I took with an "enterprise-scale" software project, and how I stole some techniques from university project management courses to meet with some success. Happy to answer any questions :)

mzarate06 5y ago
I'm curious that you mention project management courses from university. How much benefit have you found them to provide in practice? Asking b/c I've taken two courses in dev process or project management in my academic career, and neither provided substantial value or benefit to how I've lead projects professionally.
- kalleth 5y ago
  In the early stages of my career, or in startup life? Absolutely not. Very little relevance; XP/SCRUM were both covered together in a single 50 minute lecture, the rest of the PM aspects were tackling paperwork-generation methodologies like the "Rational Unified Process" and "Dynamic Systems Development Model", both of which I feel like would be _hell_ if I actually had to work within. However, there were techniques (like critical path analysis) that as I've got more senior, started working at larger companies, and started stepping down the senior leadership path, I've started to see some applicability to. Not direct applications - they still need taking with a massive pinch of salt, and modifying for modern learnings in industry, but they do start to provide some value, even if it's just learning what the grey-hairs in the exec are used to seeing :)
  - mtippett 5y ago
    Fully agree. There are huge nuggets of wisdom in old-school engineering. For example PERT is something that most people don't know about. But it's simple to apply in the real world. Get three estimates - optimistic, pessimistic and realistic. double the realistic, add the optimistic and pessimistic and divide by four. It's basically a centerweighted estimate - but it forces you to think about the the corner-cases and what could go wrong. Lots of other old school stuff that we "just turning grey" people need to translate for the new kids.
pphysch 5y ago
What tools and methods do you use for creating, sharing, and modifying project charts?
- kalleth 5y ago
  Depends how much of a perfectionist I'm feeling. For the initial development, a sharpie, index cards, and a whiteboard wall - or Lucidchart, because it's basically an in-browser whiteboard/drawing tool. Once the project is ongoing and you'll need to account for changes, I've either done it manually (which takes an age) or handed it off to PM's to oversee using either MS Project, airtable with a custom-authored set of actions/etc, or PrimaVera.
ksec 5y ago
Do you think there is a different in how US and UK tech companies in project planning?
- kalleth 5y ago
  I've never worked for a US tech company, so I wouldn't know, sorry!
justin_oaks 5y ago
How are deadline dates assigned? Is the deadline exactly the same as the estimated completion date? Realistic estimates aren't padded, but they still have significant probability of being inaccurate. After all, they're estimates, not information from the future transmitted to the past.
- kalleth 5y ago
  This is where the ideal meets the annoying reality of The Enterprise (tm). I can't talk in too much detail, but in general, the deadline date was fixed through commercial contracts signed at a high enough level that engineering didn't have sight of them. The concept and commercial case was sound, but the implementation hadn't been worked out yet, when a date was set. My strong preference would be for estimation to come first, of course, before a deadline is picked (and even then, only picked if it is really a necessary deadline), which is then based on reality, and also include some slack for unintended discoveries.
  - hu3 5y ago
    I had this "Enterprise sets the deadlines" experience just yesterday. I delivered a spreadsheet with detailed information of how long it would take to develop each feature of the new module they wanted. It totalled around 3 months of development. Fancy suit folks told me 3 months wouldn't do because sales promised it would be ready in 2 months. Then I was asked where could shortcuts be taken. In the end we had to cut features that will certainly upset our client given their initial expectation.
PUSH_AX 5y ago
Our paths crossed on the engineering team of a certain letterbox flower company, for a short time at least. Good to see you doing well Tom, nice article.
- kalleth 5y ago
  Thanks! And I couldn't possibly comment on which company that could be... Hope you're doing well too!
sarks_nz 5y ago
Thanks for the article. Curious how you ensure dev outputs match up with other (external) deadlines, for example, the sales/marketing teams firing up a promotion of your new functionality. How do you incentivise devs to hit those timeframes?

hnbad 5y ago

I have to admit I popped a monocle when I saw the estimate for what is described as a rudimentary ecommerce website without even basic features like authentication and payment come down to 43 team-weeks, or in other words 12,040 team-hours. Even at one person per team being billed as $20/hr this works out to almost a quarter million dollars. I understand the numbers are given as an example but I think the problem is that the scale of the task hardly justifies the complexity of the planning and using "teams" and "weeks" as sizes instead of "developers" and "hours/days".

kalleth 5y ago
I think I would have popped a monocle too, had that been the real example and estimate from a team! I tried to think up an accessible example that didn't require too much context on the part of the reader, so obviously, as you correctly point out, all the numbers are made up, and I'm trying to use it solely to demonstrate the workflow :)
- commandlinefan 5y ago
  > all the numbers are made up Well... have you actually applied this process successfully? If so, wouldn't you have some actual numbers to point to from a past project? Names and details changed a bit to protect the innocent, of course.
  - kalleth 5y ago
    I have, yes - or I'd feel a bit like a charlatan writing about it! The problem with the real world examples is the business domain, which was hyper complex and the specific "pieces" of work I described wouldn't have been easy to grasp for most not familiar with the esoteric side of fintech that the project took place in. So I went with a simpler, albeit contrived and more accessible example.
    - commandlinefan 5y ago
      Were they monocle-popping results?

agentultra 5y ago

This is a great breakdown of your process. I've seen many quite like it and have also been asked to make these kinds of estimates myself on projects for many of the same reasons: someone made a promise to another person, signed a contract, planned a release date for something they just made up, etc. What I find frustrating about the whole situation is that no matter what process you use for making these estimates you have a roughly 30-some-odd percent chance of being right. It almost never has anything to do with the process you used when it does go well. If it did estimating software projects would be trivial, wouldn't it? Everyone would use this process and we wouldn't have 60-some-odd-percent of large enterprise software projects going over time and budget. In reality people have used this very process, I'm sure, and have been in the 60-some-odd-precent. People have been studying this phenomenon since before I was a nerdy kid hacking on my Amiga. Having a roadmap or a plan to get from A to B is good. It will need to be readjusted as you explore the problem space and navigate the waters so to speak. But the only real guarantee we can make as engineers is that we'll make progress in my experience. I'm only giving really rough estimates in the beginning and those estimates improve as we get closer to our end goal. I only start talking about actual release dates when we get close to being finished and are mostly polishing out the rough corners and have already done a few iterations internally. If someone makes a promise they can't keep or have no business making -- in my books -- that's their mistake and they've made it a problem for everyone else.

RandomLensman 5y ago
Estimate is but one part. The other is planning for obstacles and changes, i.e. have escalation paths and responsibilities in place: Clear procedures to remove impediments, decision makers in the loop to ok scope/feature changes, resourcing agreed upfront, user engagement for testing locked-down, senior leadership aligned and kept informed regularly. Someone running the administrative side of the project, keeping people on target, etc. (could be a double hat, of course). Sounds all rather "menial", but high degrees of organization really make a difference in delivery on larger projects.
AnimalMuppet 5y ago
Yeah. I'm a big fan of making the person who made the promise fix the mess. You dreamed up something and promised it to the customer? You get to go back to the customer and eat the crow. Maybe that will teach you not to do it next time. Anything else is just enabling (or even rewarding) bad behavior. If you do, expect to get more of it. Note well: I have never been a manager, especially not an upper-level manager over both sales and engineering. I don't know how well my recommendation will fly in the real world. (Hey, I guess that makes me the guy who just sold something without knowing if it can work...)

ColinHayhurst 5y ago

> Estimates are one of the hardest parts of software development. And also fundamental. When I was directly estimating big software projects the key, for me, was to trust developers recommendations but apply a different multiplier for each developer. Multipliers ranged from x1 to x3. Those rare devs with x1 were, of course, a blessing. And those with x3 were not necessarily bad; they were often the ones working on the really hard problems. Of course, it meant getting to know those developers and a prior (which we set at x2, for new starters). Individuals were remarkably consistent in terms of their actual performance; so a x1 developer would almost always be a x1; a x1.5 would almost always be x1.5.

ragebol 5y ago
How much learning time did you typically need to arrive at that multiplier?
- ColinHayhurst 5y ago
  At the end of two completed projects from each new starter
LegitShady 5y ago
The multiplier was the same way I dealt with estimating time budget for homework in university. I would estimate the amount of time I thought it should take and multiply by 3 and it would usually take between 2-3x at the end. It's a little harder when the work is more nebulous and I don't have to do it for others but it's all based on personal estimations of productivity as well as understanding of the problem. It's not easy to do.
killjoywashere 5y ago
A Jira plug-in to use some sort of Bayesian system to adjust developer priors at the task and sprint level. That would be amazing.
- msluyter 5y ago
  Yeah I've always felt like the missing part of the loop (in the context of pointing stories) is looking at point estimates and then tracking how accurate they were, _by developer_. E.g., I throw 2 points, but it turns out the story takes something along the lines of 13 points, then my estimation was pretty low and my personal "estimation multiplier" is increased. Subsequently, actual point estimations could take the estimation multiplier into effect. Of course, this is mostly a toy idea that's fun to think about, but seems pretty untenable given that it requires keeping track of time, which most developers hate. That, and there'd no doubt be incentive to game the system by padding estimates or other shenanigans.

antondd 5y ago

This brings back memories from the days of my early career (an ex-PM survivor here). I would be curious to see some data, even anecdotal, on the success of this approach. Here's some interesting statistics from the industry (not specific to software, but you can extrapolate): http://apepm.co.uk/project-management-statistics/ http://apepm.co.uk/project-management-statistics/ In my view, traditional software project management is ineffective. I would put it somewhere between the Myers-Briggs personality test and modern day astrology.

kalleth 5y ago
Insightful, thank you! The entire project management industry doesn't have that great a "hit rate" -- consider the budget overruns for the last few Olympics, or for Crossrail. I'm just not sure why software projects are "special" -- if you can avoid it being a project and instead make it ongoing OpEx like, for example, GDS managed for the UK in 2016, then great, you've sidestepped that, but until the entire PM industry discovers how to improve overall project management techniques, I don't see why we'd consider our industry "above" them.
- some_chap 5y ago
  I'd say that software is "special" because it's ephemeral, meaning there's incredibly few limitations on the possibilities (or changes to requirements mid-project) when compared to projects involving physical items. It /can/ be managed like a physical engineering project, but the cost and time ramp up so severely that it's not practical for most situations.
- tonyedgecombe 5y ago
  You can't really do that with the Olympics though as there is a hard deadline. Once you have a fixed date then either quality or cost are going to have to give.
- marcus_holmes 5y ago
  > instead make it ongoing OpEx Every non-tech startup founder who's approached me with "how long will it take to build an MVP for my startup idea?", I've answered with this. Development is an ongoing cost, a process, not a once-off capex cost. I recommend to them going the other way. Start with "how much can you afford to pay a dev team sustainably?" then work out how many devs that works out to, then work out how long your MVP will take to build based on their estimates (and estimates are not deadlines). Not quite the same as #no_estimates (which I also try to argue for whenever possible), but close.

christophergs 5y ago

Great post. A key point you don't bring up is the aftermath, even if you do deliver. Especially in non-tech companies there still remains the tendency to view these projects as "done" after the end of the project/MVP etc., with no understanding that sites need ongoing maintenance and improvements. And that this work is still considerable.

gonzo41 5y ago
That reminds me of that scene in the series Chernobyl where the main scientist briefs everyone on the cleanup effort and ends with something like "The first battle is won and now begins the long war" with everyone being suitably cold in their response. I get the same response sometimes when I talk about the long tail of maintenance at work.
- christophergs 5y ago
  Haha, love that show. There's definitely a parody in there somewhere > Now that I know what software estimation is, I no longer need you.
noptd 5y ago
Agreed, especially for MVPs and "phase 1" projects. At my current company, it's reached a point where we flat out reject product proposals for features or changes that would need to be hacked together for an MVP without a time commitment from all necessary stakeholders on how it will properly be implemented for phase two (iff phase one is a success). It's amazing how quickly "critical" features become irrelevant to product when they understand even half the amount of work required to properly implement them.
- epalm 5y ago
  Knowing when to say ‘No’ is an important (and probably underrated) skill.

deleted 5y ago

[deleted]

ngrilly 5y ago

> It also tells us how many team-weeks this fictional, idealised project would require [...] by adding all the estimates together. I would be wary with just "adding all the estimates together". That's because we tend to estimate the median or the mode of the task duration, and not the average. Means can be added together, but not medians.

marcosdumay 5y ago
> Means can be added together Is the error distribution of task size estimations normally distributed? Because I do really expect it to have a fat tail, and if it does, you can't add means either.
- ngrilly 5y ago
  I think most of us in software engineering assume the probability distribution has a fat tail. I've seen some authors name this the "blowup factor". For instance, your most likely estimation is 10 days, best case is 5 days, and worst case is 30 days. I think adding means is still meaningful (see central limit theorem and law of large numbers).
  - marcosdumay 5y ago
    It's exactly the central limit theorem that breaks for fat tailed distribution. Some fat tail distributions also break the law of large numbers, but I don't think task size estimation is this flawed.
- jdlshore 5y ago
  There’s a variety of analyses out there and they very consistently show a log-normal distribution for release predictions. I’ve analyzed Star Citizen’s publicly available data and found the same for their task estimates. It’s very reliable. You do see truncated log-normals, though, when the estimates are padded.

dls2016 5y ago

Project management is hard. I wrote and won an SBIR award this year. A much different scale than the article, but I spent a lot of time writing the plan and budget and sourcing components and estimating software tasks. Two months in and a big chunk of that is out the window… haha. Finding connectors and other components has been a big source of pain, especially if purchasing in small quantities. Another example: I bought a consumable product which then immediately became unavailable… so do I try to make do with what I have on hand or make the decision to switch to a replacement? Stressful, but I try to have fun. And extremely satisfying when things work out!

28304283409234 5y ago

I've started estimations by their true name: Assumptions.

heresie-dabord 5y ago
For those who are new to software development, the following terminology may prove helpful: Software Project Estimumption (or Assumptimation, the professional community is split) Software Requirements GatherWhims Software Requirements Analysthetics Unit Test Coveroverage

henning 5y ago

Bullshit. You can make guesses about the future, but since that is going to inevitably change, it's a guess and can never be anything but a guess. At most companies, the scope is going to be radically expanded and looking too far in the future is a complete waste of time. Other people in other fields are held accountable for deadlines because their work does not completely change and is not severely under-specified. If it is, then they are also just guessing.

crazy1van 5y ago
> it's a guess and can never be anything but a guess Of course it's a guess. The question is how to make more accurate guesses.
- dragonwriter 5y ago
  > The question is how to make more accurate guesses. I think the question should be about how to maximize the net return on time dedicated to development, which may mean spending less time on (and investing less reliance on) estimates rather than expending unbounded effort improving the quality of estimates.
  - satyrnein 5y ago
    What's the value to Sales/Marketing/etc of knowing what's coming when? How do you estimate that? It's estimates all the way down!
NikolaNovak 5y ago
I think author addressed that explicitly as valid and true software engineer perspective, right at beginning of the article; and explained that such software engineers, and products they build, then usually slot into larger company's ecosystem (as by strict math, most IT team members will work at a large company as opposed to a startup), full of people and leaders and departments and teams and project who have plans and deadlines and dependencies, which are also valid and true. I've had either luck or misfortune to "flip a switch" from two decades of being a techie/architect, to being a mid-manager, on basically a specific date as opposed to over the years, due to project's needs; and it's like that B&W picture of two faces and one vase in middle. Both perspectives are true, even if opposing and contradictory. Good business lead will understand software engineer's perspective - even if it's not their primary view of the world, they can squint and catch a glimpse as needed. Likewise for good software leads. Obstinate unwillingness to see or ascribe merit to other perspectives puts a ceiling on everybody's progress.

pphysch 5y ago

There are arguably two major phases here, and its only the second one that most folks here find controversial. * Steps 0-2: determining "what" the project "is" (design, architecture, ontology) * Steps 3-6: procuring and allocating resources to complete it (economics, management, politics) It is tempting to decouple the two phases, and as a technologist focusing solely on architecture while leaving the economics up to leadership. However, social factors (the real people involved with the project) are an integral part of actually getting anything done, so I agree with the author's premise that the whole process should be viewed holistically (and ideally run by one technologist).

kalleth 5y ago
Well phrased, thank you! I didn't think of it in this way, but yes, that kind of phasing makes sense.

dpweb 5y ago

I found it really fluctuates based on the team members. We definitely had 3x 4x differences in productivity between the worst and best on the team. Our estimates had to be good as it determined the size of the sales deal. Most everyone on the team was there for years, so we knew each other well. We could give a solid estimate but with a new team member is more challenging.

mumblemumble 5y ago

> you should expect to be held accountable if your estimates are too far off the mark because you failed to do your due diligence when coming up with them. In a perfect world, I agree. In the real world, which has a remarkable knack for failing to live up to expectations, what I find is that companies are rarely willing to allow the development team adequate time to do their due diligence. Answers, in and of themselves, are cheap. I can give you those all day. If you want to be able to hold me accountable for their accuracy, though, then you need to be looking at my correct answer rate sheet. For me, the magic of #noestimates is the magic of open, honest cynicism. If my realistic options are silence and blowing smoke up my boss's ass, I'd really prefer it if they would allow me to choose silence. That way we can both keep our dignity.

lifeisstillgood 5y ago

- Projects only become official and tracked once someone has hacked together enough of a prototype to prove it works. - at this point all project management is pretending it takes 100 managers to land something one girl / guy got flying. - stop project managing, stop estimating, and just start treating companies as VC firms. Hire good devs, make them care about your mission, invest in those that take off. Don't take the control away from the original devs

sarks_nz 5y ago

How to make developers want to hit their deadlines with quality? Startup land. There was no feedback loop that rewarded developers to meet the estimates. Stock options weren't an option, and I didn't want them to do a sloppy job just to hit the 'deadline'.

ed312 5y ago
Bonus and/or equity grants via review feedback are a blunt but effective tool. You could probably reduce this to "My team is not highly engaged/motivated, and I think it would better meet my company's needs if they were. How can I improve that?"
skohan 5y ago
Honestly it's not developers' job to hit their estimates. If we're talking about a long-term estimate, as in "this project will be finished in 6 months", it's your job in management to find a way to do this. You've got to break the goal down into achievable sub-goals, and monitor progress along the way. Long term estimates will be wrong. Software projects generally take longer than expected, so it's up to you as a manager to anticipate this and communicate to stakeholders with the correct degree of uncertainty. If it's an external deadline which must be met, firstly you should engineer enough extra time into the timeline to handle inevitable delays. And if at any point you feel like the timeline is unachievable, it's up to you to renegotiate with stakeholders, or adjust the scope to make it achievable. And if you have the feeling your team is slacking off and not getting work done, honestly this sounds like a lack of leadership skills. It's up to you to have the kind of relationship with your developers so that they are motivated to meet the team's collective goals and take responsibility. That's basically all that being a manager is.
- sarks_nz 5y ago
  Thanks for that. I'm talking about the difference between hitting the deadline on Monday versus Friday. How to incentivise that? ie, do half an hour more work for a week, or skip the table-tennis when someone asks, etc. As a developer, I was into making sure I hit my goals, and at work to work. As a manager I do struggle with how to emphasise that ownership of product, quality, time. Why should developers care about hitting Monday with effort, instead of coasting to Friday?

galaxyLogic 5y ago

Doesn't this assume you have the "spec" before you start implementing it? But then how do you estimate how long it will take to come up with that spec? If you have a very good detailed spec you have already done much of to work to make the implementation easy. If the spec is high-level and "fuzzy" it leaves the work of "resolving the spec" to the programmer. So trying to estimate the time it takes to code a system depends on the quality of the spec, and therefore is difficult if there is no standard on how detailed the spec is to be.

kalleth 5y ago
If you don't have the spec, then you likely don't have a deadline, or a "project", really -- and something like this would be the wrong choice for an approach to follow. I'd say leaving the overall roadmap (which is all this produces, at the end of the day, if you ignore the estimation piece) fuzzy and allowing the team to work that out with users/subject matter experts is the right approach, imo.

tikhonj 5y ago

Seems like there's a structural issue at play here: the team is accountable for building a specific set of features at a specific time. High-level functionality is fixed and the timeline is fixed. If the estimate is off or something unexpected comes up, what's going to give? Answer: everything else. Anything that the planners and executives didn't consider ahead of time. Other aspects of the work: code quality, product design, accessibility, performance, robustness, edge-case-handling... Team learning, culture and satisfaction. At the limit, the team will compromise on anything that you don't need to claim a feature is "done" with a reasonably straight face. It's simply not a good system even when the estimates work out reasonably well—and, empirically, they usually don't. To be fair, this isn't entirely the fault of estimates in general or this estimation approach in particular; I believe those contribute, but it's primarily a reflection of how the company and culture are structured. If you're already in a system like that and you can't change it, trying to do estimates well might be the best option forwards, but only because you're already in a corner.

iamgopal 5y ago

I would divide the projected total task in to 5 equal part and again each part in 5 equal part, and just worry about first of the first, after every part completion we can revise the target

igouy 5y ago

https://rightingsoftware.org/ https://rightingsoftware.org/ "Based on first principles in software engineering and a comprehensive set of matching tools and techniques, Löwy’s methodology integrates system design and project design. First, he describes the primary area where many software architects fail and shows how to decompose a system into smaller building blocks or services, based on volatility. Next, he shows how to flow an effective project design from the system design; how to accurately calculate the project duration, cost, and risk; and how to devise multiple execution options."

meesles 5y ago

The more projects like these I plan and execute, the more I find the need to account for external parties affecting timelines. In the case of the author's payment system, say they go with a vendor. At that scale, there may be weeks of contract negotiations for fees, rates, minimums, etc. What if some piece of documentation is flawed and you need to have a back-and-forth with their support? What if they need time to onboard you into their systems? It makes me appreciate Apple's supply chain mastery that allows them to deliver exactly on time because they own their whole process inside and out and demand similar rigor from any vendors that supply them. If we could imitate that in software, we could eliminate a huge source of uncertainty in many projects.

beckingz 5y ago
Apple also cuts scope and delays committing to a release until they know they can deliver...

Aeolun 5y ago

> Engineering does not work in a vacuum, and there are commercial contracts that will be signed, at a high level, without your involvement. Deals that will have been agreed before you joined the organization. Indeed. And they’ll have been built on lies, damned lies and wishes.

grayclhn 5y ago

Call me crazy, but if we're going to call these goals, "estimates," they should actually be based on real data. Otherwise we're just making up numbers. If the relevant data aren't tracked or stored anywhere, that kind of tells you how serious the org is about making accurate estimates.

mtippett 5y ago

I like how he says little-a agile - what most teams use. big-a Agile I reserve for specific Agile methodologies. Most companies aren't willing to invest in the rigor and effort for big-A Agile, and so you end up with this weird-ass hybrid.

dave1999x 5y ago

No seriously, no. Dive straight into debunking #noestimates but don't get to the fact that no one ever really knows what they want.

jussy 5y ago

From what I've seen the biggest problem is that the estimates are done as part of a sales cycle or to get approval to spend money of some sort. Which is a different motivator to actually getting it done.