7 ms·
All late projects are the same (2011) [pdf]
- rossdavidh 8y agoWhile much of this is true, I think it also leaves out a very common cause of software being late, perhaps the most common. The time it will take is unknown, in some cases unknowable, and no one is willing to say so. "How long will it take?" (equivalently, "Can you do this feature set by this date?") The only honest answer is: "I don't know." Nonetheless, a date gets put on it.
- agumonkey 8y agoIsn't there some pseudo quantitative form of estimation based on a first draft system network diagram ?
- tcmb 8y agoIf you can't be flexible on the delivery date, you have to be flexible in feature scope. That means building a simple verison first that still fulfills the purpose while not being perfect or polished. Then iteratively improve it to what you and the customer ideally want. That way, if the deadline comes before you're done, you at least have a working version that fulfills the basic need and allows the kind of workflow that's necessary, even if it still has some rough edges or even gaps in rarely used areas. Of course this approach requires you to write your contracts in a way that doesn't tie you down too strictly on how exactly things will be solved.
- zdragnar 8y agoSadly, when it comes to contracts, that gets boiled down in most customers' minds to: "Vendor only commit to delivering buggy and incomplete software". It's a hard sell for most types of projects, even if it's the actual reality of most projects.
- chii 8y agoYou sell a small portion, for the same smaller price tag. Then when the deadline comes, the customer checks to see if you delivered, and decides to continue the contract or find somebody else. So the customer can have assurances that they aint gonna be bogged down with sunk costs. The developer can have more frequent deliveries.
- wwarner 8y agoDisagree. Upper and lower bounds can be realistically estimated. Whether the estimates meet the demands of the market is another question. I'm an engineer, not a CEO or sales person, but I'm a firm believer in selling first and building it later. Building it after it's been promised is one of the best ways to limit scope.
- rossdavidh 8y agoI have seen "realistic estimates" be off by multiples. Also, "build it after it's been promised" is not making any sense to me, perhaps you can restate that somehow?
- ryan-allen 8y ago> but I'm a firm believer in selling first and building it later Please elaborate! :)
- wwarner 8y agoIn a word, it's "agile". Let me first acknowledge that there my thoughts are a mixture of the types of projects that I've been working on, my own personality and my own path dependent experience. In particular, my way is intentionally short sighted and won't work for a project plan that takes five years. Also, for sure, if the sales team/ceo is wrong and the project isn't worth doing, then doing it "lazily" won't fix that. But with that out of the way, here's what I mean. The engineering team has it's own values and priorities and they just don't have anything to do with what really matters to customers. In the end, everyone brings in emotional baggage from the last project they worked on, and that's what's important until new information comes in from customers and the sales team. So what I shoot for is taking the requirements that we have and producing a long range design that will meet those and little more, and then producing features as quickly as possible and presenting it to the customer (or the customer's advocate). Why does it limit scope? Because the deadline doesn't move! To meet the most important requirements in a way that actually sells the product, lower priorities are are cut or reframed.
- chii 8y ago> Building it after it's been promised is one of the best ways to limit scope. only if those doing the promising knows what's within bounds and what's wildly off bounds! If the sales person makes wild promises because his commission depends on making the sale, but not the delivery, then the whole organization's interests are not aligned properly, thus fail. If the salesman's commission comes from successful delivery, then he can bring in an engineer who can at least balance the promises. Who knows, may be different departments can cross-contaminate, leading to a better, more well rounded organization.
- mden 8y agoand no one is willing to say so. Except most engineers are perfectly willing to say so in my experience but management decides to ignore them or force them to say a made up number promising falsely not to hold them to it. Even worse what happens very often is an engineer would give an estimate and then a manager would "bully" it down to a lower number. On the reverse of course it makes sense to have estimates otherwise how should managers or product people decide what should be done next? A big part of the issue is we don't have good knowledge how to estimate work. It's not a skill that is heavily invested in and is wrongly assumed as an inherent ability.
- rossdavidh 8y agoI have, on more than one occasion, said so, and the response was always bafflement or annoyance, and in either case and unwillingness to accept that answer. When I was younger, I would eventually give a guess, along with a "...but that's just a guess". Nowadays, I don't even give that, because the caveats get forgotten/ignored.
- codycraven 8y agoI have this sort of discussion, at minimum, three times a week (we're in-house developers). Very often the project scope isn't even defined and upstream dependencies are known to be critically lacking and will need to be augmented to even perform the work. Yet we're always pressured for dates and the caveats given are immediately forgotten and almost never communicated. The biggest pet peeve is when I give an estimate such as "I don't currently see a reason why we can't deliver X by the end of next week, assuming no other priorities come up." Of course something more important comes up in the mean time and I get scolded in a large meeting with a statement such as "You promised me I'd have X by Friday!" — this is especially infuriating when it is the PM's other project that is the higher priority interruption.
- lifeisstillgood 8y agoIt might not be viable, but I used to "publish" internally my own project one pagers, and update the timelines - any discrepancies simply became a shield for this future arguments. In short it sounds like you don't have the power to say no - but actually you do.
- wwweston 8y agoYou could argue that these are some of the many forms of "starting late": * If you just don't know how long something is going to take, and you need a date, then you're late to doing the work to know how long it will take. * If you do know how long something is going to take, and it's longer than a externally imposed date, then you know exactly how late you are. This is partly just playing with semantics, but it's also a way of talking around different views of the problem, in this case emphasizing that if dates are a feature, reasonable estimates are themselves a project. If the estimate work hasn't been properly completed and understood before dates are given, obviously, the larger project is by definition already late because there's work that needed to be done before beginning and wasn't. And I think if you take this view, it's easier to see that it's a project or team management problem rather than an engineering productivity problem.
- lodi 8y agoI completely agree; you discover timelines in the process of doing the work. The problems start when you need to give an estimate before even being allowed to start working on something.
- rossdavidh 8y agoIn at least some cases, you have to be halfway into doing the work (or more) before it becomes apparent that there is a big, previously-unknown subtask or requirement that invalidates the plan for doing it. In other words, if you really know enough to give a good estimate, you are probably mostly done.
- watwut 8y agoIt is not about it being unknowable. If it would, the estimates would be all around the place. Sometimes short sometimes long. Instead, we systematically underestimate.
- kgilpin 8y agoI think you’re missing the point of the article. His original theses was that we aren’t good enough at estimation. And in fact much hand wringing over estimation has been done in this industry over the last 6+ decades. Now he’s amending his opinion to state that estimation is not really the core issue. The core issue is that projects are doomed from the start by mismanagement. Issues such as: racing to catch up with a competitor (giving the s/w group an impossible deadline), or starting a project whose business value is so low that minor time and cost overruns reduce the actual value to below zero (at which point people start blame shifting to try and hold on to their jobs).
- jen729w 8y agoThey’re all late because customers are fools and salespeople are liars. Customer: “how long will it take (and therefore cost)?” Three vendors: “less than [the other 2 vendors].” ...iterate until “cheapest” solution is reached ... meanwhile the technical staff at the vendors are losing their minds at the sales team, telling them that what they’re specifying is impossible... Customer, to vendor [cheapest]: “Here’s a contract. We look forward to this project being delivered on time and on budget!”
- GuiA 8y agoGeneralizing, this equates to “projects fail as result of poor/dishonest communication”, which is definitely accurate.
- trainingaccount 8y agoWe sure don't want it to go over time and budget like the last 10 projects.
- Sharlin 8y agoJust need to manage the lazy developers better (read: more) this time!
- sgt 8y agoMore meetings will do the trick. A daily 30 minute standup, and a few alignment meetings scattered throughout the day to identify the technical causes as to why progress is slow. Let's however not go into too much technical detail, but ensure that the developers are aware of what's at stake here.
- avip 8y agoSalesperson to raging customer (aftermath): "We were only able to complete the bathroom in the estimated time. You are welcome to live in it for a reduced monthly fee while we build the rest of the house around you"
- jt2190 8y ago(2011)
- jt2190 8y ago> [I]nstead of obsessing over “What’s the matter with those software folks who didn’t deliver on the schedule we gave them?” we need to ask instead “Why did we ever kick off a project with such marginal expected value?
- aliswe 8y agoWell the projects small marginal value isn't really related to the late delivery of the software people. I always say that delivering software successfully has a lot to do with courage. That includes courage to say (communicate) that a spec is too sketchy or outright unrealistic to deliver on time . That someone is dysfunctional within the project. That the timeline is too aggressive and needs to be split up into phases according to worse case scenarios. If you can't do that, then you are the one to blame for tge project. If you did, though, then youre free to crash and burn happily ever after.
- jayd16 8y agoThis is why "agile" methodologies caught on. Ignoring how they're used as internal political cudgels now, the most general goal of agile is to constantly adjust scope and estimates. This way, we can accept that we're all terrible at estimates but we do our best not to paint ourselves into corners because of our terrible terrible estimates.
- kgilpin 8y agoHe doesn’t think estimation is the problem. He says that the projects are started late and that developers are handed a deadline by the business and then accused of being late when they can’t meet that deadline.
- jayd16 8y agoThat's still an estimation problem by management. In agile, all sides are supposed to understand that scope and timeline are constantly adjusted.
- dang 8y agoFrom 2016: https://news.ycombinator.com/item?id=11098466 https://news.ycombinator.com/item?id=11098466
- shoo 8y agoOne perspective is that negotiation of schedule and resources is often nothing resembling estimation but is a political game. Ed Yourdon's book "Death March" has a chapter about Negotiation . http://ptgmedia.pearsoncmg.com/images/013143635X/samplechapter/013143635X_ch03.pdf http://ptgmedia.pearsoncmg.com/images/013143635X/samplechapt... There's a list of "negotiation games" that, sadly, you'll probably recognise. And a bit of more practical advice: >> The problem I have with “estimate” is that I understand >> the word to mean, “best assessment knowing what is known >> at this moment and what can reasonably be predicted about >> the future.” From experience, what I have seen happen is >> that once an estimate is published,it becomes a rock >> solid commitment representing the maximum amount of >> resources needed to accomplish the project phase or the >> entire project. > Unfortunately, there’s an uglier aspect of the political > negotiation when you introduce the plus-or-minus > qualifier into your estimate: You’ll be accused of > uncertainty, wishy-washiness, weakness, or even > incompetence. [...] What senior management really wants > is a firm commitment — a promise that the project will > be finished on a certain deadline, with a budget of a > certain number of dollars, and a staff of a certain size. > This gives them the enormous luxury of (a) no longer > having to worry about the problem for the duration of the > project and (b) having a convenient scapegoat to blame if > the promise is broken. > Jim McCarthy, in his excellent book, Dynamics of Software > Development, suggests that the project manager needs to > confront this head-on and persuade the customers and/or > senior management that they need to share some of the > burden of uncertainty [around schedule, cost, resourcing] > that the entire project team will be living with on a > day-to-day basis. Thus, the project manager effectively > says to the customer or the senior management group, > “Look, I don’t know precisely when this project will > finish — but since I’m the project manager, I’m far more > likely than anyone else in the organization to figure it > out as soon as it can be figured out. I promise you that > once I know, I’ll tell you right away.” > Only a manager with a lot of self-confidence, and the > ability to walk away from the assignment, has the > chutzpah to say something like this in the politically > charged atmosphere of a death march project. The time to > say it is at the beginning of the project; after all, if > the customer and senior management do not respect your > ability as a project manager, and if they don’t realize > that you do have a better chance of knowing when the > project will finish than anyone else, then why are they > putting you in charge of the project in the first place? > Are you being set up as a scapegoat? Are you going to > be a “puppet manager,” with all the decisions being > made by other political manipulators in the > organization? If so, now is the time to get out! > Similarly, if you’re a lowly programmer on the project > team and you see political games like this, it may be a > strong indication that your project manager (a) doesn’t > have the confidence to believe in any estimate that he > puts forth, (b) doesn’t have the backbone to stand up > for himself and for the project team, and/or (c) has > gotten himself into a political situation where all the > key decisions will be made by people who are not > directly involved in the project. Again, this is a > strong indication that the project is doomed; and > before you get too deeply involved, it might be a > better idea to seek greener pastures. Refusing to be pressured into emitting on-the-spot "estimates" that will be regarded as commitments is a useful life skill. Teach your new hires: http://www.dadhacker.com/blog/?p=2267 http://www.dadhacker.com/blog/?p=2267
- juskrey 8y agoThe projects are late because no single project have finished before start.
- profikid 8y agoI think it also has todo with scope creeps. When a product / Service is developed, new possibilities are appear and customers are always trying to put these in the scope without adjusting the time / money necessary
- 11thEarlOfMar 8y agoThe most successful software manager I've worked with, in terms of timeliness, was the one who deconstructed the work the most before starting. What exactly was going to be done, how confident were they in their estimate? Not confident? Break it down more. More detailed requirements, more detailed GUI mock up, More pre-design. Then, when confidence was high enough, by his judgement: who was going to do the work, and how did that work fit into the organization's schedule, including the testing, bug fix cycle needs, test equipment availability, and release. He was nearly always on time, and never acceded to the customer's desired delivery date unless he could meet it. I'm not saying this works in all environments, but for his case, it did.
- mgoetzke 8y agoWhich is indeed successful. But this also takes time and sometimes sadly not allowed or deemed necessary. It also does not allow to respond to changes quickly unless a proper re-calculation is also done. Good technique for a lot of scenarios anyway.
- snarf21 8y agoThis can surely work depending on organization. The problem is when the sales people need an estimate in two days or they don't get the contract. Then it is just a wild ass guess and the normal problems ensue. The reason that software is so hard to do in most cases is that you are making something that (by definition) has never been done before. Of course stuff is going to go bad. This is largely a corporate culture problem. In these cases, I want to have the sales people come to a weekly meeting and have them estimate when they will have 100K sales for the month from new business. I don't think they would like that very much and you'd get a lot of dithering.
- heavenlyblue 8y agoThe question is whether you're working for a sales company or a tech company. Sales companies will prioritise the requirements of sales people, and tech would prefer to ignore the sales guys. Frankly, as an engineer - I would never prefer the company where the sales people can decide how much overtime I would be doing in the end. Also - they would be getting all of the bonuses, since it's them making the decisions in the first place. The good thing I find is that working for the engineering companies usually implies having clients with a head on their shoulders (hence them knowing what they want and what they're getting; and how much time it usually takes).
- tombert 8y agoWhile I 99% agree with the thesis of this post, I feel like another part to why I was late with a ton of projects earlier in my career was due to me getting prolonged depressive episodes, where I didn't want to leave my bed, let alone work on some software I really don't care about. For me, I was later diagnosed with Bipolar II, but it makes me wonder how many people just try and "walk it off" with these things, and as a result their work suffers.
- tworc 8y agoWe work, live and operate in a complex environment and I think if we start viewing the organisation its projects and the environment in which it operates more from a systems perspective we may realise that in fact, it is truly difficult to make accurate predictions given the non-linear behaviour the system exhibits that results from all the information flows (or lack thereof), interaction of parts and other dynamics such as markets, human energy levels/mood, etc. I really learned a lot from the book called Thinking in Systems and nowadays I can't help to look at projects, from within the context in which it operates. Definitely recommend the book for somebody wanting a bit of an introduction to that mental model.