8 ms·
In my experience talking to hundreds of software engineering managers, running a software project management business, and being involved on software dev teams
by kareemm 8y ago
In my experience talking to hundreds of software engineering managers, running a software project management business, and being involved on software dev teams for 20+ years, there are two reasons why teams don’t ship or ship poor software.
1. Poor engineering planning. The way to do this well is to break down a feature until they’re at about half-day sizes tasks. Identify any obvious risks or ambiguities with the technical approach and communicate them to Product and figure out how to de-risk them. Communicate costs to Product so PMs can make cost/benefit trade offs (eg “if we built it the way you specced it it’ll take two weeks. But if you make these trade offs it’ll take two days.” As a Product Manager I might not want the feature if it costs two weeks but do if it costs less than 5 days. Without costing the features in dev days I can’t makr this trade off).
2. Failure to make trade offs that drive towards shipping software. The way you ship is by declaring it’s done, even when it’s not really done. When code is being written unexpected issues always arise. It’s Product’s job to make hard trade offs that drive towards shipping. Eg “that’s an edge case bug, let’s punt it to v2” Or “let’s cut that nice to have feature and squash the showstopper bugs and ship”.
People bring up impossible deadlines set by management. In my experience most deadlines are movable and a strong PM/Eng team can convince management to push a deadline back. And if a deadline can’t be moved, it’s all the more important to know how much features will cost and make hard trade offs to get the product out the door.
- hliyan 8y agoWholeheartedly agree with #1. I'm fond of saying, "If it takes longer than four hours, you probably haven't broken it down enough. Find the monsters hiding inside your big tasks."
- sh87 8y agoI’ve seen concerns and FUD over getting lost in too many tasks. How do you manage the overhead of 20 4-hour tasks against 2-3 high level tasks?
- hliyan 8y agoWell, they're just checkboxes inside the main tasks. You just think through how you're going to perform the task (which you'll anyway have to do when you get close to actually doing it), and note down the sequence of steps. Then you try to estimate each step. Sounds simple, but it works.
- serpix 8y agoWhy bother estimating? you have the checkboxes, now just get to work. If the pace of checking those boxes is slow then you can start cutting things out. Estimates are for managers, much better to just ship it already.
- fjsolwmv 8y agoBecause if you haven't built it yet, you can't ship it already. But you are right that estimates are for managers. Managers should watch their reporter's progress and use that as evidence to build evidence. If the first task of 20 took 1 week, it's a 2year project. Update estimate as time and progress and work remaining increases.
- kqvamxurcagg 8y agoHow do you create this detailed plan? Are there any tools or frameworks?
- shadowmint 8y agoThere are literally hundreds of methods and tools; it doesn’t matter which one(s) you pick. I’m quite serious. The thing you need to realize is that you’ll get it wrong, no matter the tool... until you invest time in reviewing past estimates and plans. People think because they’re engineers they can just magically make estimates... but its a skill. If you don’t have a feedback loop to measure your progress and adjust your actions, you’ll literally never improve at it. ...that’s why even ‘Senior’ developers and ‘technical leads’ are often terrible at this; they’ve simply never bothered to seriously make an effort to learn how to do it.
- root_axis 8y agoI'm a big fan of arc42
- bazooka_penguin 8y agoHow do you estimate how much time a task will take, especially when it's a novel idea, at least for your developers?
- LargeWu 8y agoYou don't. That's the thing. Yet decision makers commit to deadlines anyway, and that's the problem.
- mirceal 8y agoYou break it down (effectively doing the work) as part of planning and after that you pretend that you have a working system. Make sure not to go into details and/or “take it offline” until the developer is tired/annoyed enough to just agree with you pipe-dream estimates /s
- bazooka_penguin 8y agoI see, so you simply hire superhumans who finish the project before UX mockups are even available. Simple stuff. I think I'm ready to start my own consulting business
- mirceal 8y agoYou got it. Make sure every task can be done in half a day or less. That is key /s
- commandlinefan 8y ago> How do you estimate how much time a task will take when it's a novel idea In my experience, they just pretend it doesn't exist and remove it from the "plan". Where I see this the most is in figuring out how things work. For some reason, non-programmers have it in their head that once you've finished college, you ought to know (even though they don't) how everything that goes on a computer works instantaneously. We're going to use Apache Kafka! Ok, need to spend some time figuring out how that works. Um, ok, that should take like an hour, right? Because there's no time in the budget to learn anything. If you push back, they say, "ok, never use anything new that you have to learn. Only use the things you already know". That doesn't work either, because the things that we have to integrate with don't work with the old things. Or, "you don't need to know how that works, Bob knows how that works, just ask him. But don't let it take any time, because there's no time in the plan for that" And Bob is sick of all your questions that you'd know the answer to if you'd just spent a two or three days reading the online documentation and installing a test system and testing it out.
- kbuchanan 8y agoI'll also wholeheartedly agree with this comment—my company has a long history of shipping on time and under budget. The caveat is that we closely follow #2: we aggressively strike requirements from the overall target if it means preserving the integrity of the project overall, with a commitment to return later. I'll add this: a product team that ships earns 1) more credibility, and 2) that credibility is reciprocated towards the team in the form of greater flexibility and freedom to set their own targets.
- AstralStorm 8y agoThis approach does not scale to maintenance. You cannot strike down anything when maintaining and/or porting. (And it is risky to even cut testing.)
- fvdessen 8y ago> The way to do this well is to break down a feature until they’re at about half-day sizes tasks. I have never seen that work in practice.
- mohaine 8y agoBreaking a task down into meaningful half day task is almost as hard is just implementing it, but usually this is attempted in a committee setting. This is not a recipe for success.
- khyryk 8y agoAgreed. Think of how ridiculous it would be to make a half-day task for figuring out X -- sometimes I've spent days, even weeks, trying to figure out how a module works. In practice, being able to break things down meaningfully and accurately to such a minute level implies that a lot of time was spent in grooming, which means we're just back to waterfall again. (Or the tasks at hand really are just that simple and/or the people involved have a lot of expertise.)
- trey-jones 8y agoI agree. Saying, "Break it down until it's in half-day size tasks" is fine. Some of the things that might interfere: 1. Are you sure you can do that in half a day? 2. You forgot about this one part of the task that you thought would take half a day. 3. There was a bug in the plugin that you were going to use to do this task in half a day, and you spent three days fixing it. 4. This task will actually take longer, but management won't buy that. Looks like it should be half a day on paper, so that's what we'll say. 5. I've never even done this one thing before, but it sounds like it should just take a couple of hours. I understand the concept - trying to say something will take a week is even more error prone. But I think most of us are still pretty bad at estimating a budget for tasks even at such a small scale.
- commandlinefan 8y agoI've never even seen it work in theory.
- mirceal 8y agoHave you ever been in a situation where communicating and breaking things down takes longer than the actual work? IMHO most disasters I’ve seen occur when ‘Product’ tries to strictly control the ‘Developers’. It just does not work for certain kinds of projects. It also diminishes the developers’ involvment and leads to a culture of mediocrity and CYAs. Also, a person that does not have to do the work will always be waaaay more optimistic about how much it takes to do the work. How do you fix it? Trust and ownership. Product and Dev need to have an interface - a boundry if you will - and the discussions and talk are about defining this interface and setting expectations. Product trusts Dev on implementation. Dev owns the delivery and makes it happen.
- lostcolony 8y agoThis is why agile tries to push the business stakeholders to define -problems- rather than -solutions-. Developers then try and come up with a solution, and pitch it to the business (effectively). Almost every issue I've had with the business/developer dichotomy (and I use that instead of product because different orgs place product differently) has been due to the business trying to pitch solutions to dev that invariably end up not actually addressing the problem (even simple things like "that's the wrong color" or similar, are because they believe they're responsible for the solution, and they have a very clear idea of what the solution should be, without actually being able to articulate or recognize the problem. Does the color actually matter? If you have a problem that implies it does, yes, if you just came up with a solution on your own and in your head that solution was yellow, no).
- mirceal 8y agoMaybe, maybe not. I’m very suspicious of things that are pitched as a panacea for everything. The real world is a messy place where even with a good team, a good plan and sensible expectations you can fail spectacularly. If there is one thing that I really like is aggressive validation of the assumptions that are being made + keeping the product as small and lean as humanly possible (and the latter should apply across the board to everything you do). Complexity and Entropy are preparing the coffin for your work even before you have the idea. You can give ‘em a run for their money, but in most cases you’re going to end up 6 feet under. As soon as you accept this and are genuinely open to failure you will be ready for the opposite.
- makmanalp 8y agoMy god, this is spot on. Re #1, this is kinda the system 2 vs 1 thing in the article. I was lucky enough to have a PM drill me into this. More specifically: The bigger a task is, the likelier it's going to go over time. The bigger a task is, also the less likely each bit is well specified and more likely that there are significant portions glossed over that you'll discover during implementation, which will delay things and/or result in them being done poorly. If you're having a hard time breaking the task down, it's probably because the task isn't well defined enough, so go back to stakeholders and define better. In this model, planning takes a lot of time and effort. Also in this model, research-y tasks are different, and instead get a fixed timebox after which you have to re-decide how much more time to spend on it if at all. Re #2, part of the problem here I think is that PMs aren't trusted / empowered with that ability or final say. "Stakeholders" get to make that decision, but with ignorance of what it's currently like in the trenches.
- wolco 8y agoMoving milestones, incomplete requirements, too many meetings, wrong tech stack, trying to break everything into halfdays to fit an agile agenda. What does work is waterfall. Whatever is in the repo at the end gets shipped.
- mkempe 8y agoIn the context of a fixed-time, fixed-price/budget project it will fail to meet these constraints unless the scope is very clearly delineated at the outset, and ruthless scope/change management is applied throughout the progress of the project. And if at some point someone realizes that there was an omission in the scope definition, the time and budget must change. If the client or management want to change the scope, in any way, the discussion must immediately start with questions such as what other feature are we dropping, how much time are we adding to the delivery plan, what are the consequences on other features or the current design and code? Even deciding to remove a feature can cause delays (e.g. having to re-structure a test scenario). Or changing a string in the user interface, which may require re-arranging the layout, re-translating the string, suddenly having to manage plurals, checking that a variation of the old string wasn't already used elsewhere, etc.
- fjsolwmv 8y agoWell sure, if you doom the project before you start, with a fixed price and scope, it will fail. The original primary point of the Agile Manifesto is to stop doing that. There's no excuse for it, unless you are required by law as a government contractor, and in thar case no one expects a successful result anyway.
- mkempe 8y agoI've seen and worked on many successful fixed-time, fixed-price projects, medium- and large-scale, not for government.
- sonnyblarney 8y agoI have to respectfully disagree with #1. Software is hard because we 'don't know the unknowns'. It's not like building a building - if it were, we could just schedule it down to each detail. We would be still all be using 'waterfall'. If we knew how to write the module, we'd just do it, test it, and voila. But most of our time is spent 'figuring stuff out'. Oh, product has asked for a tiny change (support a specific kind of font), but our libraries don't support that, meaning major possible architectural changes. The other major unspoken factor is that requirements change. In fact, this is the #1 thing, really. Requirements change because customer's needs change, because business changes because our understanding of the market changes, etc., and this is a reality of our world. So the nature of how we're going to adapt is difficult to predict. This is why the #1 predictor of outcomes is the quality of developers on the team i.e. experience, intelligence, know-how of the domain. If you're doing something where there are very few unknowns, well then it should be possible to schedule within reason.
- icebraining 8y agoIt's not like building a building - if it were, we could just schedule it down to each detail. We would be still all be using 'waterfall'. I hear that it doesn't work that well for buildings either, as you end up with people in the field detecting bugs in the plans and having to improvise fixes without input from the designers.
- sonnyblarney 8y agoFor actual buildings, they spend a fortune up front planning. Soil and dirt samples at $100K - just to do a bid on a project, let alone the actual build. Can't spend $500M on something only to find you have to tear it down. Software is not magic, but it does require thoughtfulness and experience ... (hint to all those who think startups require mostly new/young blood - it's mostly wrong, but that's for another post)
- dmichulke 8y ago> The other major unspoken factor is that requirements change Walking on water and developing software from a specification are easy - if both are frozen - Edward Berard
- ben509 8y ago> The way to do this well is to break down a feature until they’re at about half-day sizes tasks. You're identifying what the outcome of good planning looks like, not how it's done. What prevents this is that teams read "agile" as "make it up as you go along," when in reality if you're spending more than 10 minutes discussing a feature, make planning that feature a sprint task. > In my experience most deadlines are movable and a strong PM/Eng team can convince management to push a deadline back. It's not generally the team in that room discussing those deadlines, though, it's software development manager and product manager. The worst managers are software engineers who get promoted to management, and when they get into those meetings they look at their shoes and are railroaded by upper management. You really do need a manager who has the intestinal fortitude to go to bat for you.
- cm2187 8y agoI'd add #3: over-engineering. 90% of the time a simple quick solution will be as good as a super sophisticated and abstracted solution that will be delivered 1 years later, when the underlying problem it is trying to solve has changed (cf the usual tree swing cartoon [1]). [1] https://www.tamingdata.com/wp-content/uploads/2010/07/tree-swing-project-management-large.png https://www.tamingdata.com/wp-content/uploads/2010/07/tree-s...
- AstralStorm 8y agoSometimes. And then someone adds layers and layers of crud on a "simple thing" because nobody actually engineered it. And then some poor soul gets to rework all this... Basic abstractions help a lot. Specifically, having things weakly coupled.
- hinkley 8y agoWith the exception of video games, I maintain that it's always better to be 10% over estimate on every milestone than to all of a sudden be 50% off of the final deadline. To work it takes being comfortable saying "that's my/our fault", and my experience is that a lot of people who would rather die than admit fault. It's part of a larger raft of skills around transparency. So they cover up problems hoping for some miracle that never comes (or more cynically, they hope to be employed somewhere else before the shit hits the fan).
- bittcto 8y agoIf you have deadlines, you already know management is incompetent. Deadlines presume a known fixed scope. But I’ve yet to meet a CEO or sales team or whomever gets to decide what features go in who will fix the scope if a project. They are always willing to add features, but can never take them out.