21 ms·
We are clueless about how long things should take
- SeeDave 7y ago"Bro, I'll just get some other nerd to do it. Instagram was built by 12 guys and WhatsApp was acquired for $19B with only 50 engineers. Can't ship if you whine; so shape up and stop wasting time!" /s
- couchand 7y agoA few years ago I was brought in lead a team in a division of a public company. Our mandate was to replace a major component of an enterprise system that you couldn't quite call legacy because they'd never actually gotten around to replacing the real legacy system with it. My initial proposal was to build it incrementally: tackle the obviously critical functionality to get something working and ship it, then define and implement additional features on an ongoing basis. I had hoped this would fly, since the company talked a big game about being agile. But no dice. We had to have a complete plan for all the features currently supported (some of them of dubious value, many of them incomplete and inconsistent in specification), and it had to come with a specific timetable. Under protest, the team and I worked hard to produce high-level estimates, and ended up basically guessing that the thing would take six months. No dice, it had to be delivered in three. I made a suggestion along the lines of the strategy suggested in the article -- we could identify a subset of features that the team would be comfortable could be delivered within the deadline. It would probably be stronger, I argued, since it wasn't clear that all the added weight added much value. No dice: everything was priority one. I pointed out something like, "you can't fight the laws of physics". I apparently gave the impression that we'd get it done anyway, though my memory of the conversation was that I was sternly disagreeable. Our team goes ahead and starts implementing items from a value-prioritized backlog. Fast forward three months, and we have a working system that supports the most important use-cases. We considered it past ready to ship, knowing we'd need to keep iterating. The response from management, predictably, was frustration at the missing features, despite the advance warnings. Management goes into "high-pressure" mode, and for the next three months I do my best to keep the team insulated. After more or less six months of total development time, we finally replace the prior component. All the users agree that it has far fewer bugs. I and many of my team members grumble that it has far too many, on account of the fact that we weren't given the chance to ship the minimum viable product when it was ready. I'm not really sure what the moral of the story is.
- tmikaeld 7y ago"I'm not really sure what the moral of the story is. " To me, it's - never waiver from your experience/gut estimates and a no is a no.
- sjogress 7y agoIncredibly hard to do when you have the full weight of management on your shoulders. Especially if they back you into a corner in a meeting where you are the only technologist facing a whole hierarchy of management types.
- marapuru 7y agoI completely agree on how hard this is. However, I've built the best relationships with managers where I kept my foot down once in a while. I guess they like the struggle and the honesty in the end.
- tmikaeld 7y agoI've run a business longer than a decade so I know how hard it is and I've burned out several times over. But I just don't give up and a no has certainly become a NO.
- nautilus12 7y agoI feel like this is why lead technologists turn over so often. Because they say the harsh reality that no one wants to hear, and no one cares that they are right, they just care that they are not delivering to their expectations. Maybe the issue is the expectation.
- serpix 7y agoDid it result in burnouts? I was part of such a mess and two out of four team members got burnout, myself one of them. It did teach me to get the fuck out early the next time I feel the same emotions of inadequacy and pressure on me. And in the very next project after a very sudden ramp up of backlog and increased pressure I looked in the mirror and got out, cleanly and without burnout or bad feelings.
- knocte 7y agoThis post is assuming that all startup founders are non-technical people...
- ryan-allen 7y agoMany are, you just don't hear about them, because, well... yeah.
- mrfredward 7y agoHonestly, I've seen highly technical managers make this mistake worse than the non-technical ones. Non-technical people usually ask sincere questions before shoving an unrealistic deadline in your face. Technical managers may have already told someone in the C-suite "ohh yeah, that's easy" before talking to the guy who has to do the work. I've had ex-programmer managers say "I just want you to add this graph to the app. I could do it in 20 lines of python." In reality, the request is more like "Add in this plot which is the result of a long running calculation. The calculation has to run in a background thread so the app is still responsive, even though the whole program has been architected with a single-threaded design. It needs some mechanism in the UI to indicate it is making progress. We'll also want a way to cancel the task. Half the datasets are using a different sign convention, so you'll need to automatically handle that. Actually, the data is polluted with garbage, you'll need to spend an unknown amount of time debugging a legacy system to figure out where the bad inputs are coming from in order to understand what can be done to filter that out...so we can have this plot by end of day, right?" Apologies for ranting about my "internal software" days.
- quantified 7y agoFrom my experience (3 technical co-foundings, 1 additional top-tier startup, other companies) it’s rare for management to incorporate our estimates. I’ve been coached by a VP Eng on how to blow off actual estimating, in that top-tier startup. Sometimes, it may be quasi-rational in that the apparent business constraints just don’t care about the estimates, in which case, yeah, just cut to the deadline and budget and we’ll see if we can do anything that isn’t embarrassing. Management usually dictates rather than engages, that’s how you get to be CEO in the first place. “We” are usually not at all clueless. “They” tend to be.
- yachtman 7y agoThe original title was closer to what you are alluding to here. Idk why but hacker news changed it to say "We"
- Mathnerd314 7y agoIt's part of the line before the picture of "the The Parsimonious Yachtman".
- dang 7y agoOn HN, moderators edit titles that are misleading or linkbait, as the site guidelines ask: "Please use the original title, unless it is misleading or linkbait" (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html). It's a linkbait trope to use "you" in a title, because it grabs attention whether the topic has anything to do with the reader or not. That's why "you" is headline writers' favorite pronoun. Combining it with a pejorative ("you have no idea") makes it even more sensational. That's definitely the sort of title we edit. When we edit a title, we look for a representative phrase in the article itself that expresses its point in a more neutral way. That's what a moderator did in this case. The language comes from the article's own summary of itself: "After all of these years, I finally came to one simple conclusion. With all due respect: we are completely clueless about how long things should take." Reading the article text to find how it states its own conclusion, removing any residual linkbait (such as the superlative "completely"), and making that the title instead is the best way we've found to correct titles that break the site guidelines.
- brianpgordon 7y agoFantastic article. The diatribe against estimation reminds me of this old article from 1996: https://web.archive.org/web/20140604112011/http://www.thomsett.com.au/library/item/estimation-games https://web.archive.org/web/20140604112011/http://www.thomse... > It is our belief that over the 30 plus years of commercial computing has developed a series of sophisticated political games that have become a replacement for estimation as a formal process. Unrelated: can we not do this thing with the whole left side of the screen being one static image? It's really distracting.
- yachtman 7y ago> can we not do this thing with the whole left side of the screen being one static image? It's really distracting. Agreed, ive been meaning to rework this
- niknikson 7y agoTimelines kill me. When I'm half way through a project I can tell you exactly when it's going to be done. Or I can plan the shit out of it upfront and give you an accurate timeline but that doesn't fly because you need the timeline upfront. I like working - can't I just work till it's properly done?
- dropit_sphere 7y agoThis sort of thing is why engineers are paid way too much, and way too little, and why so many see the profession as a young man's game. You're just not the master of your own fate. Sure, the computers are predictable (for some value of "predictable"...) but management is not---even if they try to be. I've been mentally moving away from being paid to write software. I can see writing it for my own use, even professional use---I just don't want to be on the hook for ever-changing requirements, decided by people who are often kind, but not, in the end, competent. The best managers understand this and will give you leeway, but this is not a sustainable, repeatable thing. It lives and dies on one relationship. I wonder sometimes: what if we just all stopped writing software one day, and started just using it? Writing software is a bad deal in a lot of ways---hard, socially isolating, etc---while using it is amazing---the computer does the work for you! Don't get me wrong, I spent an hour at work today presenting on Lisp macros and loved every minute of it. But a dev career, for many, means a capped income and a razor's edge of apparent competence.
- kqr 7y ago> I wonder sometimes: what if we just all stopped writing software one day, and started just using it? Writing software is a bad deal in a lot of ways---hard, socially isolating, etc---while using it is amazing---the computer does the work for you! What's the difference, though? It's insane how much of my work the computer does when I write code these days.
- dropit_sphere 7y agoThe difference is it never actually gives you more time. The expectations increase in lockstep with the efficiency.
- hvidgaard 7y ago> he best managers understand this and will give you leeway, but this is not a sustainable, repeatable thing. It lives and dies on one relationship. This is one of the reasons you often hears about an entire team moving to a new company. They have a dynamic that works, and they do not want to risk it.
- Uptrenda 7y agoI'm not sure how to avoid answering these loaded questions though. In my experience trying to 'negotiate' with people like that: they will just ask you the same question in 5 different ways and it will eventually get so uncomfortable you'll tell them anything to move on. Another option might be to never do business with cash-starved companies. Founders and execs will be constantly worried about running out of money, acquiring customers, and pulling off miracles, and all that stress will trickle down on you probably for no real benefits. A key question to ask an employer then is how much 'run way' / money they have left? Or whether they have a revenue source. It seems fairly probing, but realistically, not everyone wants to invest heavily in a company that might not even be around in 6 months.
- jimbob45 7y agoOn the flipside, necessity is the mother of invention. How many behemoth companies do we see now that sit on their hands and don't bother innovating at all? There's surely a balance to strike here between desperate and panicking and fat and lazy.
- jdc 7y agoI think what we're looking for here are healthy buffers. Otherwise the engineer is relying on (possibly blind) trust that management won't try to "motivate" him/her into performing miracles (ie. breaking the project management triangle). https://en.wikipedia.org/wiki/Project_management_triangle https://en.wikipedia.org/wiki/Project_management_triangle
- perlgeek 7y ago> I'm not sure how to avoid answering these loaded questions though. In my experience trying to 'negotiate' with people like that: they will just ask you the same question in 5 different ways and it will eventually get so uncomfortable you'll tell them anything to move on. Once you realize what's going on, you can turn that into a game. Do you know the game where somebody asks you a bunch of questions, and they you allowed to answer with anything except "yes" or "no"? It's very hard to do, because it's such an ingrained habit. If you really want to not give an estimate, make it a game to not give one. But, give them something else instead. Work out a bunch of questions about uses cases / data volume / whatever, and say "if those were answered, we could build a prototype in a few days that would let us make a more reliable estimate" or something like that. Another comment: coming up with good estimates is work. The other day somebody asked me if I could come up with a rough estimate for a (poorly specified, IMHO) project, and my answer was: no, I don't have time. If you need it anyway, formulate it as a task in Jira, so that it gets prioritized along with all my other work. (Fun fact: we estimate our tasks in story points, so then we estimated how much time it would take us to come up with an estimate... :D )
- JMTQp8lwXL 7y agoAt certain points, this article touched on the commoditization of engineers and engineering skills. Beyond extremely simple things, it's never going to happen. Especially as you build a company and end up with a few monoliths through various acquisitions, all while concurrently hundreds of microservices. Things get too intricate and too hairy between systems. There's no amount of handwaving that will convince me that machines will replace Software Engineers anytime soon. Solving valuable, enterprise-scale problems will never be as simple as a Wix drag-n-drop solution.
- Fr0styMatt88 7y agoI'm not so sure. I think this is something that's happening right under our noses -- it's just easy to miss it if you're not looking at the right thing. Software engineers aren't being commoditized by being replaced by machines that write software. They're being commoditized by their own frameworks, libraries and tools. Take the game industry as an example. Twenty years ago, your game company needed a big team of software engineers employed to write a game engine with advanced graphics capabilities (let's assume you want advanced graphics). Today, a single developer can just download Unity or Unreal Engine and have at least the technology available to them immediately (art is different but in many ways similar; automation and process improvements are coming for those jobs too, I'm sure). So you don't need the same number of engineers for the same result. Sure, you have a big team of engineers employed at Unity Technologies or Epic Games, but that's now a shared resource. That employment is no longer duplicated at the companies that decide to use those engines. Another example is the push for 'DevOps' and 'Cloud'. Think of all those system administrator jobs and IT departments being made smaller because now you can just spin up a server on AWS or have your CI infrastructure managed by BitBucket.
- heavenlyblue 7y agoSo who is it building the next Unreal Engine?
- 0x445442 7y ago> Another example is the push for 'DevOps' and 'Cloud' It's been my experience that delivering business value is taking longer because of this, not less. It's hubris to believe that a single person can be competent enough in all these domains to replace multiple people who focus on specific domains. By distilling DBAs, Configuration Management Engineers, System Administrators and Software Developers into single people business is getting shittier products less frequently which incur more operational overhead.
- yuoiyuio_UYO 7y agoOne of the most painful types of people to work with is the CEO / founder who got lucky with a product early and came to believe they're a "product guy". As in: My early product hit whatever trend / wave / need therefore I must be a product guy (and not just lucky). The product guy has a preternatural ability to understand what the masses want. Watching them work -- witnessing their process -- is something to behold. They will steer products in a direction regardless of cost, complexity of likely outcome. The outcome, quite often, is to tank their company. Since they don't understand why they were successful in the first place, it's very likely that their success won't last. But if you're along for the ride, wow, expect the following: 1. You're the greatest (available) engineer we've ever encountered, building super-complicated XYZ is going to take this company to the next level! 2. This is taking much longer than expected and isn't matching up with our expectations but I'm 100% sure of my vision because I'm a product guy. 3. We're running out of money (because the market conditions that gave us early success have changed) and super-complicated XYZ isn't going to rescue us -- because you're a worthless piece of shit of an engineer! See what happened there? They're sometimes hard to distinguish from a vanilla bullshit artist. The bullshit artist will tell you how well capitalized he is, tell you he only wants the best (meaning he thinks you're expensive) and then try to slowly whittle your sense of self worth down until you get "the offer": The offer is game-changing, life-altering for you: Instead of continuing to pay you with money, they're going to start paying you with magic pixie dust. The magic pixie dust will make you rich "when everything comes together." When you tell bullshit artist that you don't work for magic pixie dust, that's when you learn that, in fact, you're a worthless piece of shit of an engineer. I actually respect the bullshit artist more: They're bullshitting other people but they know they're full of shit. Product guys, depressingly, bullshit themselves.
- JTbane 7y agoMakes me glad I'm not a product guy. As an engineer you only risk your own hide or the work you do. As a product manager, you risk the entire enterprise.
- friendlybus 7y agoIt baffles me someone can be a product guy without understanding the market. Surely where, how & why the product sits where it does in the marketplace is intrinsic to the value of the product to the customer to begin with.
- tobr 7y agoThe author illustrates how easy it is to be clueless about how long things should take with his parenthetical about how trivial it is to make a Wix website. Yes, you can get the “used sailboat” class of website in Wix in an hour, but that’s probably not what you need or had in mind. The fact that the tool seems to do so much of the work for you will arguably just make the expectations even more unreasonable.
- onion2k 7y agothat’s probably not what you need 99% of Wix users need a website that's just a big phone number and street address that gets indexed in Google. A "used sailboat" is exactly what they need.
- tobr 7y agoThat's only true if you assume that 99% of Wix users are Wix users because they correctly picked a simple tool for a simple need. But that's a tautology. The problem I describe arises when someone looks at Wix, gets the impression that it is a tool that magically makes it trivial to create a great website, and then assumes they can get the yacht they have in mind for the price of a used sailboat. Most of the work involved with creating a good website isn't technology, but rather strategy, writing, design, and other things where Wix won't help you or even gets in the way.
- verinus 7y agoor the are Wix users because thats all they want to pay for. If I tell sombody that a full custom site costs idk 20 times more than a Wix site they might not want to pay for that.
- anthonyoconnor 7y agoI’ve gotten plenty of requests like this: ‘we just need a rough estimate, we won’t hold you to it’. ‘Ok based on the 2 minute conversation we just had I think about 3 months’ ‘What! That seems way too long’ Other times I’ve been asked for estimates on features even though there is a hard deadline due to some external factor. I really fail to see the point of estimating anything when there is already a decided upon end date. Anyway I usually point out that they will need to just put the features in order of importance and I’ll work down the list. And they should start thinking about what can be cut. This usually leads to protests of ‘we have already cut everything we can’. But I have to laugh as we get closer to the deadline that suddenly not every feature is as important as was originally thought and magically get cut.
- serpix 7y agoThen when the deadline does whoosh by - nothing changes and the sun rises just as always. It turns out very many deadlines are not in fact hard, they were just thrown on the table in some meeting and everybody starts to act like it is the end of the world if that date is passed.
- anthonyoconnor 7y agoVery true. Plenty of projects I've worked on have had immovable dates that suddenly become movable once you get close enough to them. On the flipside though I do think a deadline is necessary for everybody involved, developers and managers alike. It really helps to limit feature creep. And it forces people to think about what they really want or need.
- hvidgaard 7y agoI always talk them into making an ordering, and it's not difficult. If my managers fail they escalate to it to me. It usually goes something like this: Me: You need to prioritize the items. PO: I cannot, they're all important. Me: If you do not, we will make them by the order we want, possibly coin flip, but probably in order from easiest to hardest. PO: Fair enough, you get them all done anyway. Me: That is not a given and you know that, but I will send you an email for confirmation that any of them can be dropped to meet the deadline, okay? PO: Hold on, can I at least pick a subset that you know will be done? Me: Sure, and don't stop until you have roughly 3 equally sized, by estimate, categories: Must-have, Ought-to-have, Nice-to-have. By the time we're heading into the "Ought-to-have" I tell them to do it again. I fear that I some day might be in a position where I do not have the weight to do this, but as it stands right now, not a single developer produce a line of code if someone waltzes in and tries to decide both scope and deadline.
- redcat7 7y agomost managers is drug&alcohol infused money starving 10 year olds with no fucking idea what they are doing and knowing only how to abuse people like their fucking abusing power figures called parents
- Mathnerd314 7y agoThe Content-Type header of the article is set to the empty string. It still renders, but I'm guessing the author hand-coded a website and screwed it up somehow. He probably should have used one of those Wix-style solutions he mentions in the article.
- deleted 7y ago[deleted]
- socialist_coder 7y agoGreat article. This is very true for game development as well. But serious question - for live products especially, you do need to have some sort of schedule where you are launching new features every X weeks. So it's important to know how long your features will take, so you can have a constant cadence of updates. Plus a lot of times you will have marketing initiatives or other things that need to be coordinated with your releases. So my point is that you cannot just remove estimates. There is a need for knowing when the current sprint / feature will be completed, and being somewhat accurate about it. I do really like the point about re-framing the conversation to start by asking the manager how long they want the engineering team to spend on the new feature. That will definitely change the dynamic and hopefully should encourage a conversation about what is realistic to do in the timeframe that the manager has in mind, and how the feature needs to change in order to achieve it. But after that, the engineer still needs to go through and create estimates to make sure what they just agreed on is actually possible, and then those same estimates are necessary to plan out the development to make sure you are on track. So yeah, you can never really remove estimates. Am I wrong?
- Vinnl 7y ago> you do need to have some sort of schedule where you are launching new features every X weeks. So it's important to know how long your features will take, so you can have a constant cadence of updates. The only way you can have new features every X weeks, is if those features take no more than X weeks to develop. You can estimate a new feature to take 2X to complete, but that would still mean it can't be released "in time". Of course, you might still want to have a regular cadence of feature announcements, or at least be able to plan them in advance. But I feel like the best way to do that is to decouple finishing a feature from releasing it. Estimates are primarily useful in deciding what tasks to pick up first. Luckily, that usually needn't be that exact - you don't need to know to the hour how long some development is going to take, just how much faster one thing will roughly be compared to the other. A manager can then decide whether it's worth it to risk picking up a larger task that might provide substantially more benefit than the smaller task.
- socialist_coder 7y ago
- jayd16 7y agoIts honestly so so nice when you have a project manager that understands being Agile(tm) is about the schedules and deliverable reacting to shifting realities as well as the engineering team. I guess too many pms don't understand that they shouldn't stick their neck out with what they don't have yet and paint themselves into a corner with promised deadlines.
- arichard123 7y agoMy understanding is that it is harder for engineers to estimate how long a job takes, than to do the job. That is to say, that the complexity of doing a time estimation task is higher than the complexity of task you are estimating. I read this in the book "Making Software: What really works and Why we believe it" [0]. The assumption in the article is that an engineer can produce reliable estimates for complex work, and appropriately guide their managers, and don't think that's true. [0] https://www.amazon.co.uk/Making-Software-Really-Works-Believe/dp/0596808321/ref=sr_1_1?keywords=making+software&qid=1569398896&s=gateway&sr=8-1 https://www.amazon.co.uk/Making-Software-Really-Works-Believ...
- hyperpallium 7y agoI recently estimated a task would take 30 minutes. It took 40 minutes to do that estimate. But, TBF, several important decisions were made during the process of estimating, such as what should be excluded from the task, basic organization, and some research. BTW the task being estimated was doing time estimates for a project (which came out to be 2-3 weeks).
- duxup 7y agoOften it seems you take the time to properly evaluate the code and changes .... often it seems you are nearly done.
- basilgohar 7y agoThis is often overlooked and undervalued — upfront time spent on understanding the problem and designing before getting down to the actual work ends-up saving a lot of actual time. But most people don't have patience for that and think it is wasted time.
- duxup 7y agoYeah I've only been coding professionally for a year now and I do SO MUCH MORE reading/researching, planning, annotating, and pseudo coding than I ever did when I started. The outcomes are so much more predictable / better quality.
- jevgeni 7y agoMy usual way of doing this, is by spitballing an estimate with the team, multiplying it by 2.5 or 3, inventing some meaningless milestones and gantt-charts and presenting it to an outraged management. Then we haggle down to something realistic + a thin buffer. The management has a sense of control, we have realistic deadlines. Caveat: I work in the financial industry, where the complexity of writing a compiler and whipping up a Tableau dashboard are perceived to be equal.
- sbhn 7y agoI only hire people who can reliably estimate. How can you call yourself a pythonista if you cant reliably estimate. Incompetent estimators have no place in my organisation.
- crimsonalucard 7y agoThese estimators don't exist. Such a reliable estimator has more incentive to use his ability to estimate tech stock prices and become rich rather then work for you. In fact hearing this line, no one will work for you. Frame it and make it your motto, see what happens.
- perilunar 7y agoThey sort of do though. As hyperpallium alluded to above, the trick is to vastly overestimate the time required, then let Parkinson's law take up the slack.
- crimsonalucard 7y agoDoesn't that prove my point? They use overestimation to hide the unpredictable nature of the project itself.
- triggercut 7y agoEstimating knowledge work is always more unwieldy, and in this context you are often working towards outcomes which ultimately have no precedent (although they may be made up of known components). It is also more difficult to initially assess the skills fit of candidates for knowledge based work, especially those that require creative problem solving, and unlike other engineering disciplines past outputs are opaque and hard to rely on as simple markers of past performance. For a project, in order to produce a good estimate you need to understand scope, then align it with precedent, adjust for your resources and productivity profile and then view all of that through a risk lens to set probable outcome ranges. For a programme, in order to produce a good estimate you need to understand and manage the risks, constraints and dependencies across all your projects and ensure that the projected benefits (both hard and soft) are still net positive, meaning the investment makes sense. From my observations at least it doesn't look like the idea of development as "investment" in a product or service is very common. I'm assuming because time to market is often the ultimate driver rather than cost, in which case, congratulations, you should increase your costs on more numerous and productive resources whilst aligning your strategy and risks to iterate on smaller scopes faster so that dead ends can be quickly parked. The problem isn't so much the estimation process as it is more generally poor portfolio/programme governance and management practices and more specifically a lack of risk management and understanding of contingency at those levels. I find IT, and Software Development more particularly, to be some of the worst offenders, but that is because the risk profiles of such projects are vastly different to the risk profiles of other types of work. The productivity of your resources is difficult to discern and a lack of precedent for similar-enough projects and knowing what their variables were, all meaning you really can't produce a reliable estimate with incomplete information.
- hyperpallium 7y agoI've always had strangely accurate time estimates. The basic idea is to divide up the work, and make a fair estimate each part, with special attention to unknowns (which get much more time). Then double everything. The individual task estimates are often out, but the overall estimate is close... as if, with a population of tasks, there's regression to an accurately estimated mean. (An alternative explanation is that I over-estimate, and Parkinsonianly, work expands to fill time.)
- triggercut 7y agoOr you've baked in the appropriate level of risk for the work at hand. You might want to look into Reference Class Forecasting. Essentially an "outside view" of past performance which is added to provide an uplift adjustment to some measure of your estimate (time, cost, benefit).
- yitchelle 7y agoSome of my colleagues would 3x their estimates and come in ahead of the schedule, looking like heros.
- lostdog 7y agoGreat article. When I worked at $OldJob, the leadership wanted an unsolved, research-grade problem solved in a few months. They demanded estimates, then refused to accept estimates beyond their timeline, then tried to hold people "accountable" for missing the estimates. Of course it was a mess, and of course we failed to hit our goals, and of course many many people burned out. But I noticed that some people handled the situation fine. They stayed on management's good side, even though they were failing to deliver along with the rest of us. I will try to distill what I observed them do: 1) They did not fight on the estimates. If a manager forced them into a certain timeline, they registered their disagreement and just accepted the new timeline. I think they realized that fighting would just make the manager judge them less capable. (Pick your battles, eh?). 2) When the schedule slipped, they would communicate it in a way that made them seem more competent instead of less. The explanation usually had three parts: unforeseen events kept us from hitting the deadline, we accomplished some great things in the meantime, and here's why we're in great shape to hit the next deadline! For example: "When we made this schedule, we did not realize that AWS nodes were so unreliable! Despite this, our team has made incredible progress on implementing a fast method of storage and solid compression! We have reworked the schedule to reflect this new information, and we are already on track for the first milestone!" It's possible that this trick just worked for the specific situation at $OldJob, but I really enjoyed learning it. They seemed to understand that certain explicit rules were not important, but that other unspoken rules needed to be followed. Are accurate estimates important? It depends on the situation! Sometimes wrong estimates can be more valuable than correct ones. Construction companies give wrong estimates all the time in order to win projects. Is staying on good terms with your boss important? Yes! Even if they are total shitbags, being adversarial won't help, only leaving will. These people demonstrated that there are often ways to fix a bad situation by breaking some explicit rules and carefully following implicit ones, and I wish I had the acuity to see these possibilities on my own.
- mcv 7y agoStakeholder management is an important but often forgotten art. Keep them up to date on what they can expect. Don't fight them, just inform them. If a deadline is unrealistic, don't fight it, just let them know you cannot commit to that deadline with any certainty. Once it's clear you're not going to meet that deadline, let them know in advance.
- djmips 7y agoWhen I was a young programmer in the early nineties, an old IBM programmer told me 'Don't accept the invitation to fail.' Good advice but what if the goal posts are moved. Best then to keep positive and communicate early and often about how the changes will affect the schedule. Work together as a team and get your manager to help you and that'll go a long way rather than keeping 'obvious' things to yourself and catching management off guard when work slips.
- TickMark 7y agoGood article. I thought about this problem quite a bit. I've been on both sides of the question. I thought I was brilliant when I came up with the solution of fixing the time frame and estimating the work that can be done in that time frame. Turns out I came up with sprints, 60 years after they were invented. The fun solution for this would be to give 'hit dice' estimates for tasks. Assign type of dice and number of them to each eastimated. How long will this take? About 1d20 days. Nobody will be happy with this, but it is the most realistic one. Cause tasks do have that variability to them. The wises thing said here is: "the reality is that if you can make a probabilistically accurate estimate, then it's likely that the task should have been automated by some other means already. " Is there an answer to this problem? Maybe abandon long term estimates entirely. Having really short term estimates, with frequent updates.
- crimsonalucard 7y agoDoesn't address how highly technical people dealing with new systems are also wildly innaccurate in their estimates. It's like a expert in tech, business and computer science trying to predict stock prices. I mean that's the expert of experts failing.
- nstart 7y ago> If engineers stop giving estimates for their work and simply ask for deadlines then it changes the dynamic of the conversation. If there's a single line you want to take away from this wonderful essay, it would be this. Parkinson's law usually takes care of the rest once the resources have been agreed upon.
- perilunar 7y agoHow good are coracles though? Not much good as a yacht, but then you can't carry a yacht on your back and walk it upstream can you? https://en.wikipedia.org/wiki/Coracle https://en.wikipedia.org/wiki/Coracle
- 11thEarlOfMar 7y agoIt strikes me that the issue is, to some degree, rooted in the transfer of the vision from one person to another. I have struggled with this as well, on the side of the 'visionary' rather than the developers. A quick search for 'how to communicate what you want with software developers' turned up what I expect is at least modestly helpful: https://www.entrepreneur.com/article/224816 https://www.entrepreneur.com/article/224816 (2012) but I'd venture that the topic really is worthy of much more in depth treatment. A key element missing at least from that post is: How to confirm that the developer does understand what the vision is? If that can be confirmed, they would then be in a position to modulate the vision vs. reality, and, even contribute some of their own creativity. I suspect that this modulation frequently occurs with a start from misunderstanding. The divergence can be dramatic.
- deleted 7y ago[deleted]
- nottorp 7y agoNice article, but what's with the web page design? On desktop, the left half of the screen is just the blog title, while the article is squeezed in the right half. Safari on Mojave if it matters.
- Sean1708 7y agoAnd it breaks keyboard scrolling (at least on Firefox) which pisses me off more than anything else about it.
- deleted 7y ago[deleted]
- johnfactorial 7y agoLOL I came here to see if someone else had noticed it's the worst mobile-first layout of all time. At least it's mobile-first I guess. :)
- w8rbt 7y agoIf managers come to developers with problems that are NP complete, no matter how well they transfer the idea/vision, or set time expectations, the problem cannot be solved efficiently. And, unless the developers have been trained in CS/Math they may not even understand that what they are being asked to do cannot be done. For example, say a manager has an idea to find the largest group of friends in a massive social network. He wants a developer to write an app for that and has a 20K budget and 2 months. You could not write this app with 10 times that budget or time. How can you determine which group of friends is the largest? https://en.wikipedia.org/wiki/Clique_problem https://en.wikipedia.org/wiki/Clique_problem https://en.wikipedia.org/wiki/List_of_NP-complete_problems https://en.wikipedia.org/wiki/List_of_NP-complete_problems
- jacobush 7y agoThere is also a sort of "corporate interaction" NP complete game. It's hard making department- or corporate-wide changes.
- ansgri 7y agoThe thing is, there are almost no cases where a problem must be solved in some strict mathematical sense. Huge part of project management is the problem reformulation to fit the existing solution to the initial requirement: what do you mean it is not a clique? it's still a group of friends, 80% of which know all others, with empirically tested chance less than 5% that any person knows less than half of others.
- papito 7y agoAgile has his concept of "horizon of predictability". Basically, you are semi-accurate within the two-week range. After that, the accuracy nosedives. At 6 months, you could be off by, well, 6 months.
- JJMcJ 7y agoHow it often works: Manager says I want an A that does B? How long? Most of the time estimate is N days, actual is N +/- 20%. Sometimes it's N days, and you get lucky it's N/3. Instead of credit, you now have a reputation of padding estimates. Sometimes it's N days, and you go oops, and it's 10*N. Now you are a slow incompetent. This is why estimation is such a no-win for developers. For areas that come closer like construction, they have centuries of prior experience and often the contractors are big enough to push back. To use the example of this article, go to a boatyard, and say I want to 150 foot yacht with a hot tub and a 20 seat theater, and my budget is $10,000. You would get laughed out of the office.
- leesec 7y agoWhen's the last time a construction job was done on time? Or even within +/- 20%?
- dagw 7y agoHappens every day all over the world. It's just such a mundane event no one writes newspaper articles about it.
- adrianN 7y agoSo, just like software projects.
- marcosdumay 7y agoWell, I've never heard about it. Both of my parents worked their entire lives on the construction industry. It's a running joke that this never happens.
- phonebanshee 7y agoSoftware people tend to assume that the rest of the world is far more organized than it actually is. In particular, they seem to think that construction projects tend to be well-oiled machines with perfectly followed specs written in advance.
- SamuelAdams 7y agoReminds me of this article [1] from Michael Wolfe. I've sent this to several PM's over the years and they've all begrudgingly agreed with me once I applied this to the current project. The beer probably helped too :D [1]: https://www.quora.com/Engineering-Management/Why-are-software-development-task-estimations-regularly-off-by-a-factor-of-2-3/answer/Michael-Wolfe https://www.quora.com/Engineering-Management/Why-are-softwar...
- dogman1233 7y agothis is basically the whole point of correctly applied Scrum
- alexhutcheson 7y agoAll the research indicates that software developers are terrible at estimating time, but are fairly accurate at estimating relative effort. The only reliable method is to break down projects into the smallest deliverable "chunks" that you can, and then estimate the relative effort of those "chunks" compared to actual, concrete tasks that the team has completed in the past. After doing this for a while, you can use these relative effort estimates to project completion dates for new projects. This is what agile "story points" are supposed to represent, but many teams unfortunately end up with things like "2 points = 1 week", which just doesn't work. If you use this method, it's absolutely critical that the team always focus on relative effort, and not on time. If you want it to continue to work, you also have to be very careful how you communicate with team members regarding their progress and estimated completion dates - don't tell them that their component is "late" - this will make them think in terms of time next time you're estimating something, and will ruin your estimates.
- nautilus12 7y agoI think the author is trying to say that we should call a spade a spade and say that by story points they really mean time anyway so might as well just ask for a deadline. I've never worked at a company where points didn't have an equivalent time translation.
- iNate2000 7y agoThe article didn't mention points at all. It said "agile", but only, like, once. It also didn't mention Evidence-Based Scheduling[1]. Also, the evidence-based-scheduling article doesn't mention points either. [1]https://www.joelonsoftware.com/2007/10/26/evidence-based-scheduling/ https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
- ldng 7y agoExcept it fails at step 1. If you have to breakdown a “Ajax photo editor” into 16 hour max chunks, as stated, you need a detailed design. And that takes times in itself. The problem is, often times, the manager want the estimate before you've taken the design time because, to them, it should be included in the estimate.
- samnwa 7y agoThat's because developers shouldn't be making estimates in isolation. They only know what they know about themselves. People who watch developers across many projects can better understand the 'likeness' of similar projects, similar inputs (in terms of developers), and then talk to developers about the differences of a given project to come to a better estimate.
- noobiemcfoob 7y agoWhy does everyone accept time estimates are accurate outside of software? Has no one hired a contractor to build out a bathroom, paint a few walls? Commissioned an artist for a sculpture? It's my experience that anything that can run over schedule will...why do developers get so much flak for it?
- colordrops 7y agoI work at a company with dozens of hardware and analysis teams, and software is one small group making up not even 5% of the company, but we are never the bottleneck. Our estimates are usually the most accurate of anyone. If we have a problem in a design, we don't have an entire supply chain to modify.
- bigYahnz 7y agoTry aftermarket engine / car builds. 2 weeks routinely turns into 6 months or longer. I wish I had that sort of leeway.
- tboyd47 7y ago> I found myself absolutely astonished that tech founders could be so clueless as to assume that a simple rest api integration should take the same amount of time as a real time transactional distributed ward’s clustering implementation for peta bytes of data, or a highly available complex distributed metastore. I would love to know the context behind this. Beautiful website by the way.
- tluyben2 7y agoIn my experience the deadlines are not a big problem; the communication about it is though. I worked with PMs that try to attack the issue by shooting into micromanagement mode (symptom: the project management system gets filled up with literally 1000s of tiny points with random deadlines in order to try to manage the fact that they don't get how software dev works); this drives most devs mental up to a point they burn out. There are only so many ' why is task xyz3835433 not done? It was set to be done yesterday?' 'Yes, because it was set by you and task xyz3835433 cannot be done in that time' 'Why did you not tell me?' 'Because I do not have time to respond to 9000 points every day; they are not relevant to my work' etc. Usually these PMs come from another discipline like management of marketing or PR campaigns, were breaking deadlines is not an option. But complexity there is far easier to manage and oversee and they do not understand (even after many metaphors) why our estimates are not spot on. They will communicate with the client (internal/external) in massive MS Project files and accompanying docs that no-one will ever read and berate the team for being so much off in the estimates. Then there is the other type of PM who will continuously manage back the client expectation. To the layman, this person seems to tell the client (internal or external) bad news on a daily basis; it'll take longer, cost more and you're getting less. It's the latter PM + team that actually will get the flowers and the cake on launch day and a have the happy devs that did not have to sleep under their desks while the former team will be near burn out, client unhappy even though they probably technically did deliver more (but also too late). Usually that former team won't get more money for it either... I have seen both in startups and fortune x companies; I have seen both as contractors and as internal teams. For the anecdotal part here; the ones with a PM like the first one in a product company internal setting, those companies all failed that I have seen/worked with. For contracting work, it can work, but it's a stressful and panicky way of working which often results in one-off contracts. You inevitably come to something like sprints and good, timely bad news talks. People who have the micromanagement type need for control are just not going to survive those as their struggle breaks down even inside a 2 week sprint period.
- wpietri 7y agoProject plans and estimates are usually fantasies made only to give executives a sense of control. For people who are actually trying to get something done in the world, I think the better thing to do is give them actual control via short-cycle processes and frequent delivery. At my last startup, we got very good at releasing early and often. What would have been projects elsewhere got broken down into very small releasable units; our average story size was under a day. Very occasionally my cofounder would have us ballpark estimate two different batches of work using arbitrary points, just so he could get a sense of the relative cost of two paths. But we never estimated in terms of dates. This worked for us because he really took advantage of the speed of iteration. Almost everything we built came with a question. In the next user test, would users understand it? Would they want to use it? Did they react as expected? When we sent some traffic to it, did people engage? Did the right people engage? Did they return to use it later, indicating value? Etc, etc. The answers to those questions would drive what we did next. Because our goal wasn't to build features, but to make things happen in the real world. And once you get used to continuous learning, it's obvious that planning too far in advance is wasteful. All of the brilliant ideas you had a month ago weren't based on what you learned in the last month. Eventually, sensible people learn to stop producing a lot of plans that never get fully used. And, of course, to stop asking for estimates on them.
- ralmeida 7y agoI understand that sometimes a project 'takes as long as it takes', but how does one deal with extrinsic deadlines then? For example, software that needs to be 'on the shelves' by the next Black Friday, or the next tax reporting season. Estimates can't predict the future, of course, but good management will know that 'make the deadline, no matter how' is not the only option - changing scope is another popular option, for example.
- cloverich 7y ago> but how does one deal with extrinsic deadlines then? Deliver working software one chunk at a time, however you have to break it down to make it so.
- kriro 7y agoI think time estimation can be learned to some degree. As an anecdote a couple of years ago I had some free time and my sister was renovating her home. I volunteered to tear down all the wallpapers (basically an entire apartment). I had no idea how long it would take so I took a kitchen timer, set it to 25 minutes and started chucking away. After a couple of these 25 minute runs, I could predict fairly well how long the task would take for a certain room. I also found it interesting that it was very easy to adapt the estimate to changing conditions. For example in the kitchen the wallpaper was soaked in fat and there were 4 layers glued over one another. After working on this for about 5 minutes to see how much harder it is (answer: a lot) I was able to estimate fairly accurately how long it would take. Granted this is for manual labor where you know exactly what will happen. Estimating more uncertain environments is a lot harder. Nevertheless, I have grown used to using this time-chunking approach for all sorts of new tasks. I think shorter timespans (10 minutes) work better for estimation but for actual work, 25 minute "sprints" are good.
- swiley 7y agoWhile I don't think estimates in software development are hard (unless you're asked to find an algorithm, that's impossible to estimate.) Manual labor and knowledge work (not counting repetitive, easily automated stuff) are pretty much completely different and it's dangerous to make analogies between the two.
- HeyLaughingBoy 7y agoWhat you're describing is known in Manufacturing Engineering as a Time Study: https://en.wikipedia.org/wiki/Time_and_motion_study https://en.wikipedia.org/wiki/Time_and_motion_study Basically someone stands around with a stopwatch and monitors a process a few times and then gets a feel for the average time the process takes. You can do the same thing in software with the caveat that you must be doing pretty much the same thing. e.g., if the last 5 times you had to build the basic framework for a CRUD program in Ruby it took 10 hours, then it's likely that the 6th time it will also take roughly 10 hours. However, doing the same thing in C means that the estimate goes flying out the window.
- 7y ago
- stephen 7y agoWow, lots of great opinions here I'll look forward to reading and learning from. My very brief take is: Estimates should be for "making the best decision now" and not judging performance. Those are orthogonal concerns. It's very fair for the business to need to weigh "well if X takes 10 days and Y takes 20 days, and X makes more money, of course we'll do X". That is what estimates are for. However, if X ends up taking 15 days, the business (CEO, product, even CTO depending on their closeness to the solution) shouldn't/can't decide if that is a performance issue--only the engineer/the engineer's manager/etc. can make that call. And maybe it is, maybe it isn't. Granted, pulling this off is really hard; tangentially I'm now working in the construction industry which has the very same problem: this home should take 9 months. It took 10! Who's fault is that? Well, that's the wrong question. The right question is how could we have known that delay sooner/better, and mitigate it if possible this time and more importantly next time. If you're interested in working on a "humane" project management system (I just made that term up and it's super early, so disclaimer/etc), reach out </shill>.
- blobs 7y agoI moved away from web development after many years of full-stack development. It almost totally destroyed the joy I had in programming. - happy Agile team? That alone will cost you 2 days of work per week due to meetings, planning etc.. - Javascript? No, it has to be Typescript for even the most futile websites nowadays, and yes, TS adds another 20% of workload - TDD, with a dedicated testing engineer?, no of course not, you do it yourself! Another 20% of added workload - A shitload of tooling from linters to bundlers and whatever else that always needs some attention - Deployement, done by a devops engineer? you must be joking right? That's also the work of the full-stack dev.. Now try to estimate how long it will take to implement a simple feature request from the PO? I always did my rough estimation, times 2. And even that was often not enough because all kinds of urgent issues needed to be resolved, so you're kind of lacking all the time which is a great recipe for burnout. I'm done with it.
- nickdandakis 7y ago- Don't default to "Agile" (or be fully "Agile" for that matter) - Don't default to Typescript (or any technology for that matter) - Don't default to TDD (or any dev methodology for that matter) - Don't default to linters and bundlers (or any intricate tooling for that matter) - Don't default to complicated infrastructure A friendly PSA that you can still, in the year of our lordt 2019, create a static site with an HTML file deployed to shared hosting over FTP. You could also create a dynamic site with some PHP thrown in there. Reach for the minimal process, tech, methodology, tools and infrastructure you need to get the next X pieces of work done, and complicate things only if the pros outweigh the cons. I'm not picking on you specifically blobs, I'm sure you walked into an organization that employed all these things with seemingly no good reason. The thing is, there's always a reason, it just might not be a great one! That said, if your goal is to estimate things more accurately, you might need to spend a little more time with your coworkers to define as many unknowns as possible.
- soulofmischief 7y ago> A friendly PSA that you can still, in the year of our lordt 2019, create a static site with an HTML file deployed to shared hosting over FTP. You could also create a dynamic site with some PHP thrown in there. Honestly that sounds like hell, we may live in a spiderweb of complexity but it's still a long way from the primitive web of the 90's which provided every conceivable kind of way to shoot yourself in the foot. The hidden issue is that new web products are becoming increasingly complex due to tight competition, but businesses somehow do not expect that increased complexity means there is a need for larger teams and budgets. All this cruft in the modern web build pipeline is stuff made by developers, for developers, to help us navigate the impossible tasks handed to us in impossible timeframes. Some of these additions are very welcome, but the proliferation of complexity in development in general is mostly due to teams and budgets and time estimates not scaling accordingly.
- taylodl 7y agoHofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law.
- swiley 7y agoIMO if you're familiar with the language and libraries it's pretty easy to predict how long writing some feature will take (provided there aren't bugs in anything you're working with, you don't get any weird emotional hangups etc. Those are fairly rare.) What's hard to predict (at least for me) is writing non trivial config files or using some new library or language or finding a new algorithm. Some of those things you might be able to estimate how long it might take to estimate the time it will take but other times you just have to say "I don't know" and time box it.
- hinkley 7y agoI’ve been participating in build pipeline work for quite a long time now, and fairly early on I noticed a pattern. If the build takes ten minutes on your local box then the tempo of code-build will tend to be 20 minutes, even when you are just fixing typos. This is partly why some people are obsessed with very short builds. Some said seven minutes was ideal to avoid the developer getting preempted. Then it was three. Now some shoot for one. I would bring this up and others would say they had noticed the same thing, but none of us knew why this phenomenon happens. Then it hit me: it’s just Hofstadler’s Law playing out in the small. If you think you have five minutes, you will start something that you estimate will be five minutes, but it will take you ten, either because you were wrong or you get distracted. There aren’t a lot of tasks that take one minute, and even if you’re wrong you only lose one minute.
- carapace 7y agoWeb designers: Thin sans-serif body font means you hate your readers' eyes. I'm grateful for "reader mode". - - - - There's a very interesting book, " Hollywood Secrets of Project Management Success" that details their system from an IT point of view. Here's a decent review: https://community.dynamics.com/nav/b/navigateintosuccess/posts/hollywood-secrets-of-project-management-success-58-a-review-of-a-sort https://community.dynamics.com/nav/b/navigateintosuccess/pos... > Inside this system lurks the biggest difference between the IT and Hollywood: movie industry exists for more than a hundred years already, they had enough time to develop and establish best practices and to prove them in practice to such an extent as to tell everyone: this is how you need to do movies; and everyone can trust it works, because it has worked for the whole industry for decades already. IT industry is very young, and exists for a few decades. (Although, IT is technically as old as the cuneiform-inscribed clay tablet, eh?) Specifically to this discussion, they (Hollywood) can set expectations reliably because have enough shared baseline experience of how long things take. One way to interpret this is to ask yourself, of some new change in process or technology, "Will this stabilize delivery times?" But to even begin to answer that kind of question you have to establish reasonable ways to map between work done and results delivered, and then set up and track metrics. (Which is easier said than done.) - - - - One problem unique to IT is the "interpretability" of our end products. Anyone can watch a movie, but most software requires at least some training to understand and use.
- cloverich 7y agoThere's no substitute for technically competent management, period. If you don't have it, you will be limited, and its something you just have to accept -- as in, make that a high priority question to ask about when interviewing for your next job. Further, its not that software developers are bad at estimating. Its that software estimation is only useful when taken as a very rough guide for strategic planning. You need to know if something will take 2 weeks or 2 years. You don't need to know if it will take 2 weeks or 4 weeks. If you think you do (as most shops I've worked do) -- you are micro managing your team. The sooner we start calling it that, the sooner we can all accepts its as much an anti-pattern in software development as it is every other facet of business. In short, estimate roughly. Agree to high level deliverables. Assert that technical management needs some control over strategic business decisions, and flexibility in implementation and timelines. If you can't get those things, temper your expectations, or look for a job with more competent leadership. The last point has become my guiding principle. Stop thinking you can "fix" an org from the bottom. You can only fix an org if they hire / promote you to do so, and empower you as such. That's what you ask for (or look for in your managers).
- shakezula 7y agoI dunno, some of the worst managers I’ve had have been great engineers and fully technically competent. It doesn’t always translate, and even during estimates or sprint planning I’ve seen them make these same mistakes.
- cloverich 7y agoI agree and should have clarified -- technical competence is _a_ requirement for good management, but you still also need to be(come) a good manager in the traditional sense. Its a different job than writing code and not everyone will be good at it (and arguably by definition, of the relatively small number of good engineers, even fewer will be good managers on top. I would expect quality technical managers to be relatively expensive).
- shakezula 7y ago
- massysett 7y agoThis is amazing. I work in a law office and have nothing to do with tech but just change the examples and he has just described my typical day that replays over and over in matters both large and small.
- nautilus12 7y agoI'm intruigued, could you elaborate more?
- whitebread 7y agoI have been having the exact same thoughts, except at a public accounting firm. A lot of the comments here describe what is typical in public accounting as well. It seems the moral of the story is that managerialism is hard to make work. Perhaps we should try something else?
- redcat7 7y agosorry for earlier post, a lot of stuff still hurts. i'm in poland. my type of manager would be a person who accepts me, gives me clear target and constraints and leaves alone, person who does his job, person who doesn't bullshit me and says the truth if we're in shit or not and realises that we are all on the same boat and we're also waist deep in shit so there is no point in fighting each other. on 16 IT companies I worked in, there was only 2 such people. that's 12.5%. shouting and batshit crazy behaviors are normal. someone here wrote that some things are cultural. a lot of stuff is cultural in poland. its a catholic fundamentalists country. so anyone who solves problems is a problem. people here had no type of french revolution or industrial revolution. basically its feudalism in modern dress. last guy i worked for was freestyling everything. he had no plan. he told me he uses his imagination and he tells people what he had imagined and they have to do it. i found out about this when i was trying to resolve communication issues that i had with him. he often imagined new stuff and forgot to tell me about it and during code reviews he blamed me for not doing what he wanted me to do. i ditched the guy because he didnt wanted to do anything with that problem. the guy sabotaged himself throught the whole time. sometimes he had some strange outbursts. there is a lot of such people in poland. a lot of office workplaces are like kindergartens. there's constant chaos. no plan. big ideas but nobody wants to wait. people who want things done asap usually are abusing others, not to mention that they are total morons cause abusing people takes time they dont have. most companies are micromanaged. thing that keeps me going is knowing that, as someone said here, this stuff is cultural and that there are some better places. although after all those companies i'm kinda like a dog from the impound. i dont know how to trust people. word nation is for me the smallest joke possible to say. i'm afraid of polish speaking people.
- heroHACK17 7y agoRelated article (albeit a more mathematical focused analysis of the same problem): https://erikbern.com/2019/04/15/why-software-projects-take-longer-than-you-think-a-statistical-model.html https://erikbern.com/2019/04/15/why-software-projects-take-l...
- the_watcher 7y agoI'm reading The Moon is a Harsh Mistress right now (fascinating, bizarre, and highly enjoyable thus far), and there's a moment where the main characters discuss a timeline described as "on the order of 50 years." This is still a fairly common phrasing, and in general, people around me generally take it mean "roughly" or "plus or minus a few," which is how one character interprets it. Another character then explains that its actual meaning is, paraphrased, "probably not less than 5 years or more than 500 years." While I had a vague awareness that "on the order of" was not mathematically equivalent to "roughly", I don't remember learning the concept in such a memorable way. Similarly, an order of magnitude are commonly understood as "a lot more than", when they also have a precise mathematical definition. I think that reason we are terrible at estimating the time something will take is that humans struggle to think in terms of timespans that are actually representative of the variance in an estimate, not to mention the fact that as additional variables are introduced, that variance will almost certainly increase (even if the midpoint decreases). I'm not sure of the causal direction here, but our misunderstanding of terms meant to succinctly express "somewhere between" sure doesn't help things. (side note: there's also a discussion about designing a revolutionary organization that really clarified my rudimentary understanding of circuits. The book is truly stunning at times.)
- alexnewman 7y agoEveryone on my team knows i can guess when they are going to finish usually to the day. They don't know how but it's pretty easy, measure biases
- ppeetteerr 7y agoThere is something to be said about pushing engineers to tighten their deadlines. It's not necessarily out of a desire to shift blame. Instead, it's often to encourage them to work hard for the same amount, and to think up novel solutions to problems where others might not have. In other words, it's an attempt at maximizing productivity. Start ups and some teams are running on legitimately small budgets and squeezing every moment of the process will net a large outcome if done consistently. Having said this, the idea that you can ask for something in 2 weeks when your eng estimated 4 weeks is ridiculous. I would automatically add 50% to my eng's estimates because their estimates are probably too ambitious to begin with (I fall to the same problem when estimating my own work). If you consistently get undercut in your estimates, get out! I've seen this behavior most often in marketing agencies and game companies. I would not work for these types of people regardless of the money they are paying.
- asperous 7y agoThis is talked about in the book "Peopleware", that states the opposite. Imposing tight deadlines often stresses people out, causing them to miss creative solutions that would save time. Instead they chose the obvious, longer, safe route. That's what the data in the book seemed to point to. Programmers with no deadlines were more productive then programmers put in the control group. However there might be data showing otherwise, I would be open to it.
- ppeetteerr 7y agoI have read that book and having worked in some companies, I can agree that tight, unrealistic deadlines are detrimental to outcomes. However, working in silicon valley, I can say that there may be need to push people to work more efficiently. It's a fine line.
- throwaway66920 7y agoI think the commonly claimed rules about adding increment your unit of time or multiplying by x are kind of childish and a wasteful. Estimating time to complete is difficult because individual items have high variance. If you have 3 items that are likely to take 2 hours +/- 2 hours, estimate 10 hours for the lot. Variance of the total goes down by a factor of n^(-1/2). Explain why you do this. Flag when variance is screwing you anyway.
- mgkimsal 7y agonot sure if it was mentioned, but I'll throw out recent other issue not often talked about - external dependencies/vendors. "integrate with external API" - had a project where there were several external data sources (financial service providers) to import on a regular basis. One had an actual API, commercial service, good docs - took a few weeks. The others are... 1. "hey, we'll send you a nightly file, except it's not always nightly, because it depends on someone running the job and if they're not here, you won't get it". 2. "here's our SOAP WSDL" - "this doesn't work" - "oh... try this other one" - "that doesn't work" - "try this one, but just don't use some of the endpoints cause they don't work" - "OK, but this doesn't really work either". Now... intersperse those sentence fragments with a minimum of 4 business days via email (sometimes going for a couple weeks because 'vacation' time). 3. X was working for 7 months, then stopped. "oh, we changed the file name and format of what we push to you yesterday". no warning, no documentation on what the change is. just... pissed off end users who are now saying "my data is wrong!" Figuring out how to take data from a file or SOAP or REST endpoint and process it - that's not terribly hard. Figuring out how to deal with more than half a dozen vendors who are not 'really' in the business of providing data, but do it half-assed anyway - there's no end to 'figuring it out', because it's a moving/changing target. I'm not naming any negative names but I'll mention that quovo.com was comparatively pleasant to work with - they're an actual commercial service. however, less than a year after we coverted a system to use them, they were bought out and some of the useful functionality seems to be sunsetted already. I'm not on that project directly anymore, but talk to some colleagues still involved in it. From the client's standpoint, it's all "integrate with external data providers". "You did one, the others can't be that hard". But each provider is a completely separate island of functionality, documentation, responsiveness and professionalism. For the record, no, you shouldn't be providing me with client SSNs as their identifiers (quovo didn't, but I'm surprised at others that do, in at least one case that's the only way they provide client identifier data at all).
- rramadass 7y agoPosting one more time; David Packard's (of HP fame) address on "How to be a Manager in a Technical Company" - https://gizmodo.com/the-hp-way-how-bill-hewlett-and-i-built-our-company-5634378 https://gizmodo.com/the-hp-way-how-bill-hewlett-and-i-built-... One thing i noticed in discussions like this one is the amount of excuses made for Management's/Manager's lack of Technical Proficiency. This has to stop. If you are in a Technical Domain and leading/managing Technical People you have to know the domain decently well. There is no other alternative. Else you are reducing the effectiveness of your Engineering Team and the Company as a whole to your level of incompetency. The resulting effects may not be visible immediately but sooner or later you will drag the company down to oblivion (eg. see what happened to the same HP).
- GrumpyNl 7y agoQuote from the article "Engineering is not getting simpler, its getting more and more complex, because we are solving harder and harder problems" I totally disagree with that , for mine work environment. We are over engineering stuff. It all has to be new and shinny. Lots of sites can still be build with a small framework, html5 and jquery.
- GoToRO 7y agoThe article completely misses the point. No amount of reasoning will change a manager's mind. They do not push people because they actually think it can be done faster, they push people because software developers are very weak and because they want THEIR project to be delivered as soon as possible so they can boast in front of the other managers.