11 ms·
Software Development Waste
- js8 5y agoAs somebody living in "Agile", I hate retrospectives. But: If our retrospective was that we actually go through a checklist of things in the 3rd column from the table, they could become really useful. Just a thought.
- Jtsummers 5y agohttps://en.wikipedia.org/wiki/Lean_software_development https://en.wikipedia.org/wiki/Lean_software_development It's frustrating to me, because it seems to be underutilized in the industry, but lean software development (as discussed by Mary Poppendieck) has been about exactly this idea for around 20 years. She mapped out the ideas from Lean to software development equivalents and mapped out ideas about what constituted "waste" in this field. These are exactly the kinds of things that people should've been looking at in retrospectives, but it's become deemphasized and is instead a shortened ritual with no useful effect.
- js8 5y agoI looked in the paper and they actually compare it to Lean, and expand up on it, exactly as you say.
- gravlaks 5y agoWhy do you hate them? I find them very useful. Everybody gets to voice their frustrations and what works, which we can learn from and improve upon. What if I our team has a concern that isn't mentioned in a checklist?
- fridif 5y agoWhile I really enjoy Agile and Lean development, my most recent employer had an extremely toxic culture where retrospectives were basically akin to the Hundred Flowers period of China. If you voiced frustration, you provided an O(1) way for them to find who to bully into quitting next. My current employer just lets me do literally whatever we want. A service is basically owned by a developer. And so far my customers absolutely love my work.
- MattGaiser 5y ago> which we can learn from and improve upon. Not OP, but on my past two teams and my present team, this part has never happened. It was/is just a box ticking exercise.
- pjmlp 5y agoNot the op, on most projects the improve upon part is mostly out of scope for the team, so it ends up being only about venting frustrations.
- ManlyBread 5y agoThe recent retrospectives I had were used against me in my yearly evaluation where it was pointed out that I "complain too much" and that it "looks bad". That's about it for my involvement in this process.
- mrslave 5y agoGotta love how they bank all these complaints for withdrawal at your annual bonus review instead of addressing them with you quickly and privately like an adult.
- MattGaiser 5y agoI once forgot something and several weeks later it ended up on my quarterly review. No reminder that it needed to be done (and it still needed doing) except for that one quarterly complaint.
- kqr 5y agoYou're attributing your hate all wrong if you blame the retrospectives for this, rather than bad management.
- Jensson 5y agoThat is normal management, dismissing it as "bad management" makes it sound as if it isn't widespread and that we shouldn't base our practices around working with such managers. Managers who don't penalize you for being negative are rare, even those who say they don't typically do it anyway.
- MattGaiser 5y agoEh, they are probably just focusing on what they can change. Changing managers is a hassle with a good chance of getting it wrong. I can clam up, say nothing, and work on other work during retro very easily.
- ahefner 5y agoYou are really making me appreciate that my line management has never had any involvement in "scrum ceremonies" including retrospectives.
- zelos 5y agoMaybe I've just been unlucky, but in my experience retrospectives are mostly virtually identical every time: nothing happens about the frustrations, so they become a complaint session instead of something positive.
- kqr 5y agoWhat can you do to change that?
- Jensson 5y agoOnly thing you can do when your manager doesn't do this job is say you will personally take on the extra responsibility for no extra pay and no reduction in other tasks and give most of the credit to your manager. Then if you are lucky all of this extra work might lead to a promotion far down the line, or it will go to the person who were more positive during meetings making the manager like him more and all your work fixing things got wasted.
- selfhoster11 5y agoRealistically, the only thing you can do in this situation is to switch teams and/or company if the problem comes from above.
- js8 5y agoWhat I don't like about our version is that we only focus on small portion of all things in the list (from the OP), mainly the Agile procedural issues. I think the purpose of having such a checklist is not to limit the discussion to other concerns, but rather serve as a reminder what all the different things can be a problem and should be discussed.
- pjmlp 5y agoIt is one hour to be used as break and vent on all things that went wrong and will hardly change due to reasons.
- mtc010170 5y agoI know the feeling. The goal is supposed to be to improve the process, but often too much time is spent going through the motions rather than meaningful discussion. In my experience, the value from retros is a function of the skill of the facilitator. Perhaps you could be a force for change on your team, by speaking bluntly about not spending enough time on that third column. Others might be receptive to it!
- catchmeifyoucan 5y agoI think not investing/knowing good tooling is another sink. Mainly setup and take down. For example, a program I recently learned about was “lerna”. If I had know about it a few months ago, it would have been less painful to manage all of our libraries. Another good example is deploying to serverless backends. Without the serverless framework, this was otherwise painful. And once we find out about these tools, there’s a ramp up cost to integrate them into the existing workflows
- js8 5y agoI just thought about this and it seems to be a paradox. I can't shake a feeling (since I started my career working with methods of the past) that programmers in the past were more productive than programmers today, despite all the talk about wonderful new tools and libraries. Let's say I can make a product in a billion different ways. That means I have to make around, say, 30 decisions (log2 of that) how to go about it. Now let's say I am more limited, and I can only make it in a million different ways; that means only 20 decisions have to be made. More decisions mean more overhead - everybody in the team needs to learn them, accept them, etc. So in the past, people had limited variety of tools, and the way they could build SW products were more limited, but it actually was more productive because lot of the decisions didn't have to be made. That's the paradox, we think that more specialized tooling leads to higher productivity, but in creative professions, where the job is to make decisions, and physical labor is not the main constraint, this is the opposite.
- secondaryacct 5y agoSure you're right, but doing a website in the 80s was ... nonsensical, so not needing a LESS-like preprocessor for CSS was possible. Or profilers were not as useful when the programs were printed on punch cards. The thing is we're orders of magnitude more productive if you look at what results we must achieve: if IBM wanted to publish news on a worldwide networks in the past they would have needed thousands of people and millions just to setup the infrastructure, let alone convincing people to buy terminals. Now an intern can do it in a few days because of all the successive layers of tools and software. What you describe doesnt need to happen if you are more dictatorial: get 2 languages allowed for productions (one ui, one logic), 1 more for back office (for statistics and reporting) and force everything in it. Do the same for libraries and dont do it by commitee, just say that s the stack, you can do go or groovy or rust elsewhere and done. Then people start discussing solving actual problems rather than war-of-the-stack every meeting.
- cratermoon 5y ago> Building the wrong feature or product A bit of background: a company I worked at had one primary product with a web interface for the customers, businesses who used the product, to do all their functions. Think AWS console but not so massive. There was another, minor product brought in via acquisition, which had its own web interface. A third group came into existence when my employer got acquired, which created a new web interface for its product, which did mostly the same things as the primary web interface but for a different set of customers. These all had separate logins, so customers had to know multiple credentials. There were also several large customers of the primary product who had multiple logins for reasons which defy my full understanding to this day. The project I was on was to bring in a system that would eliminate separate logins and provide a single sign-on system for them all. Good goal. very doable. The VP in charge of the new group and a product manager got into their heads that what we really needed was a single landing page portal, a piece of software that didn't exist and wasn't necessary for the sign on. The team wasted at least three if not six months chasing that landing page goal before the VP decided her product needed to go live and oh by the way we didn't have time to do the single sign-on any more before her product was to go to production. Ultimately the distraction killed any chance for the company to actually implement the desired sign-on that the customers were clamoring for, and to my knowledge they are still limping along with a every customer having at least three different logins, one for each system.
- pjmlp 5y agoBiggest waste on my career were a couple of multi-site projects, with several years duration, trying to reboot the world, as expected they all failed due to politics and trying to fix the wrong things. At least we all got better CVs as part of it.
- AliBoukeroui 5y agoYou are not alone, it's not easy thing.
- MattGaiser 5y ago> At least we all got better CVs as part of it. I am on a team that basically just imploded after a year of chaos. I mostly got promised a better CV for work we never got to do (and never will at this point).
- ronanyeah 5y agoI would like to hear some chaos stories.
- MattGaiser 5y agoHere is the most recent one: They fired all the product managers. All of them. At once. We call them Venture Leads, but they were effectively product managers. On Monday I am finalizing a feature for one of them and on Tuesday I have nobody to show it to. I don't even remember if I ended up pushing it. They apparently intend to resume work on all the projects, but at a later date when they hire new product managers/venture leads. This was not their first time firing product managers either. They fired the one for the first project I worked on. For more, read Glassdoor. https://www.glassdoor.ca/Reviews/AltaML-Reviews-E2534656.htm?sort.sortType=OR&sort.ascending=true&filter.iso3Language=eng https://www.glassdoor.ca/Reviews/AltaML-Reviews-E2534656.htm... Why did I join this company? A year ago they had 5 stars and were highly recommended, including by the person who referred me to the company. Who was one of the product managers.
- MattGaiser 5y ago
- MattGaiser 5y ago> Team churn This is somehow nearly universally seen as a bad thing (except at Amazon) yet companies seem completely fine with it surging.
- deleted 5y ago[deleted]
- phendrenad2 5y agoTeam churn can be a good thing. When you have to constantly explain to newbies "the way we do things here" you're incentivized to keep "the way we do things here" as simple as possible. Otherwise, it can become a big list of institutional quirks and unwritten rules that serve no purpose. Lean and mean is the way.
- anonyfox 5y agoAs a LEAN architect by myself, the Main problem that I encounter every day is that following the principles require actual thinking. Like: - why do we do this in the first place? What _is_ the customer value (and therefore is waste?) - is this complicated reporting process actually needed? - which strategic knowledge is missing so that the delivery teams can decide things on the spot? And so on… . The bigger the company, the harder to overcome human issues. At least it helps that the LEAN focus is always to deliver customer value fast, with high quality and still cheap (by eliminating everything else).
- andrekandre 5y agoi have the same intuition in the end, it seems a lot of people just want well defined rules (that work to a reasonable extent) to follow and instead use their thinking for solving the actual problem at hand (programming) its not unreasonable, especially if someone is hired to be just a "coder", there might be some hiring/expectations issues hidden in there too...
- ducvinh 5y agoFYI, you can read about those issues in more detail from one of the authors’ (Todd Sedano) website: http://sedano.org/software-development-wastes/index http://sedano.org/software-development-wastes/index
- kqr 5y agoAllen Ward has a different set of wastes he finds in most types of development work. I find it interesting to compare and contrast the two. I think Ward hits more fundamental points (e.g. this page suggests the waste lies in not consulting a domain expert, whereas Ward suggests the waste lies in the hand-off it takes to consult a domain expert in the first place, rather than have the development team become domain experts. Or when this page talks about building the wrong product, Ward talks about wishful thinking and being gut-driven rather than data-driven.) Here are some of the wastes I remember making notes on from Ward's book. If you want more details on any, I'm happy to answer questions. - Scatter: disruption in communication, communication barriers, changing priorities, distractions. - Hand-off: essentially when there's a common goal, but different people are in charge of knowledge, responsibility, action, and feedback. Important information disappears into black holes during hand-offs. - Useless information: reports, updates, analysis, etc. that is created not because it helps the customer, but because someone wants it to feel good. - Waiting: over-utilised resources cause queues which lengthen feedback, decrease urgency, increase variability of performance, etc. - Wishful thinking: making gut-driven rather than data-driven decisions, planning further out than you can reliably predict, reducing uncertainty too soon. - Discarded knowledge: spending time and effort building knowledge, but then filing it away and never reusing it.
- Moosdijk 5y agohttps://www.lean.org/leanpost/Posting.cfm?LeanPostId=1069 https://www.lean.org/leanpost/Posting.cfm?LeanPostId=1069
- lmarcos 5y agoInteresting. When I started my career as a software developer I was all about gut-driven design/development: I did what felt good and discarded excess. After some years of professional experience I became a data-driven person. I felt as if I had the true wisdom the team needed; I discarded any comment that was not backed up by data. This stage didn't last much, it just didn't feel right (oh the irony). Now I'm all about gut-driven development again. Difficult to explain to other engineers with less experience why I'm like this. In fact, I believe one must become a data-driven development before one can actually discard the approach.
- blondin 5y agoreally, thank you for sharing this post which provides a summary of the paper. it got me excited about the paper itself and unfortunately i can't access it. so thanks.
- ducvinh 5y agoOne of the authors also had a list of articles about this on his website, which I think might be similar to the paper content: http://sedano.org/software-development-wastes/index http://sedano.org/software-development-wastes/index
- leonmoonen 5y agoWhile I agree that the results of publicly funded research should be available as open access, IEEE does not (yet) provide authors universally with that option. However, a quick Google Scholar search would have led you to online copies [1] [1] https://scholar.google.com/scholar?&q=Software%20Development%20Waste https://scholar.google.com/scholar?&q=Software%20Development...
- pabs3 5y agoLooks like it is available on Sci-Hub, or you could probably email the authors and get a copy.
- allenu 5y agoYou may want to read the book Lean Software Development: An Agile Toolkit, it describes waste in the same manner and offers principles to help reduce it.
- LostDiscoveries 5y agoYou can read the paper here: https://www.researchgate.net/publication/313360479_Software_Development_Waste https://www.researchgate.net/publication/313360479_Software_... You do not need an account, you can just download the PDF.
- alexpetralia 5y agoNotably, almost all of these relate to personnel, knowledge sharing and management. While we operate day-to-day as individuals (and see the world from our individual vantage point), the success of large-scale, long-term projects rides on team dynamics.
- eyelidlessness 5y agoIn proportion of list items, sure. And in most organizations that’s probably roughly proportional. But I don’t think it should be dismissed that cognitive overhead and overly complex solutions are time killers. My team is wonderful, even astoundingly so, on most of the communication points. My legacy codebase is so challenging that I probably waste 90% of my time. This isn’t a typical situation! But I want to make sure it isn’t dismissed in a rush to wholly classify this as people problems.
- t-writescode 5y agoEither we work for the same company, or it’s more common than you think :)
- eyelidlessness 5y agoI think it’s unlikely we work for the same company. I’d recognize your handle :)
- onion2k 5y agoSomething will always be your biggest time sink. If you've got comms right then the next biggest thing will be instead. Also, it's hard to know whether something is a "waste of time" or simply "work". I've dealt with legacy code that's been built over many years and while it's been complex it's often necessarily complex in places. Working with that isn't a waste of time. It's just the job.
- deleted 5y ago[deleted]
- felipellrocha 5y agoThis is actually a really complete table of issues created by engineering that harm the organization overall...
- js8 5y agoLots of them are not created by engineering, some of them usually come from general/product/project management (wrong feature, mismanaging, rework, distress).
- rukuu001 5y agoConsultants, here's your checklist for looking like a genius when you walk into a new org to 'fix' a delivery team
- whamtet4 5y agoJust chill out and enjoy the process. Waste happens, that's life. Americans earn huge 100k + salaries yet they are all on prozac!!!
- phendrenad2 5y agoThis is the real answer. Stop being so attached to your code. You're getting paid a good wage to write stupid code that doesn't need to exist? Just voice your concerns, and then, if those who "know better" (tech leads, VPs, directors, founders) make you write it anyway? Laugh and enjoy!
- darkhorse13 5y agoIt's rarely this simple. If your team ends up wasting time thanks to poor decisions made by leadership, there is always a chance you will personally face issues because of it (this could even be losing your cushy job).
- commandlinefan 5y agoUntil you're expected to work nights and weekends to meet the deadline (that doesn't really "exist" either).
- d3nj4l 5y agoNo joke, taking this approach to work improved my mental health immensely. I do all the due diligence I can, but at the end of the day I have to recognise that I'm not in complete control, which is kind of liberating, actually.
- IAmNotAFix 5y agoYou can identify problems and think for solutions without letting it frustrate you each minute of the day.
- dgb23 5y agoThese two from the article: > Extraneous cognitive load > Psychological distress Are types of stress that introduce "waste" (or rather inefficiency). The root cause? Ignoring the human factor. Programmers, designers and other tech workers are not robots. We can't be. That's the job of the software we build. Our well being, happiness and engagement matters. Ultimately we are craftspeople, want to work with good people, take care of our tools and products and we need time and resources to do it right. We want to be proud of what we build or at least confident in how we build it. Too many programmers are anxious and worried because they are not given enough time and agency. Many lack connection to the users of their products and services, so they don't see the point. So I cannot fully agree with what you're saying. Sure, inefficiencies happen, waste is unavoidable. But the issues above are concerning. When we do things with care and consideration, give them both enough breathing room and focused attention, then we make better things and are happier for it. Mix that with a healthy dose of pragmatism and we also avoid doing the wrong or too many things.
- cfn 5y agoMaybe this is a mix of a couple of items in the list but when I read about the stacks in use nowadays I wonder if these would not fit the waste bucket. For example, there is a team I know of that is building a web application that serves maybe a couple of thousand people (if they ever acquire all of them as customers). They spent all this year building a humungous infrastructure with all the modern acronyms and have nothing to show the clients but a bunch of K8s this and Docker that.
- qaq 5y agoThis is industry wide issue. Working on a much larger project old school version SRE team of 4 managed all the infra. "Cloud Native" version team of 7 DevOps 1.5 years building out CI/CD setup that is still somewhat iffy.
- wruza 5y agoI feel much less like an impostor when I read this. Thanks, HN! Is it even worth it adopting ci/cd for my small craft or am I fine sticking with push, ssh, pull, [vi config.json]? What do they actually achieve with it? Edit: incorrect idiom
- yakshaving_jgt 5y agoYou are fine with what you have. In general, fix your pains when you feel them. Not before.
- marcosdumay 5y agoDo push, ssh, pull receive any parameter that may cause your program to misbehave if you get it wrong? If yes, then it's worth automating, if no, then no. Anyway, would you call putting the entire interaction in a shell script "ci/cd"?
- alexchamberlain 5y agoIf yes, then you have an acknowledged risk. That doesn't immediately say you should automate it - depends on the cost and _risk introduced_ by the automation.
- debarshri 5y agoI think one discussion which is relevant was discussed here - https://news.ycombinator.com/item?id=25569148 https://news.ycombinator.com/item?id=25569148 I think as a good manager one should be able to define essential and accidental complexity.
- p0nce 5y agoThere is a dark organizational pattern that I find wasteful: when sales have complete power over engineering, sales would routinely ask for features/projects at a breakneck pace for _potential_ customers instead of having a product-based mindset for real customers. This create a lot of deathmarch situations that waste human potential, end careers, and do not sell a whole lot in the end.
- FartyMcFarter 5y agoOne similar thing that I've seen happen in game development is having a big part of (or the whole) team working exclusively on a demo or trailer for marketing purposes, for weeks / months.
- OneTimePetes 5y agoWhich is then "sold" by the project lead/sales/pr on some conference/show to great success, unaware that they are the ones trading in project time for publicity. Deadlines of course remain fixed.
- Cthulhu_ 5y agoI've seen documentaries where they did that, crunching to make a demo or presentation work for E3. The worst ones are pre-rendered videos of things that they would like to do, but turns out they can't deliver - famous examples are Cyberpunk and No Man's Sky.
- peakaboo 5y agoI don't work for companies where sales are running engineering. Usually a horrible environment of code monkeys patching shit together so it barely works, and no rest at all inbetween tasks, because sales are clueless about what an engineer does.
- deleted 5y ago[deleted]
- whoomp12342 5y ago
- croes 5y agoI think "waste" is the wrong term. If you learn from your mistake then it's not waste it's experience. It often happened to me that useless parts of one project were useful for another one. Sometimes even if the previous project was a complete failure
- fukmbas 5y agoThe only waste in software development are product teams full of MBAs
- ChrisMarshallNY 5y agoI've become unenamored of the "Measured Human" lifestyle. This is where everything we do is metered, and our lives become "data-driven." It does bring results, but I feel there are significant costs; usually difficult to measure. One of the things that I will do, as I write software, is evaluate the code I'm working on, for reusability. Often, to make something reusable, I need to avoid optimizing it for a specific application, and "generalize" it. I'll add specializations into an "adapter" or "façade" layer, over the module. Most of my little SPM modules were developed this way. If I think it deserves it, I'll stop work on my principal project, and break out the subsystem into its own project and repo. Once I have it standalone, tested, and ready, I will re-integrate it, as an external SPM module. In more than one instance, I have removed the code from the principal project, because it was not something that would contribute to the main goal. But I now have this neat little SPM module, ready to be used in other projects. Here's an example of a project that never made it into any of final results, but I think is pretty neat[0]. Some of my modules are basically my "baseline." These are used in everything I do. Many would consider my workstyle "wasteful," but it works for me. [0] https://github.com/RiftValleySoftware/RVS_Spinner https://github.com/RiftValleySoftware/RVS_Spinner
- kqr 5y agoThere is a constant tension in product development between delivering value and creating reusable knowledge. Creating reusable knowledge is not in itself wasteful, but neither will it in itself deliver value. (The reason we do it is because it makes value delivery more efficient in the future.) Many people today will emphasise delivering value and sacrifice reusable knowledge. They will call it "wasteful" to create reusable knowledge, but it's not. The trick is finding the right balance, of course.
- alexchamberlain 5y agoI think we need to develop a notion of 2 sided value: there is value in delivering additional profit/revenue, but there's also value in reducing costs. Creating reusable knowledge is an example of reducing (the expected) cost.
- takotab 5y agoCouldn't help sharing this here: https://youtu.be/_NeJ3Kg6OUo https://youtu.be/_NeJ3Kg6OUo Silicon valley: product and engineering
- wilde 5y agoLove Pivotal, but also their software development methodology is so different from the rest of industry. Not sure how much the findings of this paper generalize since they’re from a relatively non-standard dev process.
- jacques_chester 5y agoI'm an ex-Pivot. I've seen many of these identified wastes in other organisations. Pivotal was not immune to these wastes, it just had some resistance.
- beckingz 5y agoWhat is a good example of capricious thrashing? Is that just alternating working on tasks without ever finishing anything (normal thrashing) more of the time?
- BeefySwain 5y agoA good explanation of a couple of these terms were found here: http://sedano.org/software-development-wastes/mismanaging-the-backlog.html http://sedano.org/software-development-wastes/mismanaging-th... "This waste is also related a tension: intransigence versus capricious trashing. Responding to change quickly is a core tenet of agile development and often thought of as the opposite of refusing to change. However, responding to change is more like a middle ground between intransigence (unreasonably refusing to change) and thrashing (changing features too often, especially arbitrarily alternating between equally good alternatives)."
- commandlinefan 5y agoAnd the root cause - in all cases - is "unreasonable/pointless deadlines".
- notacoward 5y agoThe waste I've become most sensitized to is excessive ceremony - things that you have to keep doing but don't actually contribute much to real velocity. Bad CI/CD and review processes are good examples. Even though I'm generally a huge proponent of testing and even formal methods, I've seen so much time wasted on types of testing that consume lots of developer time (not to mention machine time) but have provably never surfaced any real bugs and couldn't reasonably be expected to.