4 ms·

This reminds me of the now classic article: http://steve-yegge.blogspot.com/2006/09/good-agile-bad-agile_27.html http://steve-yegge.blogspot.com/2006/09/good-ag

by js8 5y ago

This reminds me of the now classic article: http://steve-yegge.blogspot.com/2006/09/good-agile-bad-agile_27.html http://steve-yegge.blogspot.com/2006/09/good-agile-bad-agile... It essentially claims that "good agile" is Kanban, and Kanban is essentially abolishing of any forecasting in favor of actually doing the tasks, like the CPUs do when executing programs.

mickimnet 5y ago
Not debating about estimating or not. I'm only replying to the example: CPUs can only execute, what the instructions tell them to do—like a production line—and not solve / create something (new).
- henryfjordan 5y ago
  Kanban was originally designed for production lines in car factories, so it works. If you go to any McDonalds and look at the screens in the back you'll see orders represented a lot like tickets. In Agile, "points" are just an abstraction around work to try to normalize it in the same way that every burger+fries combo should be roughly equivalent.
- nescioquid 5y ago
  Not sure why you were down-voted, but this is a good point. In fact, it is the base problem of all software methodologies. Planning depends on knowing how long an activity will take. Estimating is most reliable when you are estimating something you've already done before. Unless you're working in just such a low-novelty environment, your results will not be predictable and you should understand that the backlog is a priority-ordered list and velocity will let you give a "forecast". Kanban doesn't "fix" this -- it just changes the frame through which deadline-buzzards get to look at it.
  - js8 5y ago
    I would say, if you absolutely need an estimate on how long a project will take (and often you don't), do a proper study on it, don't just pull a number out of your a*. Software project estimation is doable, but it takes some effort. It (IMHO) cannot be replaced by a clairvoyance seance, AKA planning poker.
    - doctor_eval 5y ago
      I’ve said this previously but I don’t agree that software estimating is possible in theory. It’s a form of the halting problem. You can’t, even in theory, know how long something will take until you do it. It’s certainly possible to estimate to a rough-order-of-magnitude but the problem is that the ROM quickly becomes a target, and then a commitment. I have a couple of positions that I take. First, the old adage still applies: “better, faster, cheaper - choose two”. If you set a deadline then you have to be willing to drop features or quality, because Brooks already demonstrated you can’t add resources after you start. Second, if you do need to set a deadline then prioritise it, stick to it and celebrate it. Too many times deadlines are used as a whip, and my or my team have wiped themselves out to meet it, only to be told it wasn’t really needed. This is insanely demoralising and strongly affects productivity in the long term. My personal (and unpopular) preference is to prioritise completion of stand-alone features rather than building to deadlines, which means you can release a product at almost any time with fewer, but better quality features. This approach has a demonstrable positive impact on the ROI but because it’s more complex and provides less short term certainty, it’s frowned upon.
      - js8 5y ago
        I think in pure compsci theory you might be right, but in practice (industrial SW production), we usually limit ourselves to algorithmic solutions that do not involve halting problem, so estimation is possible. For example, an upper limit for a rewrite of a system is how long it took to create the original system (provided you understand the tradeoffs and issues of the original architecture).
        doctor_eval 5y ago
        I don’t understand. The halting problem says you can’t tell if an algorithm halts without running it - you can’t be sure any program will run to completion in a specific time. It applies to any non trivial program. And it’s well established that a rewrite is dangerous. The second system effect suggests that rewrites will always take much longer.
  - no_wizard 5y ago
    Arguably however, Kanban frames them in such a way that both business (re: usually product) folks find them agreeable, and so do engineers. If it reduces friction (as suggested) for both sides and everyone spends less time managing ceremonies as a result, then its a net win.
- js8 5y ago
  My point was that OS dispatchers don't spend time working out how long things will take to compute, to allocate resources (such as CPU time) properly into the future; what they do instead is they just put the stuff to compute into a dispatch queue and go from there. Then if it's a novel thing or million times repeated thing doesn't really matter. BTW, one indication that Scrum is bad is that nobody is doing its analogue (syncing everybody once a day?) in computing. Sane project management practices should be universal, and should also work for operating system schedulers.
doctor_eval 5y ago
I haven’t read the article (yet) but I came to the same conclusion many years ago. In a recent previous thread here on HN I was roundly criticised for saying that due-date driven management is not effective, but that’s what scrum has become, and todays thread is exactly what I was thinking about. My experience in a few businesses is that Scrum as it is generally practiced in industry has been an absolute disaster for productivity. Kanban is awesome because it doesn’t force us to spend hours and hours every week on ceremonies which don’t change the outcome one whit. In my own business, after 2+ years of scrum - including hiring a CSM - I tossed out our entire process, and replaced it with backlog management and kanban boards. The only thing we kept was the (short) daily standup. Productivity went through the roof for a number of reasons, not least of which was that because we could now spend the ceremony time working on actual code. The killer is that if you try to tell people this, they think you’re inexperienced or did scrum wrong or you don’t know what you’re talking about. But to me Scrum is a two year black hole of stagnant product development that I will never get back.
- pavel_lishin 5y ago
  > In a recent previous thread here on HN I was roundly criticised for saying that due-date driven management is not effective, but that’s what scrum has become, and todays thread is exactly what I was thinking about. Do you have any recommendations for scenarios where there are due dates? I work in a field that is largely focused around dates that aren't really negotiable; the school year ends and starts and school budgeting decisions have to be made at a certain time, so if we're rolling out certain features, they really do need to be available in, e.g., August when school starts.
  - doctor_eval 5y ago
    I guess my position is that we should assume our software engineers are professionals who are properly motivated to succeed, and with this in mind, we should take a priority-based, Kanban-style approach to building and deploying software, with significantly less emphasis on estimation and significantly more emphasis on prioritisation. I don't think anything I do is novel or special; it's just not widely adopted as far as I know. I've been using a backlog-driven, priority-based approach in my own practice for the last 5+ years, which has been extremely successful at delivering products, but remarkably less successful politically. Non-technical management seems to have come to the conclusion that we ought to be able to accurately estimate the time it will take to build stuff, despite decades of evidence to the contrary. Loads of people argue that this is the "only way" but the processes commonly implemented are more about making management feel good than about actually getting work done. Unfortunately, I've found that making management feel good is often more important than actually doing the work. All of that said, what I suggest is very boring: instead of spending the (significant) time estimating the effort required to implement each task, instead you should spend the time working with the client on prioritising the features, and internally spend time prioritising the decomposed tasks. What features/tasks absolutely must be ready by the deadline, and what can they live without? Can some features be shipped "incomplete" from your perspective? For example, can you deliver a veneer over a manual process on launch day, and automate the process later? If, upon inspection, a feature turns out to be extremely complex and time consuming, can it be re-prioritised? Can you spend time working on reducing the total amount of actual effort that's on the "critical path" to the deadline? Obviously some things just need to be done, but when you drill down you often find that a lot of time is spend on features and tasks that are not critical (by whatever definition you choose to use). The idea is simply to make sure that the important stuff gets done first, so that you are in a position to launch as soon as possible. You can and should continue to work on the project after the launch day. This approach has many well-known carry-over benefits like the ability to start user and automated testing very early. Unfortunately, as mentioned, this can be very tricky to pull off politically, which I think is why it's not more common. Once you get to this point then reporting upwards should be about features delivered, features we're working on now, and features that have not yet started. Somewhere in that list is a red line that says "we can launch now". You can directly measure velocity in terms of how many features are left until you are in the green. I want to be clear that I think rough-order-of-magnitude estimates on work-in-progress and future work are perfectly fine and reasonable for internal use. What I am against is the use of deadlines and dates to manage the project as a whole. The goal is to deploy features, and that will take as long as it takes, and no amount of stomping of feet is going to change that, assuming you have a good team. Ultimately what everyone wants is for the project to launch successfully. Software is very complex, and history (and theory) suggests that it is impossible to estimate accurately. The more time we spend on estimating, the less time we spend on working, and the more likely we are to miss our deadlines. One final note. If you use JIRA, the backlog sucks (or at least it did when I abandoned it a few years ago). You need something that you can manipulate, that you can create different views, and that can collect information that helps you manage and prioritise it, since the backlog is orders of magnitude larger than the in-progress work and in my opinion is the actual problem that needs to be solved. I've personally used ProductBoard for many years but AirTable and even a spreadsheet is good. I am keen to hear criticism and other ideas. My key thing is that saying "this feature should be finished by THIS DATE" is an absolute killer of productivity because of the amount of effort required to determine "THIS DATE" and the fact that the date is always wrong anyway.
BaseballPhysics 5y ago
> Kanban is essentially abolishing of any forecasting This isn't even remotely true. Kanban establishes specific measures, including throughput, that, combined with statistical modeling, specifically allow for forecasting to be done. Do you really think the car industry, where Kanban originates, doesn't care how many cars they're going to produce in a quarter, or whether they're ahead, behind, or on track on hitting their targets? Come on. The idea that "forecasting" and "agile" are somehow at odds with one another is a gross misunderstanding of both. Edit: As an aside, that article's use of Google as somehow the paragon of software development superiority really dates it. Google is notorious, at this point, for being an unreliable software development organization. They fail to listen to their users by building features people don't want, canceling projects that vocal users do want, and shipping incredibly buggy products (Android 12, anyone?) All of this is, at least in part, caused by many of the problems the blog post highlights but claims have been resolved, including a lack of strategic cohesiveness and issues with building and maintaining staffing for projects that are no longer sexy.
- stephendause 5y ago
  > This isn't even remotely true. Kanban establishes specific measures, including throughput, that, combined with statistical modeling, specifically allow for forecasting to be done. That's interesting to me as I hadn't heard that about Kanban before. Do you have a particular reference in mind that you think describes this well?
  - BaseballPhysics 5y ago
    First off, it's important to understand that Kanban relies on a backlog built of stories that are all of similar size and complexity. Yes, that means you need to estimate. But those estimates come in the form of "is this roughly as complex as other work the team has done?" So it's a lot less onerous to assess. Then you can measure throughput (number of stories per unit of time) and cycle time (amount of time a story takes from start to finish). Using those measures you can forecast, with monte carlo simulation as a common method. Here's a book on the topic: https://www.amazon.ca/Forecasting-Simulating-Software-Development-Projects/dp/1466454830 https://www.amazon.ca/Forecasting-Simulating-Software-Develo... There's plenty of other resources about this if you bust out your google-fu.

by js8 5y ago

mickimnet 5y ago

Not debating about estimating or not. I'm only replying to the example: CPUs can only execute, what the instructions tell them to do—like a production line—and not solve / create something (new).

henryfjordan 5y ago
Kanban was originally designed for production lines in car factories, so it works. If you go to any McDonalds and look at the screens in the back you'll see orders represented a lot like tickets. In Agile, "points" are just an abstraction around work to try to normalize it in the same way that every burger+fries combo should be roughly equivalent.
nescioquid 5y ago
Not sure why you were down-voted, but this is a good point. In fact, it is the base problem of all software methodologies. Planning depends on knowing how long an activity will take. Estimating is most reliable when you are estimating something you've already done before. Unless you're working in just such a low-novelty environment, your results will not be predictable and you should understand that the backlog is a priority-ordered list and velocity will let you give a "forecast". Kanban doesn't "fix" this -- it just changes the frame through which deadline-buzzards get to look at it.
- js8 5y ago
  I would say, if you absolutely need an estimate on how long a project will take (and often you don't), do a proper study on it, don't just pull a number out of your a*. Software project estimation is doable, but it takes some effort. It (IMHO) cannot be replaced by a clairvoyance seance, AKA planning poker.
  - doctor_eval 5y ago
    I’ve said this previously but I don’t agree that software estimating is possible in theory. It’s a form of the halting problem. You can’t, even in theory, know how long something will take until you do it. It’s certainly possible to estimate to a rough-order-of-magnitude but the problem is that the ROM quickly becomes a target, and then a commitment. I have a couple of positions that I take. First, the old adage still applies: “better, faster, cheaper - choose two”. If you set a deadline then you have to be willing to drop features or quality, because Brooks already demonstrated you can’t add resources after you start. Second, if you do need to set a deadline then prioritise it, stick to it and celebrate it. Too many times deadlines are used as a whip, and my or my team have wiped themselves out to meet it, only to be told it wasn’t really needed. This is insanely demoralising and strongly affects productivity in the long term. My personal (and unpopular) preference is to prioritise completion of stand-alone features rather than building to deadlines, which means you can release a product at almost any time with fewer, but better quality features. This approach has a demonstrable positive impact on the ROI but because it’s more complex and provides less short term certainty, it’s frowned upon.
    - js8 5y ago
      I think in pure compsci theory you might be right, but in practice (industrial SW production), we usually limit ourselves to algorithmic solutions that do not involve halting problem, so estimation is possible. For example, an upper limit for a rewrite of a system is how long it took to create the original system (provided you understand the tradeoffs and issues of the original architecture).
      - doctor_eval 5y ago
        I don’t understand. The halting problem says you can’t tell if an algorithm halts without running it - you can’t be sure any program will run to completion in a specific time. It applies to any non trivial program. And it’s well established that a rewrite is dangerous. The second system effect suggests that rewrites will always take much longer.
- no_wizard 5y ago
  Arguably however, Kanban frames them in such a way that both business (re: usually product) folks find them agreeable, and so do engineers. If it reduces friction (as suggested) for both sides and everyone spends less time managing ceremonies as a result, then its a net win.
js8 5y ago
My point was that OS dispatchers don't spend time working out how long things will take to compute, to allocate resources (such as CPU time) properly into the future; what they do instead is they just put the stuff to compute into a dispatch queue and go from there. Then if it's a novel thing or million times repeated thing doesn't really matter. BTW, one indication that Scrum is bad is that nobody is doing its analogue (syncing everybody once a day?) in computing. Sane project management practices should be universal, and should also work for operating system schedulers.

doctor_eval 5y ago

I haven’t read the article (yet) but I came to the same conclusion many years ago. In a recent previous thread here on HN I was roundly criticised for saying that due-date driven management is not effective, but that’s what scrum has become, and todays thread is exactly what I was thinking about. My experience in a few businesses is that Scrum as it is generally practiced in industry has been an absolute disaster for productivity. Kanban is awesome because it doesn’t force us to spend hours and hours every week on ceremonies which don’t change the outcome one whit. In my own business, after 2+ years of scrum - including hiring a CSM - I tossed out our entire process, and replaced it with backlog management and kanban boards. The only thing we kept was the (short) daily standup. Productivity went through the roof for a number of reasons, not least of which was that because we could now spend the ceremony time working on actual code. The killer is that if you try to tell people this, they think you’re inexperienced or did scrum wrong or you don’t know what you’re talking about. But to me Scrum is a two year black hole of stagnant product development that I will never get back.

pavel_lishin 5y ago
> In a recent previous thread here on HN I was roundly criticised for saying that due-date driven management is not effective, but that’s what scrum has become, and todays thread is exactly what I was thinking about. Do you have any recommendations for scenarios where there are due dates? I work in a field that is largely focused around dates that aren't really negotiable; the school year ends and starts and school budgeting decisions have to be made at a certain time, so if we're rolling out certain features, they really do need to be available in, e.g., August when school starts.
- doctor_eval 5y ago
  I guess my position is that we should assume our software engineers are professionals who are properly motivated to succeed, and with this in mind, we should take a priority-based, Kanban-style approach to building and deploying software, with significantly less emphasis on estimation and significantly more emphasis on prioritisation. I don't think anything I do is novel or special; it's just not widely adopted as far as I know. I've been using a backlog-driven, priority-based approach in my own practice for the last 5+ years, which has been extremely successful at delivering products, but remarkably less successful politically. Non-technical management seems to have come to the conclusion that we ought to be able to accurately estimate the time it will take to build stuff, despite decades of evidence to the contrary. Loads of people argue that this is the "only way" but the processes commonly implemented are more about making management feel good than about actually getting work done. Unfortunately, I've found that making management feel good is often more important than actually doing the work. All of that said, what I suggest is very boring: instead of spending the (significant) time estimating the effort required to implement each task, instead you should spend the time working with the client on prioritising the features, and internally spend time prioritising the decomposed tasks. What features/tasks absolutely must be ready by the deadline, and what can they live without? Can some features be shipped "incomplete" from your perspective? For example, can you deliver a veneer over a manual process on launch day, and automate the process later? If, upon inspection, a feature turns out to be extremely complex and time consuming, can it be re-prioritised? Can you spend time working on reducing the total amount of actual effort that's on the "critical path" to the deadline? Obviously some things just need to be done, but when you drill down you often find that a lot of time is spend on features and tasks that are not critical (by whatever definition you choose to use). The idea is simply to make sure that the important stuff gets done first, so that you are in a position to launch as soon as possible. You can and should continue to work on the project after the launch day. This approach has many well-known carry-over benefits like the ability to start user and automated testing very early. Unfortunately, as mentioned, this can be very tricky to pull off politically, which I think is why it's not more common. Once you get to this point then reporting upwards should be about features delivered, features we're working on now, and features that have not yet started. Somewhere in that list is a red line that says "we can launch now". You can directly measure velocity in terms of how many features are left until you are in the green. I want to be clear that I think rough-order-of-magnitude estimates on work-in-progress and future work are perfectly fine and reasonable for internal use. What I am against is the use of deadlines and dates to manage the project as a whole. The goal is to deploy features, and that will take as long as it takes, and no amount of stomping of feet is going to change that, assuming you have a good team. Ultimately what everyone wants is for the project to launch successfully. Software is very complex, and history (and theory) suggests that it is impossible to estimate accurately. The more time we spend on estimating, the less time we spend on working, and the more likely we are to miss our deadlines. One final note. If you use JIRA, the backlog sucks (or at least it did when I abandoned it a few years ago). You need something that you can manipulate, that you can create different views, and that can collect information that helps you manage and prioritise it, since the backlog is orders of magnitude larger than the in-progress work and in my opinion is the actual problem that needs to be solved. I've personally used ProductBoard for many years but AirTable and even a spreadsheet is good. I am keen to hear criticism and other ideas. My key thing is that saying "this feature should be finished by THIS DATE" is an absolute killer of productivity because of the amount of effort required to determine "THIS DATE" and the fact that the date is always wrong anyway.

BaseballPhysics 5y ago

> Kanban is essentially abolishing of any forecasting This isn't even remotely true. Kanban establishes specific measures, including throughput, that, combined with statistical modeling, specifically allow for forecasting to be done. Do you really think the car industry, where Kanban originates, doesn't care how many cars they're going to produce in a quarter, or whether they're ahead, behind, or on track on hitting their targets? Come on. The idea that "forecasting" and "agile" are somehow at odds with one another is a gross misunderstanding of both. Edit: As an aside, that article's use of Google as somehow the paragon of software development superiority really dates it. Google is notorious, at this point, for being an unreliable software development organization. They fail to listen to their users by building features people don't want, canceling projects that vocal users do want, and shipping incredibly buggy products (Android 12, anyone?) All of this is, at least in part, caused by many of the problems the blog post highlights but claims have been resolved, including a lack of strategic cohesiveness and issues with building and maintaining staffing for projects that are no longer sexy.

stephendause 5y ago
> This isn't even remotely true. Kanban establishes specific measures, including throughput, that, combined with statistical modeling, specifically allow for forecasting to be done. That's interesting to me as I hadn't heard that about Kanban before. Do you have a particular reference in mind that you think describes this well?
- BaseballPhysics 5y ago
  First off, it's important to understand that Kanban relies on a backlog built of stories that are all of similar size and complexity. Yes, that means you need to estimate. But those estimates come in the form of "is this roughly as complex as other work the team has done?" So it's a lot less onerous to assess. Then you can measure throughput (number of stories per unit of time) and cycle time (amount of time a story takes from start to finish). Using those measures you can forecast, with monte carlo simulation as a common method. Here's a book on the topic: https://www.amazon.ca/Forecasting-Simulating-Software-Development-Projects/dp/1466454830 https://www.amazon.ca/Forecasting-Simulating-Software-Develo... There's plenty of other resources about this if you bust out your google-fu.