4 ms·

But that is exactly the problem that happens all the time when you are e.g. working on a new statistical model. I've worked in software for 25 years and rarely

But that is exactly the problem that happens all the time when you are e.g. working on a new statistical model. I've worked in software for 25 years and rarely had difficulty finding a good functional breakpoint where my software could be demo'd in some form. It definitely doesn't happen "all the time" in development. But it happens all the time with fundamental research. Nor does it happen all the time with research. I've run research on a scrum basis with 2 week sprints for 6 years with great success. It just isn't that hard to break work into reasonable pieces. If it is really hard, even after years of practice, then writing software is probably not a good career choice. Do we always get it right? No. Our percentage of 'green' sprints is around 70%. But that's enough to provide some stability in planning, and to be able to track our incremental progress without too many nasty surprises. If you have discovered a way of recording velocity that has even a weak predictive efficacy for forecasting completion, please let me know. Across all software teams I’ve heard of, worked on, or collaborated with, the velocity numbers carry no forecasting efficacy whatsoever. Not even for ballpark estimates. This is a fair criticism, but accurate software estimation is hard, period, agile or not. Velocity may not be useful for telling you when a project will be complete, but it's very useful in warning you when it won't be complete, which in practice is just as valuable. I won't defend anyone who applies deep statistical analysis on sprint velocity, unless they have a very large dataset to work with.

mlthoughts2018 8y ago
> "It definitely doesn't happen "all the time" in development." It absolutely does in different business units. I'm glad this hasn't happened to you in your business units. That doesn't say anything about other types of business units. My experience is particularly in developing machine learning research that is both publishable and simultaneously directly useful for a performance-constraint business problem. In this area of work, the output of two weeks of labor is often nothing other than "I thought about this and here's what I learned... and I suspect we are no closer to anything usable." You string a few of these cycles together, and then things finally crystallize and you suddenly make a dramatic amount of progress very quickly. Then, after that point the problem becomes more like a standard engineering problem, with an iterative cycle to steadily bring a software system wrapping the research product into a deliverable state, and incremental progress is appropriate. You just seem to think only from your bubble, where the idea of incremental progress is almost always useful for you. But there are whole business domains where the idea of incremental progress is nonsense, and you have to dig into a problem in a deep and uninterrupted way for long periods of time before you can describe what's going on. Why would it be better to force people to shoehorn their work into a prescriptive model that requires you to give incremental progress, when that goes against the very nature of the work? Why not let the team customize it as needed. Maybe Person A doesn't need to check in for a month, but Person B is working on productionizing a new solution from two months ago, so Person B does need to check in more regarding incremental updates. Why not let them each have different planning cycles just using basic common sense and trusting them to do their work? I still don't see anything in your comment beyond just saying, "my work area has happened to never be like that, therefore Agile works in all situations." > "Nor does it happen all the time with research. I've run research on a scrum basis with 2 week sprints for 6 years with great success. It just isn't that hard to break work into reasonable pieces. If it is really hard, even after years of practice, then writing software is probably not a good career choice." Oh. I see. You're just trolling.
- vannevar 8y ago
  I also work in machine learning, and as I noted, have used agile methods very successfully in that field, including advanced research into the application of evolutionary algorithms to neural networks in simulated environments. There is nothing wrong with taking two weeks to just think about a problem and tinker with it. Take six weeks. Take six months. But when it is time to start making concrete progress on some kind of a timetable, on somebody's dime, then scrum is a great way to do it. Agile is first and foremost a way to implement software; it is not a way to design it from scratch. The most agile says about design is that it is an incremental and dynamic process that continues during implementation. If your point is that agile doesn't help you much if you don't know yet what you're going to build, I totally agree, and so does sound agile practice. I may indeed just be lucky, but if so then I've been lucky for an extraordinarily long time and in an extraordinarily wide range of contexts. Agile is not right for everything, but for developing software, it's great. Oh. I see. You're just trolling. Not at all. I am utterly sincere. Being able to chunk complex projects into simpler components that can fit into smaller timeframes (yes, as small as two weeks or even two days) is an essential skill for a software developer. And if that consistently thwarts you, then software development will make you very unhappy, whether you use agile development or not.
  - mlthoughts2018 8y ago
    > Being able to chunk complex projects into simpler components that can fit into smaller timeframes... is an essential skill for a software developer. Why do you feel this claim is related to Agile or Scrum in any way? When I see that claim, I agree, and I am reminded of many of the things I like about software. And I do not see how it relates to Agile / Scrum in any way. Back when I used to work on a team that used only Waterfall in a defense research company, we broke up complex problems into smaller components with different timelines and informal status check-ins with other components of the project. When I was a grad student and we had literally no formalized workflow for our lab projects culminating in open source library releases and journal paper submissions, we also broke up our problems and analyzed how to decompose them and think of them modularly. It's a skill of software design that seems to have utterly no relationship to what overarching workflow management process is used. Why do you think they must be logically inseparable?

mlthoughts2018 8y ago

> "It definitely doesn't happen "all the time" in development." It absolutely does in different business units. I'm glad this hasn't happened to you in your business units. That doesn't say anything about other types of business units. My experience is particularly in developing machine learning research that is both publishable and simultaneously directly useful for a performance-constraint business problem. In this area of work, the output of two weeks of labor is often nothing other than "I thought about this and here's what I learned... and I suspect we are no closer to anything usable." You string a few of these cycles together, and then things finally crystallize and you suddenly make a dramatic amount of progress very quickly. Then, after that point the problem becomes more like a standard engineering problem, with an iterative cycle to steadily bring a software system wrapping the research product into a deliverable state, and incremental progress is appropriate. You just seem to think only from your bubble, where the idea of incremental progress is almost always useful for you. But there are whole business domains where the idea of incremental progress is nonsense, and you have to dig into a problem in a deep and uninterrupted way for long periods of time before you can describe what's going on. Why would it be better to force people to shoehorn their work into a prescriptive model that requires you to give incremental progress, when that goes against the very nature of the work? Why not let the team customize it as needed. Maybe Person A doesn't need to check in for a month, but Person B is working on productionizing a new solution from two months ago, so Person B does need to check in more regarding incremental updates. Why not let them each have different planning cycles just using basic common sense and trusting them to do their work? I still don't see anything in your comment beyond just saying, "my work area has happened to never be like that, therefore Agile works in all situations." > "Nor does it happen all the time with research. I've run research on a scrum basis with 2 week sprints for 6 years with great success. It just isn't that hard to break work into reasonable pieces. If it is really hard, even after years of practice, then writing software is probably not a good career choice." Oh. I see. You're just trolling.

vannevar 8y ago
I also work in machine learning, and as I noted, have used agile methods very successfully in that field, including advanced research into the application of evolutionary algorithms to neural networks in simulated environments. There is nothing wrong with taking two weeks to just think about a problem and tinker with it. Take six weeks. Take six months. But when it is time to start making concrete progress on some kind of a timetable, on somebody's dime, then scrum is a great way to do it. Agile is first and foremost a way to implement software; it is not a way to design it from scratch. The most agile says about design is that it is an incremental and dynamic process that continues during implementation. If your point is that agile doesn't help you much if you don't know yet what you're going to build, I totally agree, and so does sound agile practice. I may indeed just be lucky, but if so then I've been lucky for an extraordinarily long time and in an extraordinarily wide range of contexts. Agile is not right for everything, but for developing software, it's great. Oh. I see. You're just trolling. Not at all. I am utterly sincere. Being able to chunk complex projects into simpler components that can fit into smaller timeframes (yes, as small as two weeks or even two days) is an essential skill for a software developer. And if that consistently thwarts you, then software development will make you very unhappy, whether you use agile development or not.
- mlthoughts2018 8y ago
  > Being able to chunk complex projects into simpler components that can fit into smaller timeframes... is an essential skill for a software developer. Why do you feel this claim is related to Agile or Scrum in any way? When I see that claim, I agree, and I am reminded of many of the things I like about software. And I do not see how it relates to Agile / Scrum in any way. Back when I used to work on a team that used only Waterfall in a defense research company, we broke up complex problems into smaller components with different timelines and informal status check-ins with other components of the project. When I was a grad student and we had literally no formalized workflow for our lab projects culminating in open source library releases and journal paper submissions, we also broke up our problems and analyzed how to decompose them and think of them modularly. It's a skill of software design that seems to have utterly no relationship to what overarching workflow management process is used. Why do you think they must be logically inseparable?