5 ms·

I want to agree, but there's no way to apply the scientific method to this case. Experiments would take at least several months to run, because it's not till se

I want to agree, but there's no way to apply the scientific method to this case. Experiments would take at least several months to run, because it's not till several months that you know whether your business is going to fail solely due to the dev team. And at that point it's hard to run another experiment. More importantly, the scientific method relies on an oracle (nature) to tell us whether we're right or wrong. It's not so easy to apply when there's no conclusive way to know you're mistaken. That said, I completely agree that standups are for managers. Someone once said that standups are to re-affirm that everyone reports to the manager, and pretty much nothing else.

jerf 10y ago
"Experiments would take at least several months to run, because it's not till several months that you know whether your business is going to fail solely due to the dev team." If you use the rather stodgy and formulaic scientific method, sure. But if you approach this as a multi-arm bandit problem, and instead look at the question as "how much information do I need to collect with how much statistical power before I am justified in selecting an outcome?", it becomes quite tractable for a team to just try it out. It also helps if instead of using merely "productivity" as your one and only goal that you also consider team happiness as a real and going concern. Then "Nobody likes it and it's demoralizing the engineers" becomes a perfectly valid form of feedback to use in the decision about whether to do it. I also tend to use the metric that the brain is really, really, really good at figuring out what's important, sometimes perhaps even too good, so if we start a stand-up meeting plan, and it just sort of peters out, then I let it go. (Seems like somebody wants to try it every 2 or 3 years or so, I generally just let it happen and then evaluate.) Everybody's brains are coming to the conclusion that it's not important. Rather than fighting it, roll with it. There are some things in life where your rational forebrain must override the deeper instinctive algorithms about what is important, but that's quite cognitively expensive, and I've never seen any reason to believe that stand-up meetings are bringing enough value to justify that. I emphasize the "I've never seen any reason". I hypothesize that in many ways, the work the teams I've been on have been doing is almost maximally pathologically bad for a stand-up meeting. Almost everyone is working on something independent of everybody else, so blockers are only rarely even possible. If your team is, on the other hand, chock-a-block full of blocking potential it may be useful to you. Anyways, there's more ways to find out the truth of things that trying to split the universe into two precisely equal parts, make a change in only one of them, and come back ten years later to see what happened, only for someone to complain that a sample size of one isn't enough and we really need to do this a thousand more times to have a valid result.
- sillysaurus3 10y ago
  If you use the rather stodgy and formulaic scientific method, sure. You mean the scientific method that has proven to work since the 1700's? I think we'll have to agree to disagree that you can just riff it and still get results.
  - jerf 10y ago
    Bummer, I wish I'd have caught this earlier. Look up "multi-armed bandit"; it isn't just "riffing it" like you think it is. It is a mathematically-principled technique. In fact, it is probably safe to say it's more mathematically-principled than the "scientific method" itself, as it is popularly understood.
  - AnimalMuppet 10y ago
    So what would the scientific method have you do here? Take, say, 100 projects. On half of them, do standups. On the other half, don't. Measure the outcomes. Do statistics. The problem is, you'd have to find 100 projects where they care more about the scientific rigor of the experiment than they do about the results of the project. Good luck finding that with real-world projects. jerf's approach gives you data that is less certain than you would get from a real scientific experiment. But it gives you data that is better than nothing, gathered in the real world. And you can actually run the experiment (at least in some environments) without getting fired, which is more than is likely true with a pure "scientific method" approach.
    - sillysaurus3 10y ago
      jerf's approach gives you data that is less certain than you would get from a real scientific experiment. But it gives you data that is better than nothing, gathered in the real world. The "better than nothing" is precisely where this logic goes awry. It's usually better to admit what you're actually doing: Going with your gut instincts. Pretending you're doing science adds fake legitimacy to your words and makes it easy to fool people. Unfortunately, the person you have to be concerned about fooling is yourself.
js8 10y ago
I disagree. First of all, I am sure there are scientific ways to analyze things like this, scientists deal with phenomena like this all the time. But the main problem is - if people don't actually know if the standup meeting helps, they shouldn't recommend it as a part of SCRUM (or other methodology). I know that in Agile, it's popular to say "if you think it doesn't work, change it". It's a first step towards scientific approach, but also very naive step - it can easily cause you to create rituals with no effects, just based on anecdotal coincidences. It is kind of disconcerting. On one hand, I applaud the excitement of Agile fans for critical thinking and scientific method, but at least they could study it a little - to find out that there is a little more to it than just "if it doesn't work change it". And that they shouldn't propose things that are not properly justified.

jerf 10y ago

"Experiments would take at least several months to run, because it's not till several months that you know whether your business is going to fail solely due to the dev team." If you use the rather stodgy and formulaic scientific method, sure. But if you approach this as a multi-arm bandit problem, and instead look at the question as "how much information do I need to collect with how much statistical power before I am justified in selecting an outcome?", it becomes quite tractable for a team to just try it out. It also helps if instead of using merely "productivity" as your one and only goal that you also consider team happiness as a real and going concern. Then "Nobody likes it and it's demoralizing the engineers" becomes a perfectly valid form of feedback to use in the decision about whether to do it. I also tend to use the metric that the brain is really, really, really good at figuring out what's important, sometimes perhaps even too good, so if we start a stand-up meeting plan, and it just sort of peters out, then I let it go. (Seems like somebody wants to try it every 2 or 3 years or so, I generally just let it happen and then evaluate.) Everybody's brains are coming to the conclusion that it's not important. Rather than fighting it, roll with it. There are some things in life where your rational forebrain must override the deeper instinctive algorithms about what is important, but that's quite cognitively expensive, and I've never seen any reason to believe that stand-up meetings are bringing enough value to justify that. I emphasize the "I've never seen any reason". I hypothesize that in many ways, the work the teams I've been on have been doing is almost maximally pathologically bad for a stand-up meeting. Almost everyone is working on something independent of everybody else, so blockers are only rarely even possible. If your team is, on the other hand, chock-a-block full of blocking potential it may be useful to you. Anyways, there's more ways to find out the truth of things that trying to split the universe into two precisely equal parts, make a change in only one of them, and come back ten years later to see what happened, only for someone to complain that a sample size of one isn't enough and we really need to do this a thousand more times to have a valid result.

sillysaurus3 10y ago
If you use the rather stodgy and formulaic scientific method, sure. You mean the scientific method that has proven to work since the 1700's? I think we'll have to agree to disagree that you can just riff it and still get results.
- jerf 10y ago
  Bummer, I wish I'd have caught this earlier. Look up "multi-armed bandit"; it isn't just "riffing it" like you think it is. It is a mathematically-principled technique. In fact, it is probably safe to say it's more mathematically-principled than the "scientific method" itself, as it is popularly understood.
- AnimalMuppet 10y ago
  So what would the scientific method have you do here? Take, say, 100 projects. On half of them, do standups. On the other half, don't. Measure the outcomes. Do statistics. The problem is, you'd have to find 100 projects where they care more about the scientific rigor of the experiment than they do about the results of the project. Good luck finding that with real-world projects. jerf's approach gives you data that is less certain than you would get from a real scientific experiment. But it gives you data that is better than nothing, gathered in the real world. And you can actually run the experiment (at least in some environments) without getting fired, which is more than is likely true with a pure "scientific method" approach.
  - sillysaurus3 10y ago
    jerf's approach gives you data that is less certain than you would get from a real scientific experiment. But it gives you data that is better than nothing, gathered in the real world. The "better than nothing" is precisely where this logic goes awry. It's usually better to admit what you're actually doing: Going with your gut instincts. Pretending you're doing science adds fake legitimacy to your words and makes it easy to fool people. Unfortunately, the person you have to be concerned about fooling is yourself.

js8 10y ago

I disagree. First of all, I am sure there are scientific ways to analyze things like this, scientists deal with phenomena like this all the time. But the main problem is - if people don't actually know if the standup meeting helps, they shouldn't recommend it as a part of SCRUM (or other methodology). I know that in Agile, it's popular to say "if you think it doesn't work, change it". It's a first step towards scientific approach, but also very naive step - it can easily cause you to create rituals with no effects, just based on anecdotal coincidences. It is kind of disconcerting. On one hand, I applaud the excitement of Agile fans for critical thinking and scientific method, but at least they could study it a little - to find out that there is a little more to it than just "if it doesn't work change it". And that they shouldn't propose things that are not properly justified.