3 ms·
"Experiments would take at least several months to run, because it's not till several months that you know whether your business is going to fail solely due to
by jerf 10y ago
"Experiments would take at least several months to run, because it's not till several months that you know whether your business is going to fail solely due to the dev team."
If you use the rather stodgy and formulaic scientific method, sure. But if you approach this as a multi-arm bandit problem, and instead look at the question as "how much information do I need to collect with how much statistical power before I am justified in selecting an outcome?", it becomes quite tractable for a team to just try it out.
It also helps if instead of using merely "productivity" as your one and only goal that you also consider team happiness as a real and going concern. Then "Nobody likes it and it's demoralizing the engineers" becomes a perfectly valid form of feedback to use in the decision about whether to do it.
I also tend to use the metric that the brain is really, really, really good at figuring out what's important, sometimes perhaps even too good, so if we start a stand-up meeting plan, and it just sort of peters out, then I let it go. (Seems like somebody wants to try it every 2 or 3 years or so, I generally just let it happen and then evaluate.) Everybody's brains are coming to the conclusion that it's not important. Rather than fighting it, roll with it. There are some things in life where your rational forebrain must override the deeper instinctive algorithms about what is important, but that's quite cognitively expensive, and I've never seen any reason to believe that stand-up meetings are bringing enough value to justify that.
I emphasize the "I've never seen any reason". I hypothesize that in many ways, the work the teams I've been on have been doing is almost maximally pathologically bad for a stand-up meeting. Almost everyone is working on something independent of everybody else, so blockers are only rarely even possible. If your team is, on the other hand, chock-a-block full of blocking potential it may be useful to you.
Anyways, there's more ways to find out the truth of things that trying to split the universe into two precisely equal parts, make a change in only one of them, and come back ten years later to see what happened, only for someone to complain that a sample size of one isn't enough and we really need to do this a thousand more times to have a valid result.
- sillysaurus3 10y agoIf you use the rather stodgy and formulaic scientific method, sure. You mean the scientific method that has proven to work since the 1700's? I think we'll have to agree to disagree that you can just riff it and still get results.
- jerf 10y agoBummer, I wish I'd have caught this earlier. Look up "multi-armed bandit"; it isn't just "riffing it" like you think it is. It is a mathematically-principled technique. In fact, it is probably safe to say it's more mathematically-principled than the "scientific method" itself, as it is popularly understood.
- AnimalMuppet 10y agoSo what would the scientific method have you do here? Take, say, 100 projects. On half of them, do standups. On the other half, don't. Measure the outcomes. Do statistics. The problem is, you'd have to find 100 projects where they care more about the scientific rigor of the experiment than they do about the results of the project. Good luck finding that with real-world projects. jerf's approach gives you data that is less certain than you would get from a real scientific experiment. But it gives you data that is better than nothing, gathered in the real world. And you can actually run the experiment (at least in some environments) without getting fired, which is more than is likely true with a pure "scientific method" approach.
- sillysaurus3 10y agojerf's approach gives you data that is less certain than you would get from a real scientific experiment. But it gives you data that is better than nothing, gathered in the real world. The "better than nothing" is precisely where this logic goes awry. It's usually better to admit what you're actually doing: Going with your gut instincts. Pretending you're doing science adds fake legitimacy to your words and makes it easy to fool people. Unfortunately, the person you have to be concerned about fooling is yourself.
- AnimalMuppet 10y ago> It's usually better to admit what you're actually doing: Going with your gut instincts. Not at all. If you're doing standups, try not doing them for the next month. If you're paying attention, you'll know if things got worse or better. Yes, that's anecdote rather than data. Yes, it's not statistically significant. Yes, it may be different outside your specific circumstances. But no, it's not just gut instincts - not if you're any good as a manager.