6 ms·

Tyranny of the Tools

182 points by DanielBMarkham 15y ago

InclinedPlane 15y ago
A process that doesn't self-correct is not a good process. A process that doesn't contain mechanisms which naturally guide teams back on track is not a good process. A process that tends to be followed incorrectly and misused to a degree such that it becomes a hindrance (even relative to no process at all), even in the hands of teams operating earnestly and with good intentions, is not a good process. A process that succeeds only when used by the best teams filled with the best developers is not a good process (such teams will succeed regardless). Agile is not a good process. It may contain elements which are useful, but on balance the history of agile has been a history of abuse and of use as an excuse for old fashioned micro-management. The infatuation with agile has set back the practice of software engineering in industry by a disheartening degree. The future is not "do whatever you want" nor is it "revert to the bad, old waterfall days" as agile zealots try to scare everyone into believing, but the sooner we get past the religion of "agile" the better.
- moocow01 15y ago
  It has actually been pretty demoralizing to me participating in agile techniques as a professional. Ive been in SCRUM meetings where everyone would stand in a circle and the project manager would bring in a timer and time everyone when they started speaking. If you went over 2 minutes the PM would immediately cut you off and move to the next person. On other occasions Ive had PMs ask me (in an accusatory manner) why it has taken me 4 hours to do a task I estimated at 3 hours. Keep in mind that these experiences were at a relatively small startup not a mega-corp. I don't know of any other creative, well-paid, educated profession where this would be even close to tolerated and I think its a trajedy that this sort of stuff has become common place. Unfortunately for the company the way I and other coworkers "self-corrected" the process is by moving on to another place where that doesn't occur - ended up being a great solution for me.
  - InclinedPlane 15y ago
    A few of the agile "standups" I've been involved in have been run well, at a recent job I had the opposite problem to what you describe. For several months we had a team that was about a dozen folks, and a daily standup that typically clocked in at 45 minutes to an hour and a half, every day. It boggles my mind how any manager can see 60 dev-hours a week get pissed away and fail to see the importance of it.
mbesto 15y ago
Definitely agree. What do people think about remote teams?
- TDL 15y ago
  I managed a team in the Philippines remotely from the U.S. At the same time the rest of the firm was in NYC while I lived in Chicago. The remote experience was not exactly a positive one. Although remote teams and workers are a bigger part of business life today, I would avoid it when possible. It's very hard to get everyone on the same tempo of work and it is really easy to have a miscommunication.
  - benvanderbeek 15y ago
    I also had a nightmare managing programmers in the Philippines, 75% from my incompetency, 25% from other factors. Having my guys w/in a few feet is invaluable to me now.
- bh42222 15y ago
  Webcam, skype, daily stand-up. And also agree with Daniel, there is a powerful simplicity in post-it notes. Automated tools on the other hand never improve your process. At best, at best they automate it and everything that is currently broken about it.
  - raganwald 15y ago
    Can’t remember where I read it, but I recall a story about automating air traffic control, and one of the problems they encountered was that there is a lot of information being conveyed by watching people grab the little strips of information and move them around. The state of the system is only one dimension, who’s interested in what and what they are doing is another. The claim was that humans are optimized for watching other people work, and most office automation hides all that: You never see Snively grab the Horrocks file and open it on his desk. Anecdotally, I have seen benefits from physical cards: People might see someone grab a card and speak up “Hey, are you looking at Thermoregulating Widgets? I was talking to Phred and he said...” This information is theoretically available when someone clicks “start” on a task, but somehow it is not as visceral as watching someone grab or move a note card.
SKoschnicke 15y ago
I think younger generations are getting more and more used to a digital-only workflow. I see also a shift in working habits from nine-to-five-office-work to asynchronical world-distributed work, which demands a such a workflow.
- SoftwareMaven 15y ago
  One of the things I really like about scrum is that it forces people to interact together. That builds team trust and solidarity. Those can be built in other ways, but it is much harder. If you want a great team, you need to build it somehow.
SoftwareMaven 15y ago
Whenever I think about process and tool changes, in addition to thinking about what I want to work better, I think about what I don't want to change. If something is working well and you don't strive to keep it working while monkeying with how the "process" works, you will surely break it. I saw an entire team self-destruct while building a product because management (including the inexperienced dev lead) kept wanting just a little more clarity into what was going on. Tweak after tweak increasing process, planning, and reporting at the expense of getting stuff done until the debs (myself included) couldn't take it any more. If you don't try very hard to keep what is working, you will lose it. Often, what is working is more important than what is not.
andrewvc 15y ago
It's just as easy to ignore a physical board. I've done it. There's no substitute for discipline.
- InclinedPlane 15y ago
  Whether it's a board or a spreadsheet the point is that it's just a crutch, what works is facilitating communication, breaking work down into manageable chunks, and facilitating a conversation with the whole team on every aspect of work.
- Semiapies 15y ago
  Very true. Even making the digital physical (say, printing out an email and putting on a stand on your desk) can be blown off. I can certainly buy crappy tools screwing up a process, though - much of my job is making existing tools less crappy and less disruptive to processes (or replacing them). The answer is probably to use simple tools - whatever gets the board or what you need in a convenient form without taking up much time. Stuff that doesn't have to be polished and that you determinedly don't spend much time tweaking and enhancing. I don't want to use pen and paper and a physical timer by my desk, but I don't need a "Pomodoro Technique(tm) Management System" when I can just use org-mode and a little timer script.
kabdib 15y ago
Agile started to die in my division the moment we had to go to some buggy web page and fill out our progress. Agile became just another vehicle for micromanagement. Our scrums became apologies against what we said we'd accomplish. Meetings were held by managers, outside the group, to discuss concerns about meeting our goals. There were levels of permissions on the task list; the team couldn't edit tasks, and had to get permission to make changes. Pretty soon we weren't deciding what to do and how long it would take, all that was being done for us, by managers. Scrum became another set of manacles, and something to be avoided at all cost.
akeefer 15y ago
This pretty much exactly describes the experiences that our various teams have had. We've tried all manner of software tools over the years (Jira, since we use that for bug tracking already, Jira + Greenhopper, Agile Zen, Pivotal Tracker) and we pretty much always end up coming back to having cards on a board. Every tool ends up feeling too restrictive, too big brother, and too TPS-report-coversheet. The point starts to become "get the right data into the tool" instead of "use the tool to help us do our work." We generally end up putting everything in Jira anyway for historical and statistical purposes, as well as for visibility when people are working remotely, and it simply ends up as someone's job to make sure Jira is in sync with the physical board every so often.
- nkohari 15y ago
  Sorry that that was your experience with AgileZen. Organizing work is a surprisingly difficult problem, and one for which we're very interested in providing a decent solution. I'd love to hear about your experiences and how we can do better. My email is (same username) at gmail.com, if you're interested. :)
johnwatson11218 15y ago
I am a big fan of plain text documents. Where I work we have a tool that collects requirements and then they can all be exported into Word. The problem is that the tool takes these one and two line sentences and produces a document out of them. In between each blurb there is about 3-5 lines of redundant boiler plate about who wrote the two lines of text and when. It is almost unreadable in that form. Things like keeping a consistent verb tense/voice go out the window. When I come on to a new project I like to see a well written document that has an introduction and several well written pages. I can read that and get the rest from the code. The issue is that we don't need expensive tools to write text files.
nhashem 15y ago
$144.00 $144.00 $14.14 $10.82 $60.00 Wow. This is such a great post, and sums up exactly why every implementation of "Agile" I've seen at previous employers have been completely dysfunctional. So there's an engineering team and they have a project manager. The post-its and standups and storyboards seem a little weird at first to the engineering team, but they go along with it. It's rarely perfect in the first iteration but they can see how doing this makes them more productive, and it beats getting a bunch of confused requirements from confused business owners, or working off cumbersome and inflexible product requirement documents. And it doesn't ask them to change much in the ways they're already communicating. The problem is the project manager needs a way to provide "remote views" that the OP mentioned. The CEO needs to know where the implementation hours are being spent. The finance team wants to know which projects they can capitalize. And so forth. It is part of the project manager's responsibility to provide this. This is where tools can be great. Enter in some data, enter in the stories and tasks and points and hours and projects and generate a dozen charts and graphs with the push of a button. Cool. The problem is if the project manager doesn't manage this themselves, if they don't do the tedious task of entering in this information themselves, then it's going to be a mess. Because like the OP said, engineers aren't going to update the tool. They want to do work and if the work status needs to be updated to reflect their progress, that should just automatically happen. If they have to manage the updating, then you immediately have a point of failure because the engineer is never going to do this. Why? Because it's pointless. He's working on projects. He's cranking out code. He already has a dozen windows open on his desktop at all times to text editors, terminal shells, documentation on the web, documentation in PDFs. He already communicates to his peers via IM, email, in person, using version control, or even moving post-its on a storyboard. He is already passively indicating his status a dozen times a day. Why does he need to actively manage his status in some tool that has zero value to him or anyone he works with? But the project manager implores, "just update your hours, guys, it only takes a little bit of time each day. We really want to maintain an accurate burndown chart." If only it were so simple. Because beyond being completely inconvenient, the tool doesn't even use the same units of measure that he does! "Hours per story?" Does that count the hours brainstorming the solution? The hours actually typing in code? The hours working with the sysadmins to deploy the code? The hours (well, hopefully minutes) spent thinking about the project while taking a dump? If the story is closely related to another one that he basically worked on both at the same time, should he combine his hours and divide by two? And burndown chart? Does the burndown chart mean anything to him? Does it help or improve his life in anyway? We thought this project was going to take a week, but when we started getting into the implementation we realized there's huge hurdle X and now it's going to take two weeks. There's your damn burndown chart. So the engineer hears, "just wax and shave your balls with vaseline, guys, it only takes a little bit of time each day." Because waxing and shaving his balls seems just as pointless as updating the tool. And even if it only takes a few minutes to wax and shave your balls, it's awkward enough that you will get really frustrated really quickly if this needs to become part of your workflow. If the project manager wants to the use the tool because of the convenience and power of generating reporting charts and graphs to external teams, then good for them. But if they're going to ask the engineers to update it themselves, then they should be prepared for that to be as successful as if they asked them to shave their balls, because that is literally what they are doing.
moocow01 15y ago
The more and more I work in Agile the more I realize that the way many companies use it it is just an excuse to do micromanagement and attempt to treat everyone as an interchangeable resource/cog. Now this doesn't mean agile is bad, it just means for whatever reason management in many of my experiences has twisted it into a tool that seemingly leads to the following: - Change direction every sprint to the point where an application has been morphed so many times into something new that it makes it a heroic task to actually engineer something that ends up performant without needing a complete rewrite - Takes the pressure of the business/management folks to project, plan or foresee anything because they believe they can change direction at any point in time - Cut down projects into such fine grained tasks that it takes the fun out of the work Im sure in those cases agile was done wrong for whatever reason or another but consequently as a result of my experiences I've stopped putting up with it when I can. I'm definitely ignorant about project management but Ive found within a small team just talking and coordinating like normal human beings about how your going to work together and what your going to do works pretty well. I guess I'd call my approach the "no project manager" technique.
- smhinsey 15y ago
  I used to work at a company that absolutely loved to do a variation on your first point. When faced with a multiple choice question, the answer would end up being "all of the above," but framed in such a way that "oh, we're merely keeping our options open." When that story comes up for implementation, what do you know but apparently MajorCustomer has heard all about how we're offering support for every computer ever made by human hands and is thrilled! They want to know when they can expect a demo! Agile can also be a great way to uncover people who think they are "separate but more equal" too, when they decide that the concept of a sprint whose content is mutually agreed upon and doesn't change mid-stream is just a quaint little detail that doesn't apply to Important People. Maybe I'm just a little bit bitter.
dreamdu5t 15y ago
Every place I've worked where we tracked our hours, only proved that realistically estimating hours on a software project was completely impossible.
carsongross 15y ago
While I agree with the author generally, I should say the one online tool I've found that has really helped me, particularly when I'm doing remote development with other people, is workflowy. It's just a hierarchical list of stuff and you can mark items as being done. The radical simplicity (and slick UI) stays out of my way, and yet provides a centralized list of stuff to do for the group I'm working with. It's kind of like a shared emacs org-mode on the web. My main concern is that they will add more features to it.

182 points by DanielBMarkham 15y ago

A process that doesn't self-correct is not a good process. A process that doesn't contain mechanisms which naturally guide teams back on track is not a good process. A process that tends to be followed incorrectly and misused to a degree such that it becomes a hindrance (even relative to no process at all), even in the hands of teams operating earnestly and with good intentions, is not a good process. A process that succeeds only when used by the best teams filled with the best developers is not a good process (such teams will succeed regardless). Agile is not a good process. It may contain elements which are useful, but on balance the history of agile has been a history of abuse and of use as an excuse for old fashioned micro-management. The infatuation with agile has set back the practice of software engineering in industry by a disheartening degree. The future is not "do whatever you want" nor is it "revert to the bad, old waterfall days" as agile zealots try to scare everyone into believing, but the sooner we get past the religion of "agile" the better.

moocow01 15y ago
It has actually been pretty demoralizing to me participating in agile techniques as a professional. Ive been in SCRUM meetings where everyone would stand in a circle and the project manager would bring in a timer and time everyone when they started speaking. If you went over 2 minutes the PM would immediately cut you off and move to the next person. On other occasions Ive had PMs ask me (in an accusatory manner) why it has taken me 4 hours to do a task I estimated at 3 hours. Keep in mind that these experiences were at a relatively small startup not a mega-corp. I don't know of any other creative, well-paid, educated profession where this would be even close to tolerated and I think its a trajedy that this sort of stuff has become common place. Unfortunately for the company the way I and other coworkers "self-corrected" the process is by moving on to another place where that doesn't occur - ended up being a great solution for me.
- InclinedPlane 15y ago
  A few of the agile "standups" I've been involved in have been run well, at a recent job I had the opposite problem to what you describe. For several months we had a team that was about a dozen folks, and a daily standup that typically clocked in at 45 minutes to an hour and a half, every day. It boggles my mind how any manager can see 60 dev-hours a week get pissed away and fail to see the importance of it.

mbesto 15y ago

Definitely agree. What do people think about remote teams?

TDL 15y ago
I managed a team in the Philippines remotely from the U.S. At the same time the rest of the firm was in NYC while I lived in Chicago. The remote experience was not exactly a positive one. Although remote teams and workers are a bigger part of business life today, I would avoid it when possible. It's very hard to get everyone on the same tempo of work and it is really easy to have a miscommunication.
- benvanderbeek 15y ago
  I also had a nightmare managing programmers in the Philippines, 75% from my incompetency, 25% from other factors. Having my guys w/in a few feet is invaluable to me now.
bh42222 15y ago
Webcam, skype, daily stand-up. And also agree with Daniel, there is a powerful simplicity in post-it notes. Automated tools on the other hand never improve your process. At best, at best they automate it and everything that is currently broken about it.
- raganwald 15y ago
  Can’t remember where I read it, but I recall a story about automating air traffic control, and one of the problems they encountered was that there is a lot of information being conveyed by watching people grab the little strips of information and move them around. The state of the system is only one dimension, who’s interested in what and what they are doing is another. The claim was that humans are optimized for watching other people work, and most office automation hides all that: You never see Snively grab the Horrocks file and open it on his desk. Anecdotally, I have seen benefits from physical cards: People might see someone grab a card and speak up “Hey, are you looking at Thermoregulating Widgets? I was talking to Phred and he said...” This information is theoretically available when someone clicks “start” on a task, but somehow it is not as visceral as watching someone grab or move a note card.

SKoschnicke 15y ago

I think younger generations are getting more and more used to a digital-only workflow. I see also a shift in working habits from nine-to-five-office-work to asynchronical world-distributed work, which demands a such a workflow.

SoftwareMaven 15y ago
One of the things I really like about scrum is that it forces people to interact together. That builds team trust and solidarity. Those can be built in other ways, but it is much harder. If you want a great team, you need to build it somehow.

SoftwareMaven 15y ago

Whenever I think about process and tool changes, in addition to thinking about what I want to work better, I think about what I don't want to change. If something is working well and you don't strive to keep it working while monkeying with how the "process" works, you will surely break it. I saw an entire team self-destruct while building a product because management (including the inexperienced dev lead) kept wanting just a little more clarity into what was going on. Tweak after tweak increasing process, planning, and reporting at the expense of getting stuff done until the debs (myself included) couldn't take it any more. If you don't try very hard to keep what is working, you will lose it. Often, what is working is more important than what is not.

andrewvc 15y ago

It's just as easy to ignore a physical board. I've done it. There's no substitute for discipline.

InclinedPlane 15y ago
Whether it's a board or a spreadsheet the point is that it's just a crutch, what works is facilitating communication, breaking work down into manageable chunks, and facilitating a conversation with the whole team on every aspect of work.
Semiapies 15y ago
Very true. Even making the digital physical (say, printing out an email and putting on a stand on your desk) can be blown off. I can certainly buy crappy tools screwing up a process, though - much of my job is making existing tools less crappy and less disruptive to processes (or replacing them). The answer is probably to use simple tools - whatever gets the board or what you need in a convenient form without taking up much time. Stuff that doesn't have to be polished and that you determinedly don't spend much time tweaking and enhancing. I don't want to use pen and paper and a physical timer by my desk, but I don't need a "Pomodoro Technique(tm) Management System" when I can just use org-mode and a little timer script.

kabdib 15y ago

Agile started to die in my division the moment we had to go to some buggy web page and fill out our progress. Agile became just another vehicle for micromanagement. Our scrums became apologies against what we said we'd accomplish. Meetings were held by managers, outside the group, to discuss concerns about meeting our goals. There were levels of permissions on the task list; the team couldn't edit tasks, and had to get permission to make changes. Pretty soon we weren't deciding what to do and how long it would take, all that was being done for us, by managers. Scrum became another set of manacles, and something to be avoided at all cost.

akeefer 15y ago

This pretty much exactly describes the experiences that our various teams have had. We've tried all manner of software tools over the years (Jira, since we use that for bug tracking already, Jira + Greenhopper, Agile Zen, Pivotal Tracker) and we pretty much always end up coming back to having cards on a board. Every tool ends up feeling too restrictive, too big brother, and too TPS-report-coversheet. The point starts to become "get the right data into the tool" instead of "use the tool to help us do our work." We generally end up putting everything in Jira anyway for historical and statistical purposes, as well as for visibility when people are working remotely, and it simply ends up as someone's job to make sure Jira is in sync with the physical board every so often.

nkohari 15y ago
Sorry that that was your experience with AgileZen. Organizing work is a surprisingly difficult problem, and one for which we're very interested in providing a decent solution. I'd love to hear about your experiences and how we can do better. My email is (same username) at gmail.com, if you're interested. :)

johnwatson11218 15y ago

I am a big fan of plain text documents. Where I work we have a tool that collects requirements and then they can all be exported into Word. The problem is that the tool takes these one and two line sentences and produces a document out of them. In between each blurb there is about 3-5 lines of redundant boiler plate about who wrote the two lines of text and when. It is almost unreadable in that form. Things like keeping a consistent verb tense/voice go out the window. When I come on to a new project I like to see a well written document that has an introduction and several well written pages. I can read that and get the rest from the code. The issue is that we don't need expensive tools to write text files.

nhashem 15y ago

$144.00 $144.00 $14.14 $10.82 $60.00 Wow. This is such a great post, and sums up exactly why every implementation of "Agile" I've seen at previous employers have been completely dysfunctional. So there's an engineering team and they have a project manager. The post-its and standups and storyboards seem a little weird at first to the engineering team, but they go along with it. It's rarely perfect in the first iteration but they can see how doing this makes them more productive, and it beats getting a bunch of confused requirements from confused business owners, or working off cumbersome and inflexible product requirement documents. And it doesn't ask them to change much in the ways they're already communicating. The problem is the project manager needs a way to provide "remote views" that the OP mentioned. The CEO needs to know where the implementation hours are being spent. The finance team wants to know which projects they can capitalize. And so forth. It is part of the project manager's responsibility to provide this. This is where tools can be great. Enter in some data, enter in the stories and tasks and points and hours and projects and generate a dozen charts and graphs with the push of a button. Cool. The problem is if the project manager doesn't manage this themselves, if they don't do the tedious task of entering in this information themselves, then it's going to be a mess. Because like the OP said, engineers aren't going to update the tool. They want to do work and if the work status needs to be updated to reflect their progress, that should just automatically happen. If they have to manage the updating, then you immediately have a point of failure because the engineer is never going to do this. Why? Because it's pointless. He's working on projects. He's cranking out code. He already has a dozen windows open on his desktop at all times to text editors, terminal shells, documentation on the web, documentation in PDFs. He already communicates to his peers via IM, email, in person, using version control, or even moving post-its on a storyboard. He is already passively indicating his status a dozen times a day. Why does he need to actively manage his status in some tool that has zero value to him or anyone he works with? But the project manager implores, "just update your hours, guys, it only takes a little bit of time each day. We really want to maintain an accurate burndown chart." If only it were so simple. Because beyond being completely inconvenient, the tool doesn't even use the same units of measure that he does! "Hours per story?" Does that count the hours brainstorming the solution? The hours actually typing in code? The hours working with the sysadmins to deploy the code? The hours (well, hopefully minutes) spent thinking about the project while taking a dump? If the story is closely related to another one that he basically worked on both at the same time, should he combine his hours and divide by two? And burndown chart? Does the burndown chart mean anything to him? Does it help or improve his life in anyway? We thought this project was going to take a week, but when we started getting into the implementation we realized there's huge hurdle X and now it's going to take two weeks. There's your damn burndown chart. So the engineer hears, "just wax and shave your balls with vaseline, guys, it only takes a little bit of time each day." Because waxing and shaving his balls seems just as pointless as updating the tool. And even if it only takes a few minutes to wax and shave your balls, it's awkward enough that you will get really frustrated really quickly if this needs to become part of your workflow. If the project manager wants to the use the tool because of the convenience and power of generating reporting charts and graphs to external teams, then good for them. But if they're going to ask the engineers to update it themselves, then they should be prepared for that to be as successful as if they asked them to shave their balls, because that is literally what they are doing.

moocow01 15y ago

The more and more I work in Agile the more I realize that the way many companies use it it is just an excuse to do micromanagement and attempt to treat everyone as an interchangeable resource/cog. Now this doesn't mean agile is bad, it just means for whatever reason management in many of my experiences has twisted it into a tool that seemingly leads to the following: - Change direction every sprint to the point where an application has been morphed so many times into something new that it makes it a heroic task to actually engineer something that ends up performant without needing a complete rewrite - Takes the pressure of the business/management folks to project, plan or foresee anything because they believe they can change direction at any point in time - Cut down projects into such fine grained tasks that it takes the fun out of the work Im sure in those cases agile was done wrong for whatever reason or another but consequently as a result of my experiences I've stopped putting up with it when I can. I'm definitely ignorant about project management but Ive found within a small team just talking and coordinating like normal human beings about how your going to work together and what your going to do works pretty well. I guess I'd call my approach the "no project manager" technique.

smhinsey 15y ago
I used to work at a company that absolutely loved to do a variation on your first point. When faced with a multiple choice question, the answer would end up being "all of the above," but framed in such a way that "oh, we're merely keeping our options open." When that story comes up for implementation, what do you know but apparently MajorCustomer has heard all about how we're offering support for every computer ever made by human hands and is thrilled! They want to know when they can expect a demo! Agile can also be a great way to uncover people who think they are "separate but more equal" too, when they decide that the concept of a sprint whose content is mutually agreed upon and doesn't change mid-stream is just a quaint little detail that doesn't apply to Important People. Maybe I'm just a little bit bitter.

dreamdu5t 15y ago

Every place I've worked where we tracked our hours, only proved that realistically estimating hours on a software project was completely impossible.

carsongross 15y ago

While I agree with the author generally, I should say the one online tool I've found that has really helped me, particularly when I'm doing remote development with other people, is workflowy. It's just a hierarchical list of stuff and you can mark items as being done. The radical simplicity (and slick UI) stays out of my way, and yet provides a centralized list of stuff to do for the group I'm working with. It's kind of like a shared emacs org-mode on the web. My main concern is that they will add more features to it.