5 ms·

The most important lesson for people in power in big tech companies: FORMAL PROCESS IS THE ENEMY. Don't let the accumulation of gatekeepers and approvers and ot

The most important lesson for people in power in big tech companies: FORMAL PROCESS IS THE ENEMY. Don't let the accumulation of gatekeepers and approvers and other capital-P Process let things become 100x harder than they need to be. Do not confuse difficulty with prudence. Your organization will become slow and ossified by default unless you take specific steps to maintain flexibility. One of these steps is to install a culture of change. Things that are easily undone should not require approval to be done in the first place. Do not allow long-timers in your organization use "caution" or "good engineering practice" as an excuse to slow everyone else down. Emphasize that the most important part of software development is moving fast. Let new people try new things. Minimize the number of people who can say "no". Most of all, do not just _believe_ people who talk about best practices and software "quality" and stuff like that. Most of the time, they're just finding fancy-sounding ways of saying "nothing should change unless I say so".

kbos87 9y ago
I have a hunch that you've never worked at a growing mid-sized or large company that is clinging to its startup roots and is averse to any sort of process at all. Being unwilling to adopt a necessary amount of structure and process is just as much of a hindrance once you hit a certain size. That may be a newer or less common problem, but it's a very real one.
- el_benhameen 9y ago
  Agreed. I've often found that those who are resistant to any amount of (reasonable) formal process are those (usually long timers) who were most empowered by the implicit process and rules. In the end, they're also saying "nothing should change unless I say so".
  - abraae 9y ago
    Very insightful.
  - 35bge57dtjku 9y ago
    This and the parent comment describe exactly what happened at the company I just left. I wonder if the newly hired TPM will ever have any real power there.
- linker3000 9y ago
  I can totally identify with this. I joined a company that had grown from 2-3 people in a shed to 40-50, providing global B2C, B2B and company Intranet design and development (A Drupal house). I was tasked with providing some structure and process to (initially) the support team in order for the company to be in a position to cope with expanding workload and look 'corporate' to chase larger accounts. When I joined, the company was pretty much firefighting all the time and diverting people from project to project according to which customer shouted the loudest at the time. Every change I proposed was questioned to death and generally dismissed, I was micro-managed within an inch of my life, and I found the two key Directors just could not let-go of any control of their 'baby' (the company), to a point where any tasks I tried to assign to team members were met with pushback because the Directors had already assigned them other things, told them to work on other projects or asked them to report back before accepting any work from me. One time when I was asked to work on a proposal to take over the migration, management and support of the Intranet for a large local University, I completed the initial RFP document and reported back to the Directors with a plan for a meetup with the potential customer because some key points needed elaboration, and some of the SLA terms could not be met without additional team resource, but they told me they had already phoned the Uni and agreed to go ahead 'as is'. I left after 9 months.
dclowd9901 9y ago
Risk is the enemy of establishment and profit. You _will not_ convince a powerful train to change direction simply because there's no tracks ahead.
banned1 9y ago
"Emphasize that the most important part of software development is moving fast." No, that is not the case most times. The most important part of software development is, as with any other part of the company, to maximize the value of the company. That may be speed, but if you have 1m users the most important thing may be to preserve trust, to acquire new customers, to automate existing infrastructure, etc. etc. Speed of software development is not a goal in many environments. Thoughtful execution may be hindered by a cowboy attitude in development. I'm not saying that speed in development is not sometimes the goal (maybe in startups within an enterprise trying to test an idea), but often it's not the most important thing.
- deleted 9y ago
  [deleted]
- inopinatus 9y ago
  "Maximising company value" isn't a useful end goal you can give to anyone but the CxO team. For everyone else, it's a motherhood statement. Trotting it out as though it's some kind of engineering compass isn't helpful. To this you've appended the hackneyed conflation between "speed" and "cowboy attitude". It's a common category error, but in truth, rapid iteration is one of the enablers of high quality and is a huge effectiveness multiplier for top shelf dev teams. Slow development merely enables lazy programmers (since it conceals their apathy) and gives us waterfall disasters. "Thoughtful execution" is possible in any context, and most thought processes are greatly enhanced by evidential techniques such as rapid prototyping / feature spiking. Fast+incompetent = cowboy attitude, sure, but the problem there isn't the fast bit.
  - geofft 9y ago
    I've found maximizing company value to be a pretty good end goal as an individual contributor (and it's definitely the thing I'm most excited about doing when I walk in the door in the morning). It's how you tell whether "fix this bug" or "implement this feature" is higher priority, or whether "do this thing now in our current system" or "experiment with this completely different system" is higher priority. You can do all of those at speed; the question is which to do. And getting it wrong dooms your company, because you're wasting effort. In a well-functioning company, I don't need to make that decision myself, of course: I have management who has evaluated the cost of having the bug and the benefit of having the feature, or who's experienced and smart enough to determine whether doing the short-term thing now or investing in the longer-term thing will be likely to have better payoff. Because this process is there, if I trust my management, I don't have to spend-time second-guessing them (or worse, doing the research myself), and then I get to move quickly. But again, moving quickly isn't the point. It's a tool on the way to delivering value.
ryandrake 9y ago
I didn't take that away from the article at all. This wasn't an argument against formal process. It was more a story about acting like you're a big company, while pretending you're a small company, and the pitfalls of doing this. Where "acting" encompasses pay, stock incentives, other kinds of talent motivation, recruiting, leadership style, innovation, and yes formal process. There's nothing wrong with process and best practices. Process is often scar tissue left behind after critical mistakes in the past. A company without some formal process is a company young enough to not have made disastrous mistakes yet. I've worked in a company of more than 50 that was basically without a trace of software development process. It was a clown circus. Nothing could be released on time reliably, quality was a joke, there was no documentation, no code review, and no plan or roadmap. Sure, they moved fast, but it didn't matter.
erikpukinskis 9y ago
I just learned about Hegel so now everything is a nail, but... You really can't make categorical statements like "formal process is the enemy". You put formal process into place, and that creates a dialectic where you're waiting for someone to come in and champion the antithesis: "formal process is the enemy". Then that works for a bit until it doesn't and someone synthesizes: "sometimes formal is better sometimes informal is better". Then that becomes the thesis. Then there's a new dialectic in place. Someone will come in and say "the character of our company is such that we're generally better with formal processes". Then someone synthesizes that, and suggests that all processes are on a trajectory from formal to informal and the company is good at expediting that. Then someone realizes we need a process for regular deformalization. This continues forever. None of these ideas are really that much more generically true than any other. So to embrace Hegel, the goal is just to question yourself as far as you reasonably can, so as to get the best view on the current situation and only the current situation. Make a choice and move on, knowing the next situation is likely to be different. Edit: I should add this is the process by which we hurry into The Future, which some people might point out means turning our backs on Everything That Is Holy and is therefore a tool of Satan, but I'll leave you to explore that with the rapidly growing universe of anti-Hegelian YouTubers.
- freekh 9y ago
  I really liked this comment :) I am looking forward to pondering upon this way of reasoning for this entire Saturday :)
jjtheblunt 9y ago
That's exactly why I left Apple engineering, and I've not regretted it, somewhat sadly, somewhat happily.
geofft 9y ago
> Emphasize that the most important part of software development is moving fast. This doesn't seem to me to be true. Can you expand on your thoughts here? I'd agree that the ability to move fast is an essential part of effective software development, but I don't think it's the most important (the most important is producing business value accurately and reliably), and I think that moving fast for the sake of moving fast is likely to result in moving quickly in the wrong direction. In particular, it's easy to be so worried about moving fast that you have no time to take new business needs/opportunities into account (see also Tom DeMarco's Slack), and it's easy to get excited about shiny new technology and spend a lot of time implementing it and roleplaying Google for no reason, which is a great way to move very quickly while not delivering a cent of business value. Finding somewhere with formal process, with full-time managers who aren't reluctantly moved to management for career advancement, with clear hierarchy and goals, was an important part of my most recent job search, simply because, as a good engineer, I hate to see my work go to waste. I have heard the same sentiment from lots of my talented friends.
- quotemstr 9y ago
  Unless you take conscious steps to remain flexible and to eliminate barriers to productivity, you're going to find your ability to "produc[e] business value" diminished because effort will get sucked into process instead, and for reasons nobody can identify. Soon enough, you'll find yourself wondering why you have so many developers, all of whom appear to be doing work, yet without getting anything done. By default, all software development organizations slow down. I say that developer velocity is the most important goal because, without focusing on this velocity, you end up solidifying, like a cooling lava, and it becomes difficult to accomplish any of your other goals.
  - geofft 9y ago
    Well, okay, you and I are agreeing that velocity is nice to have, but why is it the most important goal? I can make the exact same argument about automated regression tests. As the size of your codebase grows and the number of customers (i.e., people triggering edge cases) grows, unless you take conscious steps to make sure that every build doesn't regress previous bugs, all your effort will get sucked into diagnosing and fixing things that you've fixed before. And all your developers will be fixing bugs but you won't be fixing new bugs and certainly not shipping features. In particular, if you want to refactor / rewrite parts of your code to improve development velocity, you'll suffer from the well-documented problem (see e.g. https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-... ) of losing all the knowledge of the weird edge cases you've fixed along the way. So, testing is strictly more important than development velocity! I can make the same argument about a half-dozen other development best practices, and I'll have actual citations to back them up. Why is development speed the most important goal, why is formal process the enemy of speed (instead of its friend, because it makes sure work doesn't get duplicated or wasted!), and what is your evidence for these claims? In particular, if your claim is true, we would not expect to see as much feature development / new products from old, large companies (I'm thinking of Oracle, Microsoft, Apple, Amazon, etc.) as we do. We would expect to see these companies grind to a halt and perform poorly in the markets (investors really want to see growth, not just continuing to be as good as you were last year), and that's not what's happening.
sanityUnbounded 9y ago
I recommend Creativity, Inc. to every person in a management position whom I have the opportunity to recommend it to. Although Et Catmull presents a depiction of Pixar and it's growth as a largely artistic endeavor, I feel that it is applicable to those that manage engineers especially. The ratio of students that decided to major in engineering because they were good at math in high school vs. the students who decided to major in engineering because they they wanted to make something inspiring, I feel ( and let me stress my inexperience as a current undergrad, without experience in the actual field ) is vastly favored in career growth towards those that decide to create instead of simply learn. While there is always room and lucrativity to tie engineers on a leash and juice mathematics and logic out of them until they reach an existential crisis, there is always (ALWAYS) room within the budget to let them set out upon a path which is in tune with their not necessarily young, but foolish and hopeful beliefs.
rleigh 9y ago
Used appropriately, a formal process provides structure to the workflows within an organisation. Everyone knows the process, and it provides a framework and expectations for everyone to work within. In organisations without any process, it can lead to difficulty in getting stuff done because there's no responsibility or accountability for who is doing what, and when. Who is reviewing my PR, which CI systems tested and deployed the prospective change, why was the work done, when does it get merged, and by whom? Etc. Basic daily routine. You also have the situation that people invent rules and requirements in the absence of a documented process, which can change arbitrarily and are impossible to work with because it depends upon the changing whims of whoever feels they are in charge. Clairvoyancy and divination should not be a job requirement! Process becomes problematic when the process grows and ossifies and becomes the end itself, rather than the means to an end. You end up with it being impossible to get work done because the process is so burdensome. When your bosses care more about the process being followed to the letter rather than the actual completion of the work which the process should be enabling, that's IMO the point where it needs a rethink. Unfortunately, organisations have a tendency to accrete the stuff; I've in the past argued for less to the amazement and incredulity of my bosses. It's also a factor that process is used as a blunt instrument to wield power over others, and I think that's certainly a driver as well. But often organisations favour accountability and rules over creativity and efficiency, even if you spend 95% of your time with process, and 5% actually doing productive stuff...

kbos87 9y ago

I have a hunch that you've never worked at a growing mid-sized or large company that is clinging to its startup roots and is averse to any sort of process at all. Being unwilling to adopt a necessary amount of structure and process is just as much of a hindrance once you hit a certain size. That may be a newer or less common problem, but it's a very real one.

el_benhameen 9y ago
Agreed. I've often found that those who are resistant to any amount of (reasonable) formal process are those (usually long timers) who were most empowered by the implicit process and rules. In the end, they're also saying "nothing should change unless I say so".
- abraae 9y ago
  Very insightful.
- 35bge57dtjku 9y ago
  This and the parent comment describe exactly what happened at the company I just left. I wonder if the newly hired TPM will ever have any real power there.
linker3000 9y ago
I can totally identify with this. I joined a company that had grown from 2-3 people in a shed to 40-50, providing global B2C, B2B and company Intranet design and development (A Drupal house). I was tasked with providing some structure and process to (initially) the support team in order for the company to be in a position to cope with expanding workload and look 'corporate' to chase larger accounts. When I joined, the company was pretty much firefighting all the time and diverting people from project to project according to which customer shouted the loudest at the time. Every change I proposed was questioned to death and generally dismissed, I was micro-managed within an inch of my life, and I found the two key Directors just could not let-go of any control of their 'baby' (the company), to a point where any tasks I tried to assign to team members were met with pushback because the Directors had already assigned them other things, told them to work on other projects or asked them to report back before accepting any work from me. One time when I was asked to work on a proposal to take over the migration, management and support of the Intranet for a large local University, I completed the initial RFP document and reported back to the Directors with a plan for a meetup with the potential customer because some key points needed elaboration, and some of the SLA terms could not be met without additional team resource, but they told me they had already phoned the Uni and agreed to go ahead 'as is'. I left after 9 months.

dclowd9901 9y ago

Risk is the enemy of establishment and profit. You _will not_ convince a powerful train to change direction simply because there's no tracks ahead.

banned1 9y ago

"Emphasize that the most important part of software development is moving fast." No, that is not the case most times. The most important part of software development is, as with any other part of the company, to maximize the value of the company. That may be speed, but if you have 1m users the most important thing may be to preserve trust, to acquire new customers, to automate existing infrastructure, etc. etc. Speed of software development is not a goal in many environments. Thoughtful execution may be hindered by a cowboy attitude in development. I'm not saying that speed in development is not sometimes the goal (maybe in startups within an enterprise trying to test an idea), but often it's not the most important thing.

deleted 9y ago
[deleted]
inopinatus 9y ago
"Maximising company value" isn't a useful end goal you can give to anyone but the CxO team. For everyone else, it's a motherhood statement. Trotting it out as though it's some kind of engineering compass isn't helpful. To this you've appended the hackneyed conflation between "speed" and "cowboy attitude". It's a common category error, but in truth, rapid iteration is one of the enablers of high quality and is a huge effectiveness multiplier for top shelf dev teams. Slow development merely enables lazy programmers (since it conceals their apathy) and gives us waterfall disasters. "Thoughtful execution" is possible in any context, and most thought processes are greatly enhanced by evidential techniques such as rapid prototyping / feature spiking. Fast+incompetent = cowboy attitude, sure, but the problem there isn't the fast bit.
- geofft 9y ago
  I've found maximizing company value to be a pretty good end goal as an individual contributor (and it's definitely the thing I'm most excited about doing when I walk in the door in the morning). It's how you tell whether "fix this bug" or "implement this feature" is higher priority, or whether "do this thing now in our current system" or "experiment with this completely different system" is higher priority. You can do all of those at speed; the question is which to do. And getting it wrong dooms your company, because you're wasting effort. In a well-functioning company, I don't need to make that decision myself, of course: I have management who has evaluated the cost of having the bug and the benefit of having the feature, or who's experienced and smart enough to determine whether doing the short-term thing now or investing in the longer-term thing will be likely to have better payoff. Because this process is there, if I trust my management, I don't have to spend-time second-guessing them (or worse, doing the research myself), and then I get to move quickly. But again, moving quickly isn't the point. It's a tool on the way to delivering value.

ryandrake 9y ago

I didn't take that away from the article at all. This wasn't an argument against formal process. It was more a story about acting like you're a big company, while pretending you're a small company, and the pitfalls of doing this. Where "acting" encompasses pay, stock incentives, other kinds of talent motivation, recruiting, leadership style, innovation, and yes formal process. There's nothing wrong with process and best practices. Process is often scar tissue left behind after critical mistakes in the past. A company without some formal process is a company young enough to not have made disastrous mistakes yet. I've worked in a company of more than 50 that was basically without a trace of software development process. It was a clown circus. Nothing could be released on time reliably, quality was a joke, there was no documentation, no code review, and no plan or roadmap. Sure, they moved fast, but it didn't matter.

erikpukinskis 9y ago

I just learned about Hegel so now everything is a nail, but... You really can't make categorical statements like "formal process is the enemy". You put formal process into place, and that creates a dialectic where you're waiting for someone to come in and champion the antithesis: "formal process is the enemy". Then that works for a bit until it doesn't and someone synthesizes: "sometimes formal is better sometimes informal is better". Then that becomes the thesis. Then there's a new dialectic in place. Someone will come in and say "the character of our company is such that we're generally better with formal processes". Then someone synthesizes that, and suggests that all processes are on a trajectory from formal to informal and the company is good at expediting that. Then someone realizes we need a process for regular deformalization. This continues forever. None of these ideas are really that much more generically true than any other. So to embrace Hegel, the goal is just to question yourself as far as you reasonably can, so as to get the best view on the current situation and only the current situation. Make a choice and move on, knowing the next situation is likely to be different. Edit: I should add this is the process by which we hurry into The Future, which some people might point out means turning our backs on Everything That Is Holy and is therefore a tool of Satan, but I'll leave you to explore that with the rapidly growing universe of anti-Hegelian YouTubers.

freekh 9y ago
I really liked this comment :) I am looking forward to pondering upon this way of reasoning for this entire Saturday :)

jjtheblunt 9y ago

That's exactly why I left Apple engineering, and I've not regretted it, somewhat sadly, somewhat happily.

geofft 9y ago

> Emphasize that the most important part of software development is moving fast. This doesn't seem to me to be true. Can you expand on your thoughts here? I'd agree that the ability to move fast is an essential part of effective software development, but I don't think it's the most important (the most important is producing business value accurately and reliably), and I think that moving fast for the sake of moving fast is likely to result in moving quickly in the wrong direction. In particular, it's easy to be so worried about moving fast that you have no time to take new business needs/opportunities into account (see also Tom DeMarco's Slack), and it's easy to get excited about shiny new technology and spend a lot of time implementing it and roleplaying Google for no reason, which is a great way to move very quickly while not delivering a cent of business value. Finding somewhere with formal process, with full-time managers who aren't reluctantly moved to management for career advancement, with clear hierarchy and goals, was an important part of my most recent job search, simply because, as a good engineer, I hate to see my work go to waste. I have heard the same sentiment from lots of my talented friends.

quotemstr 9y ago
Unless you take conscious steps to remain flexible and to eliminate barriers to productivity, you're going to find your ability to "produc[e] business value" diminished because effort will get sucked into process instead, and for reasons nobody can identify. Soon enough, you'll find yourself wondering why you have so many developers, all of whom appear to be doing work, yet without getting anything done. By default, all software development organizations slow down. I say that developer velocity is the most important goal because, without focusing on this velocity, you end up solidifying, like a cooling lava, and it becomes difficult to accomplish any of your other goals.
- geofft 9y ago
  Well, okay, you and I are agreeing that velocity is nice to have, but why is it the most important goal? I can make the exact same argument about automated regression tests. As the size of your codebase grows and the number of customers (i.e., people triggering edge cases) grows, unless you take conscious steps to make sure that every build doesn't regress previous bugs, all your effort will get sucked into diagnosing and fixing things that you've fixed before. And all your developers will be fixing bugs but you won't be fixing new bugs and certainly not shipping features. In particular, if you want to refactor / rewrite parts of your code to improve development velocity, you'll suffer from the well-documented problem (see e.g. https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-... ) of losing all the knowledge of the weird edge cases you've fixed along the way. So, testing is strictly more important than development velocity! I can make the same argument about a half-dozen other development best practices, and I'll have actual citations to back them up. Why is development speed the most important goal, why is formal process the enemy of speed (instead of its friend, because it makes sure work doesn't get duplicated or wasted!), and what is your evidence for these claims? In particular, if your claim is true, we would not expect to see as much feature development / new products from old, large companies (I'm thinking of Oracle, Microsoft, Apple, Amazon, etc.) as we do. We would expect to see these companies grind to a halt and perform poorly in the markets (investors really want to see growth, not just continuing to be as good as you were last year), and that's not what's happening.

sanityUnbounded 9y ago

I recommend Creativity, Inc. to every person in a management position whom I have the opportunity to recommend it to. Although Et Catmull presents a depiction of Pixar and it's growth as a largely artistic endeavor, I feel that it is applicable to those that manage engineers especially. The ratio of students that decided to major in engineering because they were good at math in high school vs. the students who decided to major in engineering because they they wanted to make something inspiring, I feel ( and let me stress my inexperience as a current undergrad, without experience in the actual field ) is vastly favored in career growth towards those that decide to create instead of simply learn. While there is always room and lucrativity to tie engineers on a leash and juice mathematics and logic out of them until they reach an existential crisis, there is always (ALWAYS) room within the budget to let them set out upon a path which is in tune with their not necessarily young, but foolish and hopeful beliefs.

rleigh 9y ago

Used appropriately, a formal process provides structure to the workflows within an organisation. Everyone knows the process, and it provides a framework and expectations for everyone to work within. In organisations without any process, it can lead to difficulty in getting stuff done because there's no responsibility or accountability for who is doing what, and when. Who is reviewing my PR, which CI systems tested and deployed the prospective change, why was the work done, when does it get merged, and by whom? Etc. Basic daily routine. You also have the situation that people invent rules and requirements in the absence of a documented process, which can change arbitrarily and are impossible to work with because it depends upon the changing whims of whoever feels they are in charge. Clairvoyancy and divination should not be a job requirement! Process becomes problematic when the process grows and ossifies and becomes the end itself, rather than the means to an end. You end up with it being impossible to get work done because the process is so burdensome. When your bosses care more about the process being followed to the letter rather than the actual completion of the work which the process should be enabling, that's IMO the point where it needs a rethink. Unfortunately, organisations have a tendency to accrete the stuff; I've in the past argued for less to the amazement and incredulity of my bosses. It's also a factor that process is used as a blunt instrument to wield power over others, and I think that's certainly a driver as well. But often organisations favour accountability and rules over creativity and efficiency, even if you spend 95% of your time with process, and 5% actually doing productive stuff...