8 ms·

It seems like a lot of people here have been in an ineffective large organisation and an effective startup. I've worked at an effective large organisation and a

by ajb 5y ago

It seems like a lot of people here have been in an ineffective large organisation and an effective startup. I've worked at an effective large organisation and an ineffective startup, so perhaps I can shed some light on what these kind of processes are for: * You don't get to be a large organisation without accumulating generations of previous products. Well, unless you're google and can regularly fire your customers without going bust. but part from them, you start to need processes just to track what's going on. (The ineffective startup accumulated previous product attempts too, and is still paying for a lot of pointless infra because no-one still there knows which ones can be switched off). * At a large scale, it starts to be pretty difficult to maneuver if each dept of 30-50 people has built or contracted all their own infra & services. Some divergence is useful for agility, but a lot of it is just waste and actually slows you down when you want two depts to work together on something - or even prevents useful collaboration. An effective organisation will make the tradeoff consciously. * When you complete dozens of projects per year, it starts to be possible to invest in researching and rolling out best practices in areas where a startup just has to go with the simple/obvious answer. When my startup employer moved, it was pretty onerous, and there was a bunch of stuff we never found again. The large org had so many offices that it had a full time dept just for moving offices. (They weren't dumb, each move consolidated offices -but they kept buying other companies). When they moved us, it was like magic - we went home on a friday and went to work on a monday in the new office, and everything was set up. Well, we had to set up the lab ourselves, but

___luigi 5y ago
This is GREAT summary! I work in a large organization, in an industry that is naturally slow. The work (in the industry) is slow because it's critical to deliver stable and reliable solution because large part of population and economy rely on this industry SW/HW. I have heard a lot of complains on how the process is slow, but in many cases, building on top of existing work takes a long time. It takes long time to understand legacy work, to go through older studies and projects, and to find spots to add contributions. Not all of us work on green field projects, but many of us maintain code/work that doesn't look "cool".
PragmaticPulp 5y ago
> It seems like a lot of people here have been in an ineffective large organisation and an effective startup. I was at an effective startup that turned into an ineffective large organization before my own eyes. Your three (excellent) points ring very true. However, the company actually tried to implement each of your three points and recognized their importance. The problem was in the execution. Old products: These were neither retired nor given appropriate resources, so they instead decayed over time. Empty promises about longevity were made and then broken. Attempts were made to outsource maintenance to countries with the cheapest engineers, which backfired when each update become more buggy than the last. The end result was a lot of customers and early adopters getting burned and a destruction of our reputation in the industry. Shared infrastructure: The C-levels saw shared infrastructure as an opportunity to reduce costs and speed up development, but they took it too far. At the worst, they wanted to have one gigantic team of back-end engineers, one gigantic team of front-end engineers, and so on, spread across every project across the entire company. Individual engineers were assigned to many projects and each had to deal with multiple managers fighting for their time. End result was that nothing got done and politics reigned supreme as the way to get attention on your project. Best practices: The company hired “subject matter experts” to enforce best practices. They were expected to be involved in everything across the company. Imagine being hired as the “security best practices owner” and then told that you’re accountable for the security of everything at the whole company. Instead of promoting best practices, they went into full defensive mode and interfered with the shipment of every product and feature that they didn’t have time to work on. We would have products ready to ship and have the best practices experts panicking to halt it because they wouldn’t have time to look at it for months after completion. Getting them involved at the start was a pipe dream. The overarching issue was that they tried to do big company practices in the move fast and break things startup style. For every well-intentioned move toward being a proper company they managed to screw it up by trying to inject startup-style speed and improvisation.
- ido 5y ago
  The overarching issue was that they tried to do big company practices in the move fast and break things startup style. For every well-intentioned move toward being a proper company they managed to screw it up by trying to inject startup-style speed and improvisation. What would have been a better way of handling the transition?
costcofries 5y ago
This is a great summary! Large orgs are designed to move slow, it's a necessity of calculated moves. For example, single policy, product or process changes can affect thousands of people and many millions of dollars. These things take time as the ripple effects are far and wide. This is not to say that innovation isn't possible. In my experience, the way to forge ahead in large orgs is to achieve progress by example, advocacy and influence.
ehnto 5y ago
From what I hear, even Google is mired by years of product, organisational and tooling debt.
- datavirtue 5y ago
  Mire is a good word to describe what is going through my mind when I see their recent search results.
- schnable 5y ago
  That would make sense as to why they kill of products, in attempt to reign this in. In reality, killing products is still pretty uncommon even for Google. Just today, I logged into Google Calendar on my iPhone and the mobile UI looked like it hadn't changed since 2008 and it was missing a ton of features.
  - jeffbee 5y ago
    The mobile web UI for calendar has indeed been abandonware for about 5 years and even before that it was starting to look pretty crusty. I think largely because nobody cares and everyone on mobile uses the app. The mobile web interface of Gmail hasn't changed in years either but it is still maintained and also wonderfully good.
- vkou 5y ago
  Yes, and it actively manages it. Occasionally, one of the potential outcomes of this management becomes visible to the outside world, when it sunsets a product. Other times, lack of investment in this management also becomes visible, when a product is clearly languishing. As in any other company, headcount is always limited, and some products are funded by nothing but the willpower of particular passionate engineers/managers, which results in a very low lottery bus factor.
tazjin 5y ago
> Well, unless you're google and can regularly fire your customers without going bust This is only true for external products. When I was at Google it always seemed to me that the majority of engineering effort is focused on the inside, and that the same effect exists there.
toomuchtodo 5y ago
What would you say are the drivers that enable effectiveness in an org, regardless of its size?
- ajb 5y ago
  Just off the top of my head: An effective responsibility culture and structure. Those who are responsible for something must have the means to effect it. There must be a means of agreeing who is responsible for something and an effective way to find who is responsible when someone is blocked. Things should not languish because no-one is responsible, nor be trampled in because everyone wants a piece. (This is culture as well as structure, because everyone needs to internalise what a good responsibility structure looks like so that they negotiate it into place in a distributed way, rather than having it imposed top down.) Prioritise making intelligent decisions over raw speed. In my effective employer, if your project was late, you were required to take a step back and really work out when it was realistically deliverable and make sure you genuinely understood what was now needed - better to say ONCE 'it needs another 3 months' than 'just a another 2 weeks' 8 times. (The ineffective company had a 5 day horizon and would burn everyone out constantly trying to deliver next week) Related to this is that profligate agility can work against intelligence. The effective org could afford what would seem like a huge amount of process, but it actually took less of our time than the agile process in the ineffective org because we didn't need to change our minds as often. Making effective use of staff's skills. The means everything from: Leaders need to recognise what they aren't expert in, not having a destructive people culture, giving people larger pieces of work that they can master rather than splitting everything into a million fragments, allowing for development rather than people getting type-cast.
  - andrekandre 5y ago
    > profligate agility can work against intelligence thank you for putting into words something i've been inuiting for a while now... i couldn't put a finger on what was going at some workplaces i've been in, but this really resonates with me it seems in our chase for "agility" above all else, we are sacrificing our intelligence along the way...
- 5y ago
zwieback 5y ago
My experience as well, although my large org sometimes seems to move to slowly it's loads better than the startups I've been at. One thing I'm realizing is really useful, and which we're doing more and more, is putting "program managers" in charge of projects, totally separate from "project managers", which really should be called "people managers". With that split we can have a senior engineer who is interested in management work across multiple teams without worrying about the day to day personnel issues. This only works if everyone supports this approach but here it's working well.
- serverholic 5y ago
  This sounds a lot like the matrix team structure. Can program managers assign work to engineers? And if so, how do you deal with project and program managers giving conflicting tasks?
  - zwieback 5y ago
    Yes, the program managers can assign work and they tend to work closely with the involved people managers to balance workload.
    - lugged 5y ago
      I hate matrix management so much. It's gimmicky and a pain in the arse. I know management does do a job, but it's hard to justify needing two instead of one. My view I that matrix style management is a solution to a communication problem you haven't figured out. Instead of fixing the communication throughput problem, your adding more bandwidth and scaling out the number of nodes in hopes it magically fixes the problem. And it sort of does fix the communication problems, that's why people like it. The trade off is that as an IC its arese to have to jerk two cocks at the same time.
- robertlagrant 5y ago
  This sounds like terminology issues. Classically a project manager manages a project (a think with a start and an end, reasonably well defined) and a programme manager manages a set of related projects. People managers depending on intention sound like either engineering team leads, engineering heads of, or line managers.
hintymad 5y ago
Another reason that I observed is that large organizations are extremely afraid of mistakes regardless of expected loss. Such companies appoint gatekeepers, and those gatekeepers are by nature risk averse. The gatekeepers will simply reject any idea by default, unless there is a well-proven precedent. As a result, anything moves slowly.
- megablast 5y ago
  > Another reason that I observed is that large organizations are extremely afraid of mistakes regardless of expected loss. Not what I have seen. Big companies with big profits will throw millions away on dumb ideas on the chance they make money, when they do not.
  - hintymad 5y ago
    Also true. In the meantime, wanting to use Guava 33 instead of Guava 27 will get rejected immediately. Using Spark instead of AWS Batch is an absolute no no. I guess what qualifies as risk is also highly asymmetrical.
hammock 5y ago
Can you give an example (non-code related, if you have one) of the first bullet?
- ajb 5y ago
  Documentation. When you're working on a new build, everything is frequently just in your head or those of your teammates, and that isn't a problem initially. And if you have one main product, you don't need to spend a lot of time on documenting which resources were for what. When you have a bunch of old ones, which you may not actually often spend much time on, it's much more important to be able to figure out why that old thing is running, exactly what was shipped to customers when (in the case of physical product). Otherwise you risk having to do archaeology every time you turn round. Or just not having the information when some customers says that X has stopped working on a device that you shipped 5 years ago but guaranteed for 10.
ashtonkem 5y ago
> Some divergence is useful for agility, but a lot of it is just waste and actually slows you down when you want two depts to work together on something - or even prevents useful collaboration. An effective organisation will make the tradeoff consciously. Often an effective strategy for this to purposefully diverge and converge. Allow a new product to strike it out on its own, and if it survives follow back up and homogenize the infra so that maintenance is cheaper. Takes a lot of labor, but that’s usually something large organizations have in spades.
mcguire 5y ago
"At a large scale, it starts to be pretty difficult to maneuver if each dept of 30-50 people has built or contracted all their own infra & services." This is one of the biggest risks I've seen, for both large organizations and small ones. Large organizations are built of small organizations, and a bunch of effective small organizations will each be moving and shaking, and building or buying things that become part of their infrastructure. Then one day you wake up and discover that your overall organization is paying for 27 different solutions to the same problem. Then, any attempt to collapse that down to something reasonable results in zero productivity for quite a while and whole lot of pissed off people.
- konschubert 5y ago
  The question is, is it really worth it to collapse it down?
  - potatolicious 5y ago
    At some level, absolutely. Your organizations aren't usually totally independent units - they need to interoperate with other organizations. To the extent that the spaghetti of infrastructure prevents effective interop, that's a problem. And as the need for interop grows between two orgs the cost of the various hacks, shims, and other cheap adaptations, escalates, until unification of infrastructure starts looking awwwwfully tempting. Of course, unification comes with its own set of problems.
  - ryathal 5y ago
    It depends, but generally yes. Collapsing to 1 may or may not be a good thing, but you probably want less than 3-5 different DBMS, CMS, Cloud providers, printing services, source control systems, office supply/equipment, etc.
    - mcguire 5y ago
      And suddenly I'm reminded of the web app I put together that needed three DB drivers...
  - 5y ago
coding123 5y ago
> google and can regularly fire your customers They don't fire the customers (advertisers) they fire the users.
pbreit 5y ago
I will be curious to see how founder-led companies operate as they get large. Mainly companies like AirBnB, Cloudflare & Coinbase. I don't think it's impossible for a company to get large and still be somewhat effective.

by ajb 5y ago

___luigi 5y ago

This is GREAT summary! I work in a large organization, in an industry that is naturally slow. The work (in the industry) is slow because it's critical to deliver stable and reliable solution because large part of population and economy rely on this industry SW/HW. I have heard a lot of complains on how the process is slow, but in many cases, building on top of existing work takes a long time. It takes long time to understand legacy work, to go through older studies and projects, and to find spots to add contributions. Not all of us work on green field projects, but many of us maintain code/work that doesn't look "cool".

PragmaticPulp 5y ago

> It seems like a lot of people here have been in an ineffective large organisation and an effective startup. I was at an effective startup that turned into an ineffective large organization before my own eyes. Your three (excellent) points ring very true. However, the company actually tried to implement each of your three points and recognized their importance. The problem was in the execution. Old products: These were neither retired nor given appropriate resources, so they instead decayed over time. Empty promises about longevity were made and then broken. Attempts were made to outsource maintenance to countries with the cheapest engineers, which backfired when each update become more buggy than the last. The end result was a lot of customers and early adopters getting burned and a destruction of our reputation in the industry. Shared infrastructure: The C-levels saw shared infrastructure as an opportunity to reduce costs and speed up development, but they took it too far. At the worst, they wanted to have one gigantic team of back-end engineers, one gigantic team of front-end engineers, and so on, spread across every project across the entire company. Individual engineers were assigned to many projects and each had to deal with multiple managers fighting for their time. End result was that nothing got done and politics reigned supreme as the way to get attention on your project. Best practices: The company hired “subject matter experts” to enforce best practices. They were expected to be involved in everything across the company. Imagine being hired as the “security best practices owner” and then told that you’re accountable for the security of everything at the whole company. Instead of promoting best practices, they went into full defensive mode and interfered with the shipment of every product and feature that they didn’t have time to work on. We would have products ready to ship and have the best practices experts panicking to halt it because they wouldn’t have time to look at it for months after completion. Getting them involved at the start was a pipe dream. The overarching issue was that they tried to do big company practices in the move fast and break things startup style. For every well-intentioned move toward being a proper company they managed to screw it up by trying to inject startup-style speed and improvisation.

ido 5y ago
The overarching issue was that they tried to do big company practices in the move fast and break things startup style. For every well-intentioned move toward being a proper company they managed to screw it up by trying to inject startup-style speed and improvisation. What would have been a better way of handling the transition?

costcofries 5y ago

This is a great summary! Large orgs are designed to move slow, it's a necessity of calculated moves. For example, single policy, product or process changes can affect thousands of people and many millions of dollars. These things take time as the ripple effects are far and wide. This is not to say that innovation isn't possible. In my experience, the way to forge ahead in large orgs is to achieve progress by example, advocacy and influence.

ehnto 5y ago

From what I hear, even Google is mired by years of product, organisational and tooling debt.

datavirtue 5y ago
Mire is a good word to describe what is going through my mind when I see their recent search results.
schnable 5y ago
That would make sense as to why they kill of products, in attempt to reign this in. In reality, killing products is still pretty uncommon even for Google. Just today, I logged into Google Calendar on my iPhone and the mobile UI looked like it hadn't changed since 2008 and it was missing a ton of features.
- jeffbee 5y ago
  The mobile web UI for calendar has indeed been abandonware for about 5 years and even before that it was starting to look pretty crusty. I think largely because nobody cares and everyone on mobile uses the app. The mobile web interface of Gmail hasn't changed in years either but it is still maintained and also wonderfully good.
vkou 5y ago
Yes, and it actively manages it. Occasionally, one of the potential outcomes of this management becomes visible to the outside world, when it sunsets a product. Other times, lack of investment in this management also becomes visible, when a product is clearly languishing. As in any other company, headcount is always limited, and some products are funded by nothing but the willpower of particular passionate engineers/managers, which results in a very low lottery bus factor.

tazjin 5y ago

> Well, unless you're google and can regularly fire your customers without going bust This is only true for external products. When I was at Google it always seemed to me that the majority of engineering effort is focused on the inside, and that the same effect exists there.

toomuchtodo 5y ago

What would you say are the drivers that enable effectiveness in an org, regardless of its size?

ajb 5y ago
Just off the top of my head: An effective responsibility culture and structure. Those who are responsible for something must have the means to effect it. There must be a means of agreeing who is responsible for something and an effective way to find who is responsible when someone is blocked. Things should not languish because no-one is responsible, nor be trampled in because everyone wants a piece. (This is culture as well as structure, because everyone needs to internalise what a good responsibility structure looks like so that they negotiate it into place in a distributed way, rather than having it imposed top down.) Prioritise making intelligent decisions over raw speed. In my effective employer, if your project was late, you were required to take a step back and really work out when it was realistically deliverable and make sure you genuinely understood what was now needed - better to say ONCE 'it needs another 3 months' than 'just a another 2 weeks' 8 times. (The ineffective company had a 5 day horizon and would burn everyone out constantly trying to deliver next week) Related to this is that profligate agility can work against intelligence. The effective org could afford what would seem like a huge amount of process, but it actually took less of our time than the agile process in the ineffective org because we didn't need to change our minds as often. Making effective use of staff's skills. The means everything from: Leaders need to recognise what they aren't expert in, not having a destructive people culture, giving people larger pieces of work that they can master rather than splitting everything into a million fragments, allowing for development rather than people getting type-cast.
- andrekandre 5y ago
  > profligate agility can work against intelligence thank you for putting into words something i've been inuiting for a while now... i couldn't put a finger on what was going at some workplaces i've been in, but this really resonates with me it seems in our chase for "agility" above all else, we are sacrificing our intelligence along the way...
5y ago

zwieback 5y ago

My experience as well, although my large org sometimes seems to move to slowly it's loads better than the startups I've been at. One thing I'm realizing is really useful, and which we're doing more and more, is putting "program managers" in charge of projects, totally separate from "project managers", which really should be called "people managers". With that split we can have a senior engineer who is interested in management work across multiple teams without worrying about the day to day personnel issues. This only works if everyone supports this approach but here it's working well.

serverholic 5y ago
This sounds a lot like the matrix team structure. Can program managers assign work to engineers? And if so, how do you deal with project and program managers giving conflicting tasks?
- zwieback 5y ago
  Yes, the program managers can assign work and they tend to work closely with the involved people managers to balance workload.
  - lugged 5y ago
    I hate matrix management so much. It's gimmicky and a pain in the arse. I know management does do a job, but it's hard to justify needing two instead of one. My view I that matrix style management is a solution to a communication problem you haven't figured out. Instead of fixing the communication throughput problem, your adding more bandwidth and scaling out the number of nodes in hopes it magically fixes the problem. And it sort of does fix the communication problems, that's why people like it. The trade off is that as an IC its arese to have to jerk two cocks at the same time.
robertlagrant 5y ago
This sounds like terminology issues. Classically a project manager manages a project (a think with a start and an end, reasonably well defined) and a programme manager manages a set of related projects. People managers depending on intention sound like either engineering team leads, engineering heads of, or line managers.

hintymad 5y ago

Another reason that I observed is that large organizations are extremely afraid of mistakes regardless of expected loss. Such companies appoint gatekeepers, and those gatekeepers are by nature risk averse. The gatekeepers will simply reject any idea by default, unless there is a well-proven precedent. As a result, anything moves slowly.

megablast 5y ago
> Another reason that I observed is that large organizations are extremely afraid of mistakes regardless of expected loss. Not what I have seen. Big companies with big profits will throw millions away on dumb ideas on the chance they make money, when they do not.
- hintymad 5y ago
  Also true. In the meantime, wanting to use Guava 33 instead of Guava 27 will get rejected immediately. Using Spark instead of AWS Batch is an absolute no no. I guess what qualifies as risk is also highly asymmetrical.

hammock 5y ago

Can you give an example (non-code related, if you have one) of the first bullet?

ajb 5y ago
Documentation. When you're working on a new build, everything is frequently just in your head or those of your teammates, and that isn't a problem initially. And if you have one main product, you don't need to spend a lot of time on documenting which resources were for what. When you have a bunch of old ones, which you may not actually often spend much time on, it's much more important to be able to figure out why that old thing is running, exactly what was shipped to customers when (in the case of physical product). Otherwise you risk having to do archaeology every time you turn round. Or just not having the information when some customers says that X has stopped working on a device that you shipped 5 years ago but guaranteed for 10.

ashtonkem 5y ago

> Some divergence is useful for agility, but a lot of it is just waste and actually slows you down when you want two depts to work together on something - or even prevents useful collaboration. An effective organisation will make the tradeoff consciously. Often an effective strategy for this to purposefully diverge and converge. Allow a new product to strike it out on its own, and if it survives follow back up and homogenize the infra so that maintenance is cheaper. Takes a lot of labor, but that’s usually something large organizations have in spades.

mcguire 5y ago

"At a large scale, it starts to be pretty difficult to maneuver if each dept of 30-50 people has built or contracted all their own infra & services." This is one of the biggest risks I've seen, for both large organizations and small ones. Large organizations are built of small organizations, and a bunch of effective small organizations will each be moving and shaking, and building or buying things that become part of their infrastructure. Then one day you wake up and discover that your overall organization is paying for 27 different solutions to the same problem. Then, any attempt to collapse that down to something reasonable results in zero productivity for quite a while and whole lot of pissed off people.

konschubert 5y ago
The question is, is it really worth it to collapse it down?
- potatolicious 5y ago
  At some level, absolutely. Your organizations aren't usually totally independent units - they need to interoperate with other organizations. To the extent that the spaghetti of infrastructure prevents effective interop, that's a problem. And as the need for interop grows between two orgs the cost of the various hacks, shims, and other cheap adaptations, escalates, until unification of infrastructure starts looking awwwwfully tempting. Of course, unification comes with its own set of problems.
- ryathal 5y ago
  It depends, but generally yes. Collapsing to 1 may or may not be a good thing, but you probably want less than 3-5 different DBMS, CMS, Cloud providers, printing services, source control systems, office supply/equipment, etc.
  - mcguire 5y ago
    And suddenly I'm reminded of the web app I put together that needed three DB drivers...
- 5y ago

coding123 5y ago

> google and can regularly fire your customers They don't fire the customers (advertisers) they fire the users.

pbreit 5y ago

I will be curious to see how founder-led companies operate as they get large. Mainly companies like AirBnB, Cloudflare & Coinbase. I don't think it's impossible for a company to get large and still be somewhat effective.