3 ms·

This is one of those utopian engineer fallacies that is on the same level as reliable networking and fsync. People don't work like computers and don't respond

This is one of those utopian engineer fallacies that is on the same level as reliable networking and fsync. People don't work like computers and don't respond like API endpoints. There will always be backchannels, backburners, pidgeonholes, and all the other political games humans play happening even with an "API between them". The problem is always management, and the solution is never "more management", IMO.

dacryn 3y ago
an API does not mean there are no hidden dependencies, and that is often the biggest failure. If an API call is not stateless, or requires to be chained with other calls, you're in for a world of pain in the long run
- nostrebored 3y ago
  But this is equivalent to saying “a monolith can never work because it’s highly coupled”. In both cases you need to follow best practices to make things work. API design and alignment with consumers of the API is table stakes.
DanielHB 3y ago
The whole point of the article is that silos are not intrinsically bad, partially because silos reduces communication (and therefor managers) required Two teams agreeing on an API between themselves instead of one mega team fulfilling the needs of several client teams
- throw04323 3y ago
  > Two teams agreeing on an API between themselves I think that depends on the type of service that the team provides. If you have a central team that many other teams interact with, they risk becoming a bottleneck. They may not be interested in maintaining custom APIs for each team interaction and you will need to agree on a contract that all can live with. Another risk is that the team providing the service also have their own backlog, including work they want to do themselves and requests from other teams. This can cause unwanted dependencies and delays where managers try to fight to be prioritized on the expense of others.
  - packetlost 3y ago
    All I'm saying is it appears to have worked at mega-scale for Amazon.
    - acjohnson55 3y ago
      Most of us don't have mega-scale problems, though. A tremendous amount of waste has been created by applying FAANG tech and processes to completely different contexts.
      - packetlost 3y ago
        Sure, but scaling of an organization is not the same as scaling for traffic in a technical sense. There are so many companies that employ comparable numbers of engineers that are not big tech companies.
        acjohnson55 3y ago
        That makes sense to me. I think you're right that if you're a large enterprise, it may well make sense to adopt a very API-centric strategy for how teams interface, even if the scale doesn't demand it from a tech perspective.
    - throw04323 3y ago
      I'm not saying it can't work, but that there are risks involved. I have worked for several companies ranging from local startups to global enterprise (not FAANG). Each company tried the silo approach when they migrated to micro services and it caused significant delays and dependencies. They would have been better off if they focused more on larger domain services with fewer external dependencies. I am open to the idea that Amazon has been able to avoid these problems, but it's clearly not a silver bullet. In general I have to say I'm sceptical about comparisons with FAANG, because they live in a completely separate part of the technology sector. They have income similar to small countries and can live with inefficiencies that can break a startup.
- dkarl 3y ago
  Agreeing means the providing team understanding and meeting the needs of the consuming team. Teams that can work together to accomplish this wouldn't be called "silos." However, when two teams work together to create an API, or a process or some other self-service mechanism, sometimes the API so good that the teams no longer need to talk to each other. The practices and relationships that enabled communication fade away. Walls go up, silos form, but nobody notices anything wrong, because it seems like efficiency is getting better and better. Over time, though, people start to notice that projects that encounter a need to change the API always fail. The API has become legacy, baggage, a problem. There may still be somebody on the providing team who remembers that they used to get together in the same room with developers from the consuming team to come up with solutions together, and they'll naively suggest that as a solution, but there are now too many assumptions baked in at the management level for that to be allowed to happen. A change in the working relationship between teams means the managers will fight over what this means for different managers' prestige. Somebody's cheese is going to get moved. Managers gear up to go to war over things like that, so upper management dictates a solution that minimizes inter-manager violence, a solution that carefully circumscribes the kinds and amounts of contact that members of the two teams are allowed to have. Voila: silos with windows, and engineers sitting in their respective windows forlornly waving at each other like lovers separated by their parents.

dacryn 3y ago

an API does not mean there are no hidden dependencies, and that is often the biggest failure. If an API call is not stateless, or requires to be chained with other calls, you're in for a world of pain in the long run

nostrebored 3y ago
But this is equivalent to saying “a monolith can never work because it’s highly coupled”. In both cases you need to follow best practices to make things work. API design and alignment with consumers of the API is table stakes.

DanielHB 3y ago

The whole point of the article is that silos are not intrinsically bad, partially because silos reduces communication (and therefor managers) required Two teams agreeing on an API between themselves instead of one mega team fulfilling the needs of several client teams

throw04323 3y ago
> Two teams agreeing on an API between themselves I think that depends on the type of service that the team provides. If you have a central team that many other teams interact with, they risk becoming a bottleneck. They may not be interested in maintaining custom APIs for each team interaction and you will need to agree on a contract that all can live with. Another risk is that the team providing the service also have their own backlog, including work they want to do themselves and requests from other teams. This can cause unwanted dependencies and delays where managers try to fight to be prioritized on the expense of others.
- packetlost 3y ago
  All I'm saying is it appears to have worked at mega-scale for Amazon.
  - acjohnson55 3y ago
    Most of us don't have mega-scale problems, though. A tremendous amount of waste has been created by applying FAANG tech and processes to completely different contexts.
    - packetlost 3y ago
      Sure, but scaling of an organization is not the same as scaling for traffic in a technical sense. There are so many companies that employ comparable numbers of engineers that are not big tech companies.
      - acjohnson55 3y ago
        That makes sense to me. I think you're right that if you're a large enterprise, it may well make sense to adopt a very API-centric strategy for how teams interface, even if the scale doesn't demand it from a tech perspective.
  - throw04323 3y ago
    I'm not saying it can't work, but that there are risks involved. I have worked for several companies ranging from local startups to global enterprise (not FAANG). Each company tried the silo approach when they migrated to micro services and it caused significant delays and dependencies. They would have been better off if they focused more on larger domain services with fewer external dependencies. I am open to the idea that Amazon has been able to avoid these problems, but it's clearly not a silver bullet. In general I have to say I'm sceptical about comparisons with FAANG, because they live in a completely separate part of the technology sector. They have income similar to small countries and can live with inefficiencies that can break a startup.
dkarl 3y ago
Agreeing means the providing team understanding and meeting the needs of the consuming team. Teams that can work together to accomplish this wouldn't be called "silos." However, when two teams work together to create an API, or a process or some other self-service mechanism, sometimes the API so good that the teams no longer need to talk to each other. The practices and relationships that enabled communication fade away. Walls go up, silos form, but nobody notices anything wrong, because it seems like efficiency is getting better and better. Over time, though, people start to notice that projects that encounter a need to change the API always fail. The API has become legacy, baggage, a problem. There may still be somebody on the providing team who remembers that they used to get together in the same room with developers from the consuming team to come up with solutions together, and they'll naively suggest that as a solution, but there are now too many assumptions baked in at the management level for that to be allowed to happen. A change in the working relationship between teams means the managers will fight over what this means for different managers' prestige. Somebody's cheese is going to get moved. Managers gear up to go to war over things like that, so upper management dictates a solution that minimizes inter-manager violence, a solution that carefully circumscribes the kinds and amounts of contact that members of the two teams are allowed to have. Voila: silos with windows, and engineers sitting in their respective windows forlornly waving at each other like lovers separated by their parents.