3 ms·

> Letting/making devs own the services they deploy/develop is not the full picture, nor is it a real goal, but still it sums up the problem and solution space c

> Letting/making devs own the services they deploy/develop is not the full picture, nor is it a real goal, but still it sums up the problem and solution space concisely. What I'm trying to explain is, that I don't believe that this is a realistic expectation, because owning a service in production includes aspects of the stack that are not related to development, like networking, storage, and various non-dependency security-related concerns. When you task a platform team with building an easy-to-use abstraction for developers, those abstractions will inevitably leak. When they leak, one of two things happens: either the developers page the platform engineers, in which case you're back to shared ownership and the need to coordinate the concerns and responsibilities of multiple teams, or the developers are expected to fix the leak themselves, in which case, most of the time, the developers will find themselves out of their depth, most likely doing something inefficient at best or flat out wrong at worst.

pas 5y ago
That's why ideally you have someone in your team who is trusted by the platform team to do what needs to be done to unblock your main goals/tasks. The platform team of course can help (or hinder), but the ability to do your own stack helps uncouple problems, helps teams progress. Ideally there's an equilibrium. Some teams will inevitably be outliers, but the majority will benefit from the platform. There's always a degree of sharing/coordination/communication. After all the teams are part of the same company. (Eg. when it comes to data access, API stability, it's a must to coordinate. Though famously Amazon "simply" solved this by a companywide mandate, requiring everything to have an API, and there's probably some story about their stability too.) So, in any case the coupling should be loose. Basically an almost wait-free (but at least lock-free) algorithm.
throwdbaaway 5y ago
I agree with the concern regarding leaky abstractions, but ultimately the developers should still own their services and be on-call for it, e.g. - On-call dev deploys a buggy change that messes up some transactional data. Product owner passes the complaints received by customer service. On-call dev rolls back the deployment, writes a script to fix the bad data, and then prepares a post-mortem. - On-call dev and on-call dba are paged because the database for the service is slow. On-call dev checks the service metrics and logs, and confirms that this is not caused by a deployment nor abnormal traffic. On-call dba checks the database metrics and logs, and finds some runaway queries due to silly optimizer. On-call dba kills the runaway queries, and proposes to add an index hint. On-call dev agrees, works on it, and asks on-call dba to review the change. - On-call sec is notified because of a new rce vulnerability. On-call sec comes up with the steps to identify and remedy the issue. All on-call devs are notified to follow the steps, to ensure that their services are not impacted.

pas 5y ago

That's why ideally you have someone in your team who is trusted by the platform team to do what needs to be done to unblock your main goals/tasks. The platform team of course can help (or hinder), but the ability to do your own stack helps uncouple problems, helps teams progress. Ideally there's an equilibrium. Some teams will inevitably be outliers, but the majority will benefit from the platform. There's always a degree of sharing/coordination/communication. After all the teams are part of the same company. (Eg. when it comes to data access, API stability, it's a must to coordinate. Though famously Amazon "simply" solved this by a companywide mandate, requiring everything to have an API, and there's probably some story about their stability too.) So, in any case the coupling should be loose. Basically an almost wait-free (but at least lock-free) algorithm.

throwdbaaway 5y ago

I agree with the concern regarding leaky abstractions, but ultimately the developers should still own their services and be on-call for it, e.g. - On-call dev deploys a buggy change that messes up some transactional data. Product owner passes the complaints received by customer service. On-call dev rolls back the deployment, writes a script to fix the bad data, and then prepares a post-mortem. - On-call dev and on-call dba are paged because the database for the service is slow. On-call dev checks the service metrics and logs, and confirms that this is not caused by a deployment nor abnormal traffic. On-call dba checks the database metrics and logs, and finds some runaway queries due to silly optimizer. On-call dba kills the runaway queries, and proposes to add an index hint. On-call dev agrees, works on it, and asks on-call dba to review the change. - On-call sec is notified because of a new rce vulnerability. On-call sec comes up with the steps to identify and remedy the issue. All on-call devs are notified to follow the steps, to ensure that their services are not impacted.