3 ms·

> 'But only I can fix that complex race condition, that popped up half a year after development' - well if it was good enough with the bug for people not to not

> 'But only I can fix that complex race condition, that popped up half a year after development' - well if it was good enough with the bug for people not to notice it for half a year, it's going to be good enough for another half, until the new devs can fix it. From my time at AWS, and in light of the recent DynamoDB dns race condition bug, this is a line of thinking that is problematic. There is so much complexity in the inner workings of any one of those services. Many of those services are now composed of many teams, each having turned over completely multiples of times. Yes there are runbooks and knowledge passed down from each generation, to some extent. But when you hit some critical outage, edge case, etc., those people running the systems have such a different relationship to the system having simply maintained it vs. built it.

nradov 11mo ago
From a certain perspective, that line of thinking is only "problematic" if customers stop paying. In other words, the product only has to maintain a certain minimum quality threshold and doing any more is a waste of money. I don't necessarily agree with that myself but it is a common attitude.
- captainkrtek 11mo ago
  For sure, as long as the product keeps selling and it stays up, great, the business is still viable. I think its more analogous to a car with some problems that may be going unchecked; it still drives and gets you from A->B, until it doesn’t.
kamaal 11mo ago
>>But when you hit some critical outage, edge case, etc., those people running the systems have such a different relationship to the system having simply maintained it vs. built it. Long stays in most companies are practically non-existent these days, apart from a few folks who are present for long, unfortunately these people are often called lifers, coasters etc. Simply put, no one person can say they contributed most to the building of a system. Barring very few rare projects. So you already need the runbook, documentation model to run things, for the reason people themselves work along those lines.

nradov 11mo ago

From a certain perspective, that line of thinking is only "problematic" if customers stop paying. In other words, the product only has to maintain a certain minimum quality threshold and doing any more is a waste of money. I don't necessarily agree with that myself but it is a common attitude.

captainkrtek 11mo ago
For sure, as long as the product keeps selling and it stays up, great, the business is still viable. I think its more analogous to a car with some problems that may be going unchecked; it still drives and gets you from A->B, until it doesn’t.

kamaal 11mo ago

>>But when you hit some critical outage, edge case, etc., those people running the systems have such a different relationship to the system having simply maintained it vs. built it. Long stays in most companies are practically non-existent these days, apart from a few folks who are present for long, unfortunately these people are often called lifers, coasters etc. Simply put, no one person can say they contributed most to the building of a system. Barring very few rare projects. So you already need the runbook, documentation model to run things, for the reason people themselves work along those lines.