4 ms·

Experienced something similar, but it was not an outage. Had several knowledgeable people leave the company, and they all happened to be experts in a particular

by alimov 4y ago

Experienced something similar, but it was not an outage. Had several knowledgeable people leave the company, and they all happened to be experts in a particular service. Positions weren’t backfilled even though the people gave advance notice. About two months later we ended up getting out butts kicked when nobody knew the details of the service implementation and the service was expected to be updated to support some new features.. couldn’t get it updated for about 3-4 weeks because we couldn’t afford an outage.

marcosdumay 4y ago
> couldn’t get it updated for about 3-4 weeks That doesn't sound like a large issue (but the details can completely change it). Maybe it was the right decision.
alisonatwork 4y ago
Arguably those folks who left didn't do quite as good a job as they perhaps should have when they were still there. A high quality developer leaving a service behind should have already written sufficient documentation so that another high quality developer (especially one at the same company) can ramp up more quickly than 3-4 weeks. I think this is just another symptom of many tech companies throwing up their hands and pretending like "legacy" services are inescapable technical debt, when really they just never bothered to emphasize to their employees that services should be built in a maintainable way from the outset.
- mecha_ghidorah 4y ago
  This is true but only so long as they have been given time to do it. I've left a company, with notice, and they had me working on building stuff almost to my last day. Sure I did have a few "hand over" sessions in my schedule where I was expected to walk other devs through some stuff I was the owner for in a broad sense, but they never gave me time to produce solid docs for anything even knowing I was headed out the door
  - Perseids 4y ago
    It can get worse: me and my teammate trained a new dev for half a year with the explicit intention for him to take over should one of us leave. When the day came, said new dev was moved to another team, the product was given to an outsourcer and I was told to train them how to develop in two 2h sessions before I left. (The product was a legacy mess we were in the (multi year) process of refactoring into something maintainable. Documenting the intentions midway or even the status before was beyond the time budget we were allotted. All of this while doing feature development, without having proper requirements or user stories or access to customers who understand their use case, of course.)
    - mecha_ghidorah 4y ago
      That sounds rough. It's very painful not having time to do things properly, and it's also painful when you don't have proper requirements so you can't even be totally sure what properly even is. I semi-recently left a job (the one I was talking about originally) after more than once being told "we just want a toy, proof of concept version and any time you spend on things like performance is a waste of time and we will scold you for it, even if there isn't any other work for you to do" before they decided to ship my "toy proof of concept" to a client requiring an ungodly rework of it while it was running in a production environment. I honestly think people denigrate having at least some actual design work and requirements done upfront as being "too waterfall" and think doing it is a bad idea so we as an industry end up with not even the intentions of how a piece of code should work written down and no idea how stuff works once the original author moves on
  - alisonatwork 4y ago
    That's exactly my point. If companies aren't expecting their developers to document their services - while the developers are still employed and building those services in the first place - that's a failure in my books. It's exactly the sort of corporate culture that props up this mindset of "promotion-driven development".
    - mecha_ghidorah 4y ago
conradfr 4y ago
Kind of experienced that when interviewing for one of the well-known music steaming companies years ago. They had one critical service in Erlang and the developer who coded it had left years ago and nobody knew the language so they wanted to rewrite it. I was starting using Elixir back then and said I could maybe took that code over but they chose the other candidate ;)

by alimov 4y ago

marcosdumay 4y ago

> couldn’t get it updated for about 3-4 weeks That doesn't sound like a large issue (but the details can completely change it). Maybe it was the right decision.

alisonatwork 4y ago

Arguably those folks who left didn't do quite as good a job as they perhaps should have when they were still there. A high quality developer leaving a service behind should have already written sufficient documentation so that another high quality developer (especially one at the same company) can ramp up more quickly than 3-4 weeks. I think this is just another symptom of many tech companies throwing up their hands and pretending like "legacy" services are inescapable technical debt, when really they just never bothered to emphasize to their employees that services should be built in a maintainable way from the outset.

mecha_ghidorah 4y ago
This is true but only so long as they have been given time to do it. I've left a company, with notice, and they had me working on building stuff almost to my last day. Sure I did have a few "hand over" sessions in my schedule where I was expected to walk other devs through some stuff I was the owner for in a broad sense, but they never gave me time to produce solid docs for anything even knowing I was headed out the door
- Perseids 4y ago
  It can get worse: me and my teammate trained a new dev for half a year with the explicit intention for him to take over should one of us leave. When the day came, said new dev was moved to another team, the product was given to an outsourcer and I was told to train them how to develop in two 2h sessions before I left. (The product was a legacy mess we were in the (multi year) process of refactoring into something maintainable. Documenting the intentions midway or even the status before was beyond the time budget we were allotted. All of this while doing feature development, without having proper requirements or user stories or access to customers who understand their use case, of course.)
  - mecha_ghidorah 4y ago
    That sounds rough. It's very painful not having time to do things properly, and it's also painful when you don't have proper requirements so you can't even be totally sure what properly even is. I semi-recently left a job (the one I was talking about originally) after more than once being told "we just want a toy, proof of concept version and any time you spend on things like performance is a waste of time and we will scold you for it, even if there isn't any other work for you to do" before they decided to ship my "toy proof of concept" to a client requiring an ungodly rework of it while it was running in a production environment. I honestly think people denigrate having at least some actual design work and requirements done upfront as being "too waterfall" and think doing it is a bad idea so we as an industry end up with not even the intentions of how a piece of code should work written down and no idea how stuff works once the original author moves on
- alisonatwork 4y ago
  That's exactly my point. If companies aren't expecting their developers to document their services - while the developers are still employed and building those services in the first place - that's a failure in my books. It's exactly the sort of corporate culture that props up this mindset of "promotion-driven development".
  - mecha_ghidorah 4y ago

conradfr 4y ago

Kind of experienced that when interviewing for one of the well-known music steaming companies years ago. They had one critical service in Erlang and the developer who coded it had left years ago and nobody knew the language so they wanted to rewrite it. I was starting using Elixir back then and said I could maybe took that code over but they chose the other candidate ;)