5 ms·

Where have you worked where there were enough resources to effectively document all of your backend code? My job is currently mostly detective work, and it’s n

Where have you worked where there were enough resources to effectively document all of your backend code? My job is currently mostly detective work, and it’s neither a function of incompetence nor laziness, it’s just a function of needing to prioritize business objectives over sacrifice-able things like documentation.

cranekam 4y ago
Documentation also just doesn't cover the kind of issue the GitLab post is talking about. Sure, good docs on what talks to what and so on will help someone learn the lay of the land but it's not going to pinpoint the cause of a spike of 502 errors. Working that kind of thing out requires looking at the problem with a fresh set of eyes to see what's really happening, not what should be happening according to the docs. Some of the best troubleshooters I know figure things out by just looking at what's going on. What's this server talking to? Why is this backend slower? What else is happening on this host? Which network links? etc. None of this is in a manual.
- hericium 4y ago
  > Documentation also just doesn't cover the kind of issue the GitLab post is talking about. Sure, good docs on what talks to what and so on will help someone learn the lay of the land but it's not going to pinpoint the cause of a spike of 502 errors. And that's perfectly fine. This post was published so GL folks can say "we figured out something tricky/interesting and our scale is what made it more tricky". I love figuring out "Frankenstein" type of bugs, errors and problems myself. What I dislike figuring out is something that shouldn't require any figuring out at all. I read GitLab or Cloudflare's posts with pleasure. These doesn't seem to be written solely as technical recruitment "we do cool stuff" posts.
hericium 4y ago
Touching production stuff without prior contact with it seems risky. I was "forced" to do so a couple of times but always considered companies putting anyone ill-equipped on prod as... doing maybe too much savings at the cost of service quality. I left every single company which wanted me to spend a lot of time (weeks, months) to figure out their custom stuff due to no documentation and lack of people who worked on this and knew what makes the thing tick or what to do when it stops. Having noone or a single person kind-of knowing how to do something in a company is a managerial knee shot. I value my learning/working time maybe more than an average person due to focusing issues. Spending it on figuring something which could have been documented but wasn't absolutely isn't my favorite thing to do. Weeks of detective work are weeks when I'm deprived of ability to learn something useful for my next job. So while this GitLab post doesn't specify what kind of things are there to figure out, I consider stating that there's always something to figure out as unprofessional as it points out possibly unprepared staff.
- roflc0ptic 4y ago
  Thanks for the detailed response! My own sense here is that detective work is a deep part of being a software developer, because fundamentally we can’t escape the particulars of the problems we work on, but I guess that really depends on the complexity of the systems we’re working on.
- Brian_K_White 4y ago
  This is unrealistic. The people writing stuff always vastly outnumber you, whether you are a company, a developer, or the on-call fireman. Figuring out things you neither wrote nor fully studied beforehand, without breaking everything, is not some waste of valuable time, that IS the valuable expertise itself. If you don't like doing that then that just means you're not a good fit for that role, not that there's some failing somewhere else that it's even needed. It's not possible to document everything, and even if it were, no one would ever be qualified to do the job by this standard, because 3 things changed in production just while the on-call was making coffee. This story was utterly normal and the only bad thing in it is just generally how tall and shaky and complex "normal" has become by now. But that's not anything any single company can do much about. Large web services need load balancing and orchestration and a lot of different moving parts, and every one of those parts are changing every minute because somewhere in the world a productive developer just committed a new line of code in something you use.
  - morelisp 4y ago
    > But that's not anything any single company can do much about. Well, one company can certainly make it worse, by e.g. ignoring years of API ergonomics around process spawning and give you a vaguely-specified "command" string, then pumping VC money into their marketing budget to make themselves an "indispensable" daily tool. (Mostly I agree with you - just want to point out that in addition to the necessary complexity, the tooling around it also produces a lot of unnecessarily wrong-by-default behavior.)
    - Brian_K_White 4y ago
      " the tooling around it also produces a lot of unnecessarily wrong-by-default behavior." It does. What I meant by one company doing something about it was, in realistic terms, no one can do much about the fact that the rest of the world uses a lot of complex systems that aren't as robust as earlier simpler systems. You have to use the current tools and that's just how they are now. You can try to be less stupid as possible, but you can't change the ecosystem you need to live in and interoperate with.
deleted 4y ago
[deleted]

cranekam 4y ago

Documentation also just doesn't cover the kind of issue the GitLab post is talking about. Sure, good docs on what talks to what and so on will help someone learn the lay of the land but it's not going to pinpoint the cause of a spike of 502 errors. Working that kind of thing out requires looking at the problem with a fresh set of eyes to see what's really happening, not what should be happening according to the docs. Some of the best troubleshooters I know figure things out by just looking at what's going on. What's this server talking to? Why is this backend slower? What else is happening on this host? Which network links? etc. None of this is in a manual.

hericium 4y ago
> Documentation also just doesn't cover the kind of issue the GitLab post is talking about. Sure, good docs on what talks to what and so on will help someone learn the lay of the land but it's not going to pinpoint the cause of a spike of 502 errors. And that's perfectly fine. This post was published so GL folks can say "we figured out something tricky/interesting and our scale is what made it more tricky". I love figuring out "Frankenstein" type of bugs, errors and problems myself. What I dislike figuring out is something that shouldn't require any figuring out at all. I read GitLab or Cloudflare's posts with pleasure. These doesn't seem to be written solely as technical recruitment "we do cool stuff" posts.

hericium 4y ago

Touching production stuff without prior contact with it seems risky. I was "forced" to do so a couple of times but always considered companies putting anyone ill-equipped on prod as... doing maybe too much savings at the cost of service quality. I left every single company which wanted me to spend a lot of time (weeks, months) to figure out their custom stuff due to no documentation and lack of people who worked on this and knew what makes the thing tick or what to do when it stops. Having noone or a single person kind-of knowing how to do something in a company is a managerial knee shot. I value my learning/working time maybe more than an average person due to focusing issues. Spending it on figuring something which could have been documented but wasn't absolutely isn't my favorite thing to do. Weeks of detective work are weeks when I'm deprived of ability to learn something useful for my next job. So while this GitLab post doesn't specify what kind of things are there to figure out, I consider stating that there's always something to figure out as unprofessional as it points out possibly unprepared staff.

roflc0ptic 4y ago
Thanks for the detailed response! My own sense here is that detective work is a deep part of being a software developer, because fundamentally we can’t escape the particulars of the problems we work on, but I guess that really depends on the complexity of the systems we’re working on.
Brian_K_White 4y ago
This is unrealistic. The people writing stuff always vastly outnumber you, whether you are a company, a developer, or the on-call fireman. Figuring out things you neither wrote nor fully studied beforehand, without breaking everything, is not some waste of valuable time, that IS the valuable expertise itself. If you don't like doing that then that just means you're not a good fit for that role, not that there's some failing somewhere else that it's even needed. It's not possible to document everything, and even if it were, no one would ever be qualified to do the job by this standard, because 3 things changed in production just while the on-call was making coffee. This story was utterly normal and the only bad thing in it is just generally how tall and shaky and complex "normal" has become by now. But that's not anything any single company can do much about. Large web services need load balancing and orchestration and a lot of different moving parts, and every one of those parts are changing every minute because somewhere in the world a productive developer just committed a new line of code in something you use.
- morelisp 4y ago
  > But that's not anything any single company can do much about. Well, one company can certainly make it worse, by e.g. ignoring years of API ergonomics around process spawning and give you a vaguely-specified "command" string, then pumping VC money into their marketing budget to make themselves an "indispensable" daily tool. (Mostly I agree with you - just want to point out that in addition to the necessary complexity, the tooling around it also produces a lot of unnecessarily wrong-by-default behavior.)
  - Brian_K_White 4y ago
    " the tooling around it also produces a lot of unnecessarily wrong-by-default behavior." It does. What I meant by one company doing something about it was, in realistic terms, no one can do much about the fact that the rest of the world uses a lot of complex systems that aren't as robust as earlier simpler systems. You have to use the current tools and that's just how they are now. You can try to be less stupid as possible, but you can't change the ecosystem you need to live in and interoperate with.

deleted 4y ago

[deleted]