5 ms·

Instead we're... listen to this... we're going to take a software developer right. Just a normal developer right. We're going to make them be the database exper

by tmpz22 23d ago

Instead we're... listen to this... we're going to take a software developer right. Just a normal developer right. We're going to make them be the database expert right. And the cloud expert. And we're going to put them on call. We're going to have them debug linux logs, and optimize our AWS costs. They'll be there for client escalation work. And big sales calls. From time to time we'll even have them do front end work. And get this. We pay them the exact same.

zbentley 23d ago
I think a DBA/ops/infrastructure person as an imposed bottleneck is a useful capability in some environments. But I won't follow you as far as "expecting developers to have expertise in how and where their software runs is unreasonable". Like, yeah, it sucks that added DevOps responsibilities etc. don't come with adjusted compensation/time allocation expectations. I'm with you there. But it's simultaneously true that a ton of "just regular developer" people are significant liabilities because they don't understand anything about the environment where their software runs. That liability manifests operationally (if someone's just running integration tests on Windows for their Java business logic changes and don't have any familiarity with e.g. the Linux, container, or cloud environments where their code runs, they're going to be useless when their code breaks in production and operations staff needs context), and it also makes them less effective when writing code--this culture of "developers should just live in business logic and not have to context-switch or fill their brains with other levels of the stack" is what leads to full table scans, lack of awareness of memory use, N+1 query hell, looping microservice dependencies, misunderstanding of what HTTP fields are set on requests that are mutated by load balancers, mistaken assumptions about how many instances of code can run and what concurrency/thread/coroutine behaviors are present, and so on. Those are very common problems, and it's incumbent on developers in every specialty to gain familiarity with how and where their code runs in order to write and maintain that code effectively. If your code runs on Linux in Kubernetes, all of your developers should know how to read Linux system logs, check database sessions/queries issued by parts of the application, ls/grep/cat/strace/ps their way around, interpret k8s/application dashboards, check application logs both in log storage and as they're emitted from a process, exec into a container, restart pods, check deployment liveness, etc. Even if they don't have permission to do those things in production. That was true in 2005 when they deployed their code to IIS on Windows Server/MSSQL, too--just with different operational specifics. That's a low bar that's often unmet, and all sorts of teams suffer from that failure. Those skills can be trained, kept up to date, and hired for; I don't think there's a great excuse for not expecting them.
- tmpz22 23d ago
  Apologies my comment was very tongue and cheek. I understand what you're getting at with regards to dedicated DB style teams being a potential bottleneck and not effective for every organization, as for example most orgs get by fine with a simple RDS instance with minimal management and optimization. What I'm calling out practically is a de-specialization in roles and the trade-offs it imposes. For example, the "jack of all trades master of one" that was already propagating under the guise of "Full stack engineer" or "DevOps" is further exasperated when those consolidated roles are also delegating critical knowledge acquisition to LLMs. Further, as we consolidate roles and delegate knowledge acquisition to llms, we're also paying people less, giving less job security, lowering the general quality of life not just for developers, but for operations staff and users, and generally promoting total consolidation within big tech.
  - zbentley 23d ago
    No apology needed; I understand what you're getting at, and I broadly agree. It's a spectrum between "extremely easy-to-hire people that operate in such a narrow niche that they're an operational liability with limited capabilities" and "expect everyone to be an expert at every level of the stack". The right point on that spectrum is different depending on context, but I do think that a majority of software shops would be better served by moving their required skillset more towards the generalist end of that spectrum, because the default is often far too niche (driven by poor tradeoffs and short-termism in service of growth/hiring, usually). I wanna re-emphasize that this is not a new problem. It's not because of DevOps culture or cloud complexity or scale or whatever. Very limited-specialty people were always operational liabilities and had limited positive impact on feature delivery once you accounted for the help they needed to do anything that spanned multiple levels of the stack. There are just more engineers working on more systems with tighter timeline expectations now, so it seems like the complexity incumbent on the engineering role went up in general. It didn't (it went up in some situations and down in some situations), we just started noticing operational pain more often. I definitely do agree that there's widespread ignorance of the velocity and difficulty-of-work tradeoffs that arise from requiring a wider range of specialties from engineers, and a similarly widespread failure to adjust compensation and timeline expectations accordingly.
throwaway894345 23d ago
A decade ago we would hire them fresh from some Ruby on Rails bootcamp so we could pay them less :shrug:
chasd00 23d ago
yes, that was the cloud and "devops" promise. ..or what it just another sham?
WarcrimeActual 23d ago
>Instead we're... listen to this... we're going to take a software developer right. Just a normal developer right. Maybe I'm old, and I am, but I just can't get past this point with such annoying writing. Like if you actually spoke like this people would hate you.
- dwedge 23d ago
  I have the same with the "if only there was a ..." sarcastic rebuttals. They belong 15 years ago and even then they were tiresome
- ezekiel68 23d ago
  I think the writing and the voice are both brilliant, in this case.
FLeXMurphy 23d ago
I read this in Steve Jobs voice. But maybe I was supposed to use Dr. Ian Malcolm instead?
mawadev 23d ago
Eerily accurate how it works these days, I wish you weren't correct. I met a DBA wizard (he looked like the creator of c++) at a banking IT dept and this guy intuitively sensed what you needed and how its done.
deleted 23d ago
[deleted]
Foobar8568 23d ago
And don't forget contract management with the supplier, L1-L2-L3 support, all in one, and integrating as the supplier is useless and your contract is shit. Oh and you will do also business analysis with the business as claude is too complex for them (read any version of the nocode initiative).
neya 23d ago
> And get this. We pay them the exact same. Why are you complaining? You should be grateful of the learning opportunity not everyone gets to have. Sure, we pay you peanuts for it. But, it's ultimately for your own good. Don't mind my yacht and Ferrari, though.
PunchyHamster 23d ago
That kinda how moving from programmers + sysadmins to devops looks like. Managers went happy coz now they don't need to have hire sysadmins, while in reality they hire sysadmins, call them devops, and have them know some programming on the side. And the "savings" from not having onprem infrastructure are burned on expensive cloud and debugging cloud blackboxes

by tmpz22 23d ago

zbentley 23d ago

I think a DBA/ops/infrastructure person as an imposed bottleneck is a useful capability in some environments. But I won't follow you as far as "expecting developers to have expertise in how and where their software runs is unreasonable". Like, yeah, it sucks that added DevOps responsibilities etc. don't come with adjusted compensation/time allocation expectations. I'm with you there. But it's simultaneously true that a ton of "just regular developer" people are significant liabilities because they don't understand anything about the environment where their software runs. That liability manifests operationally (if someone's just running integration tests on Windows for their Java business logic changes and don't have any familiarity with e.g. the Linux, container, or cloud environments where their code runs, they're going to be useless when their code breaks in production and operations staff needs context), and it also makes them less effective when writing code--this culture of "developers should just live in business logic and not have to context-switch or fill their brains with other levels of the stack" is what leads to full table scans, lack of awareness of memory use, N+1 query hell, looping microservice dependencies, misunderstanding of what HTTP fields are set on requests that are mutated by load balancers, mistaken assumptions about how many instances of code can run and what concurrency/thread/coroutine behaviors are present, and so on. Those are very common problems, and it's incumbent on developers in every specialty to gain familiarity with how and where their code runs in order to write and maintain that code effectively. If your code runs on Linux in Kubernetes, all of your developers should know how to read Linux system logs, check database sessions/queries issued by parts of the application, ls/grep/cat/strace/ps their way around, interpret k8s/application dashboards, check application logs both in log storage and as they're emitted from a process, exec into a container, restart pods, check deployment liveness, etc. Even if they don't have permission to do those things in production. That was true in 2005 when they deployed their code to IIS on Windows Server/MSSQL, too--just with different operational specifics. That's a low bar that's often unmet, and all sorts of teams suffer from that failure. Those skills can be trained, kept up to date, and hired for; I don't think there's a great excuse for not expecting them.

tmpz22 23d ago
Apologies my comment was very tongue and cheek. I understand what you're getting at with regards to dedicated DB style teams being a potential bottleneck and not effective for every organization, as for example most orgs get by fine with a simple RDS instance with minimal management and optimization. What I'm calling out practically is a de-specialization in roles and the trade-offs it imposes. For example, the "jack of all trades master of one" that was already propagating under the guise of "Full stack engineer" or "DevOps" is further exasperated when those consolidated roles are also delegating critical knowledge acquisition to LLMs. Further, as we consolidate roles and delegate knowledge acquisition to llms, we're also paying people less, giving less job security, lowering the general quality of life not just for developers, but for operations staff and users, and generally promoting total consolidation within big tech.
- zbentley 23d ago
  No apology needed; I understand what you're getting at, and I broadly agree. It's a spectrum between "extremely easy-to-hire people that operate in such a narrow niche that they're an operational liability with limited capabilities" and "expect everyone to be an expert at every level of the stack". The right point on that spectrum is different depending on context, but I do think that a majority of software shops would be better served by moving their required skillset more towards the generalist end of that spectrum, because the default is often far too niche (driven by poor tradeoffs and short-termism in service of growth/hiring, usually). I wanna re-emphasize that this is not a new problem. It's not because of DevOps culture or cloud complexity or scale or whatever. Very limited-specialty people were always operational liabilities and had limited positive impact on feature delivery once you accounted for the help they needed to do anything that spanned multiple levels of the stack. There are just more engineers working on more systems with tighter timeline expectations now, so it seems like the complexity incumbent on the engineering role went up in general. It didn't (it went up in some situations and down in some situations), we just started noticing operational pain more often. I definitely do agree that there's widespread ignorance of the velocity and difficulty-of-work tradeoffs that arise from requiring a wider range of specialties from engineers, and a similarly widespread failure to adjust compensation and timeline expectations accordingly.

throwaway894345 23d ago

A decade ago we would hire them fresh from some Ruby on Rails bootcamp so we could pay them less :shrug:

chasd00 23d ago

yes, that was the cloud and "devops" promise. ..or what it just another sham?

WarcrimeActual 23d ago

>Instead we're... listen to this... we're going to take a software developer right. Just a normal developer right. Maybe I'm old, and I am, but I just can't get past this point with such annoying writing. Like if you actually spoke like this people would hate you.

dwedge 23d ago
I have the same with the "if only there was a ..." sarcastic rebuttals. They belong 15 years ago and even then they were tiresome
ezekiel68 23d ago
I think the writing and the voice are both brilliant, in this case.

FLeXMurphy 23d ago

I read this in Steve Jobs voice. But maybe I was supposed to use Dr. Ian Malcolm instead?

mawadev 23d ago

Eerily accurate how it works these days, I wish you weren't correct. I met a DBA wizard (he looked like the creator of c++) at a banking IT dept and this guy intuitively sensed what you needed and how its done.

deleted 23d ago

[deleted]

Foobar8568 23d ago

And don't forget contract management with the supplier, L1-L2-L3 support, all in one, and integrating as the supplier is useless and your contract is shit. Oh and you will do also business analysis with the business as claude is too complex for them (read any version of the nocode initiative).

neya 23d ago

> And get this. We pay them the exact same. Why are you complaining? You should be grateful of the learning opportunity not everyone gets to have. Sure, we pay you peanuts for it. But, it's ultimately for your own good. Don't mind my yacht and Ferrari, though.

PunchyHamster 23d ago

That kinda how moving from programmers + sysadmins to devops looks like. Managers went happy coz now they don't need to have hire sysadmins, while in reality they hire sysadmins, call them devops, and have them know some programming on the side. And the "savings" from not having onprem infrastructure are burned on expensive cloud and debugging cloud blackboxes