3 ms·

Honest question, what would you call a role that: * Is on call * Manages internal software (grafana, Prometheus, salt stack, etc) * is the first line of defe

Honest question, what would you call a role that: * Is on call * Manages internal software (grafana, Prometheus, salt stack, etc) * is the first line of defense for issues in the field, works with support and the engineering team to handle problems * Manages a distributed fleet of servers (uses off the self and/or custom code to do so) * Builds internal tools/automations to improve the reliability of our platform and/or address problems automatically * Pushes us to make software/OS updates I ask because that’s what I’d love to hire. Someone who owns the production environment and has the time/space to focus on improving/maintaining it. “Ops” or “Production engineer” is what I’ve referred to it as in the past but this seems like a good place to ask what others would call it.

liveoneggs 2y ago
I've settled on "systems engineer" -- I just try to keep "systems" in the title to imply that my job is largely -- "making many things work together to run the business" I think my current title has the word "cloud" in it or something and my boss rejects (with prejudice) "SRE" or "devops" explicitly to avoid being labeled as operations.
cedws 2y ago
>what would you call a role that … Overworked?
deleted 2y ago
[deleted]
Chilinot 2y ago
"Platform Engineer" has started to grow as a name for pretty much this. An engineer that builds the platform that all things run on. For example, the k8s clusters, the observability stack, has on-call, and builds the internal tools and automations (called the IDP, or internal developer platform).
- liveoneggs 2y ago
  IDP already has a very important meaning as a part of any platform :)
  - Chilinot 2y ago
    Yea, it's confusing, i give you that.
- attendant3446 2y ago
  Apparently not everyone gets it. I once went to an AWS meetup and introduced myself as a "platform engineer", but not everyone understood what I did =) But personally, I like "platform engineer" more than "ops".
jfindley 2y ago
As long as that very description is spelled out clearly near the top of the job ad, it doesn't matter terribly much (within reason) what you call it - job searchers will try various different strings to find it. Personally, I'd call it an ops role, however. In general, most of the issues I've seen with these sorts of roles aren't the naming of them but rather the third bullet in your list. Having an ops role that's responsible for all the problems with software they didn't build and weren't allowed to have meaningful input to the design/implementation of isn't healthy. It sucks up their time and energy fighting fires they didn't create and likely aren't really empowered to fix. This is the problem with this role (as often implemented) that Rachel talks about in her first paragraph. If you really care about having a good and reliable product you need people involved with the design and implementation that are deeply invested in making it reliable and maintainable in the long term - which means either making your dev roles shoulder some of the oncall burden or having people that straddle the ops and dev teams. Or both. If you're having difficulty filling this role, perhaps this is the real problem?
- joshstrange 2y ago
  I haven’t started trying to hire for this position yet, it’s been something I’ve been thinking of for a while though. Right a small number of developers, myself included, are on call and I’d like to reduce that a bit or just share the burden. My goal isn’t to throw crap over the fence and say “make it work” but rather empower someone to make maintaining and growing our platform their main goal. The developers (again, myself included) are not great at the ops side of things and can rarely focus on the infrastructure itself due to other priorities (yes, we can talk about how that itself is an issue). If I could clone myself and one of specialize in ops and the other on programming for the platform I would in a heartbeat. Infra/Ops and programming are two different mindsets (much like managing people or qa differs from writing code). Switching between them is hard and you pay a penalty to do so. Not to mention there are skills (networking is high on that list) that I’m not good at. I can scrape by but that’s not where my skills lie. That’s why I’d like to hire someone who is good at it, who _does_ enjoy it, and who push for changes from a ops perspective that I can’t due to time or skill.
  - 2y ago
Galanwe 2y ago
What I've experienced working well is to consider platform/infra as just another dev team. They should experience the same good practices (staging changes, tests, documentation, clean code) and duties (level 2 on calls, regular postmortems from ops, etc). Ops/prod/support eng are generally more business related and less technical than infra/platform. They make sure the processes run as they should, or operate with agreed procedures when something is raised. Some issues may be lightly technical ("theres an alert on free disk space of server X", "provider Y changed their SFTP keys without notice", etc) and some business related ("provider Z is late to push updates", "it's a holiday on country of provider A", "service of team B raises an exception when situation C occurs"). They are often level 1, in between the actual dev teams (escalating issues to them, or asking for better stability, logs, resiliency, etc) and the platform/infra team. Being faced with production issues is a burden for focus, etc. I wouldn't want my platform team to be on level 1 handling these stuff or nothing get done. Still, I wouldn't want them to be completely free from it or they would loose track with reality, just as any other dev team. To summarize, I would say: - A fleet of business dev team - A fleet of platform / infra team - A support/ops/prod team handling level 1 through procedures written by dev/platform teams, and raising to infra or business dev for level 2. Regular feedback sessions with each other team to have them stay on touch with real life. Some light coding / improvements tasks paired with infra or dev to get them a better understanding of the underlying layers. From the description you provided, that's what I would call a "platform engineer". Though the titles are sensitive for some people, and there's a lot of discrepancies between companies, so i usually let people from platform choose the title they prefer, between SRE / DevOps / platform eng / cloud eng / infra eng.
deleted 2y ago
[deleted]
pjmlp 2y ago
It used to be systems administrator, or network engineer, depending how deep into infrastructure it goes.
kgeist 2y ago
At our organization, a bunch of devs which were interested in improving/automating infrastructure split into a separate DevOps team. They first called themselves DevOps, then renamed themselves to SRE, then again to Production. Every time they changed their name they prepared presentations for the whole company explaining the name change and why the previous name wasn't correct after all. Everyone is still confused and always forgets what's their current official name. Other than that, there's also: - a separate IT department which manages networks, telephony and access - the ones who push people to update software/OS is InfoSec, a separate team - ordinary developers can make changes to k8s/grafana/Prometheus configs for their services, if it's reviewed by Production as the owners (they're understaffed compared to the multiple dev teams) - there's also the "Core" team which deals with the microservice platform on top of bare infrastructure (how microservices should communicate, etc.) Often the lines are blurred and it's not always clear who you call if you have a problem.
nunez 2y ago
That's reliability engineering, IMO.

liveoneggs 2y ago

I've settled on "systems engineer" -- I just try to keep "systems" in the title to imply that my job is largely -- "making many things work together to run the business" I think my current title has the word "cloud" in it or something and my boss rejects (with prejudice) "SRE" or "devops" explicitly to avoid being labeled as operations.

cedws 2y ago

>what would you call a role that … Overworked?

deleted 2y ago

[deleted]

Chilinot 2y ago

"Platform Engineer" has started to grow as a name for pretty much this. An engineer that builds the platform that all things run on. For example, the k8s clusters, the observability stack, has on-call, and builds the internal tools and automations (called the IDP, or internal developer platform).

liveoneggs 2y ago
IDP already has a very important meaning as a part of any platform :)
- Chilinot 2y ago
  Yea, it's confusing, i give you that.
attendant3446 2y ago
Apparently not everyone gets it. I once went to an AWS meetup and introduced myself as a "platform engineer", but not everyone understood what I did =) But personally, I like "platform engineer" more than "ops".

jfindley 2y ago

As long as that very description is spelled out clearly near the top of the job ad, it doesn't matter terribly much (within reason) what you call it - job searchers will try various different strings to find it. Personally, I'd call it an ops role, however. In general, most of the issues I've seen with these sorts of roles aren't the naming of them but rather the third bullet in your list. Having an ops role that's responsible for all the problems with software they didn't build and weren't allowed to have meaningful input to the design/implementation of isn't healthy. It sucks up their time and energy fighting fires they didn't create and likely aren't really empowered to fix. This is the problem with this role (as often implemented) that Rachel talks about in her first paragraph. If you really care about having a good and reliable product you need people involved with the design and implementation that are deeply invested in making it reliable and maintainable in the long term - which means either making your dev roles shoulder some of the oncall burden or having people that straddle the ops and dev teams. Or both. If you're having difficulty filling this role, perhaps this is the real problem?

joshstrange 2y ago
I haven’t started trying to hire for this position yet, it’s been something I’ve been thinking of for a while though. Right a small number of developers, myself included, are on call and I’d like to reduce that a bit or just share the burden. My goal isn’t to throw crap over the fence and say “make it work” but rather empower someone to make maintaining and growing our platform their main goal. The developers (again, myself included) are not great at the ops side of things and can rarely focus on the infrastructure itself due to other priorities (yes, we can talk about how that itself is an issue). If I could clone myself and one of specialize in ops and the other on programming for the platform I would in a heartbeat. Infra/Ops and programming are two different mindsets (much like managing people or qa differs from writing code). Switching between them is hard and you pay a penalty to do so. Not to mention there are skills (networking is high on that list) that I’m not good at. I can scrape by but that’s not where my skills lie. That’s why I’d like to hire someone who is good at it, who _does_ enjoy it, and who push for changes from a ops perspective that I can’t due to time or skill.
- 2y ago

Galanwe 2y ago

What I've experienced working well is to consider platform/infra as just another dev team. They should experience the same good practices (staging changes, tests, documentation, clean code) and duties (level 2 on calls, regular postmortems from ops, etc). Ops/prod/support eng are generally more business related and less technical than infra/platform. They make sure the processes run as they should, or operate with agreed procedures when something is raised. Some issues may be lightly technical ("theres an alert on free disk space of server X", "provider Y changed their SFTP keys without notice", etc) and some business related ("provider Z is late to push updates", "it's a holiday on country of provider A", "service of team B raises an exception when situation C occurs"). They are often level 1, in between the actual dev teams (escalating issues to them, or asking for better stability, logs, resiliency, etc) and the platform/infra team. Being faced with production issues is a burden for focus, etc. I wouldn't want my platform team to be on level 1 handling these stuff or nothing get done. Still, I wouldn't want them to be completely free from it or they would loose track with reality, just as any other dev team. To summarize, I would say: - A fleet of business dev team - A fleet of platform / infra team - A support/ops/prod team handling level 1 through procedures written by dev/platform teams, and raising to infra or business dev for level 2. Regular feedback sessions with each other team to have them stay on touch with real life. Some light coding / improvements tasks paired with infra or dev to get them a better understanding of the underlying layers. From the description you provided, that's what I would call a "platform engineer". Though the titles are sensitive for some people, and there's a lot of discrepancies between companies, so i usually let people from platform choose the title they prefer, between SRE / DevOps / platform eng / cloud eng / infra eng.

deleted 2y ago

[deleted]

pjmlp 2y ago

It used to be systems administrator, or network engineer, depending how deep into infrastructure it goes.

kgeist 2y ago

At our organization, a bunch of devs which were interested in improving/automating infrastructure split into a separate DevOps team. They first called themselves DevOps, then renamed themselves to SRE, then again to Production. Every time they changed their name they prepared presentations for the whole company explaining the name change and why the previous name wasn't correct after all. Everyone is still confused and always forgets what's their current official name. Other than that, there's also: - a separate IT department which manages networks, telephony and access - the ones who push people to update software/OS is InfoSec, a separate team - ordinary developers can make changes to k8s/grafana/Prometheus configs for their services, if it's reviewed by Production as the owners (they're understaffed compared to the multiple dev teams) - there's also the "Core" team which deals with the microservice platform on top of bare infrastructure (how microservices should communicate, etc.) Often the lines are blurred and it's not always clear who you call if you have a problem.

nunez 2y ago

That's reliability engineering, IMO.