10 ms·

This is so well said and on the nose that it hurt to read. I am entering into my third major project in a row where it’s almost the exact same situation (and s

This is so well said and on the nose that it hurt to read. I am entering into my third major project in a row where it’s almost the exact same situation (and same consultant I’ve bumped into) - site starts to scale, never had an ops team - realizes oh shit we cant manage this ourselves but who wants to hire some hacker nerd for $150k a year that “gEnErAtEs nO BuSiNesS rEvEnUe” so they hire a consultant or consultancy firm - firm builds them a solution that will technically “scale” but no one in the organization bothered to learn how it works - it works for a while til it doesnt and then the “oh shit” moment becomes “oh shit we need a dedicated devops team” - devops team attempts to work with the shitty consultant solution (editing yaml) and does an okay job until something breaks horribly again or they become a massive bottleneck - finally they allow someone to come in and clean house and you get an infra platform like the author of this blog is describing Platform engineering is the right word. “devops” has literally no meaning. I’m only 6 years or so into my career, but I can count on a single hand the amount of devops engineers I’ve worked with that can actually “dev” - and by dev, I mean do stuff like dig into application code and suggest modifications for the infrastructure, or write their own full fledged libraries. Let’s get rid of devops and hire platform engineers. The problem is it’s a really weird and rare combination of skillsets, in a field no one really wants to do except complete weirdos (I am one). I’ve long said that DevOps isnt a career you choose, it just happens to you. Well said! thank you so much for this post.

dijit 3y ago
20 years ago, when I was trying to enter the field, sysadmins were almost all former developers who had a pretty fundamental understanding of operating systems. Somewhere along the way "sysadmin" became a euphemism for helpdesk, and some folks thought "devops" (as a job title) was new, but it was just the same old sysadmins really, or new devs becoming new sysadmins with the fresher title, they felt themselves the vanguard of a new paradigm, I'm sure. Now we're in the same situation again, where "devops" (as a job title) can have no development experience. So we need a new term, apparently. Maybe gatekeeping is useful to stop this job title treadmill.
- JohnMakin 3y ago
  I think part of the reason this trend has been going on, and I have observed a similar thing as you point out here, is that businesses have tried to get rid of this “role” for a long time now and by constantly reinventing and renaming it. They have failed miserably. When people ask what I do, I call myself a systems engineer or “backend engineer” if that makes more sense to them. it’s more apt for what is actually needed and asked of me.
  - boffinAudio 3y ago
    Things went wrong when we started calling Computerization, "Information Technology". This was a totally misguided re-branding of the field. It attracted bureaucratic personalities and classist managerial types, and utterly destroyed the field. Like you, I couch my job description: I'm a "Systems Software Developer, I develop software systems" .. because its true, thats what I do, even if its a majority of the time working on embedded projects. And if someone then responds with "ooh, so you're in IT", I immediately go to another corner of the party and avoid them like the intolerable bores they are ..
    - flushit 3y ago
      > And if someone then responds with "ooh, so you're in IT", I immediately go to another corner of the party and avoid them like the intolerable bores they are .. No need to be arrogant towards people outside the field. My grandma doesn't know the difference between the random IT guy that updates your Windows printer driver and a Linux kernel dev, but I'd still not call her an "intolerable bore".
      - FrustratedMonky 3y ago
        Guess Problem is that a lot of managers also don't know, and have same understanding as your Grandma. I've seen plenty of companies take a very highly paid engineer and have them install printer drivers on peoples PC's.
jen20 3y ago
> we need a dedicated devops team Here’s where it went wrong. > “devops” has literally no meaning. No, it does, you’ve just chosen to let vendors, consultants and managers redefine it as “whatever we were already doing”.
- hnthrowaway0315 3y ago
  But if you don't have a dedicated ops team, some people get pigeon-holed for the role in the team. Not necessarily the best thing for them, because now they face pressures from both sides. I'd recommend the lead takes the role.
  - jen20 3y ago
    This is a simple organizational problem with a simple solution: don’t do that. Instead, ensure everyone on the team has a production pager, and that the person responsible for work prioritization gets every single page in order that they feel the pain of that (this may well be the lead, like you say!). Align incentives to create simple, reliable software.
    - maayank 3y ago
      > and that the person responsible for work prioritization gets every single page in order that they feel the pain of that. This sounds like an incredibly toxic work environment
      - jen20 3y ago
        It’s substantially less toxic than the people who prioritize work only prioritizing feature delivery, and not operability.
      - ryukafalz 3y ago
        Having one single person get all the pages does sound like it's going a bit far. The point though is for there to be a tight feedback loop between problems in the infrastructure and the ability to solve them. If this isn't the case, someone else in the organization is feeling that pain, but often they aren't empowered to solve them.
datadeft 3y ago
> Platform engineering is the right word. First of all platform engineering is two words. second of all what is my understanding that devops = platform engineering? Do you have a dictionary definition for these? No, so it does not make sense to say that you should use X instead of Y. Btw. systems engineering and technical operations are the things we usually mean by devops, deploying, configuring and scaling applications in an IT infrastructure and dealing with different aspects of running these applications. DevOps was just an approach to remove the silos that some companies built between development and operations bringing good old tribalism to technology because we cannot exist without tribalism. > Let’s get rid of devops and hire platform engineers. I wish this problem was so simple.
- JohnMakin 3y ago
  Sorry, english is not my first language, but I’m glad you pointed out my confusion about how many words are in “platform engineering.” > DevOps was just an approach to remove the silos that some companies built between development and operations I know this is always the intention but it is never implemented this way, it becomes a “throw crap over the wall” situation again because now you’ve made a second team and management sees that stuff as “their” job and the silos develop. I don’t profess to know the correct way to do it, but if I was managing an IT team, I’d assign a dedicated “Ops” guy to every dev team who would occasionally also write code. I dont care what you call this person. I’m in favor of platform engineer because it’s a lot more defined than “DevOps.” People have wildly different ideas about what “senior” devops is too. With a term like platform engineer, my gauge for if a person is a senior at that role is by the quality of the platform they can deliver to me. it’s vastly better, but again, I don’t give a crap what you call it as long as organizations finally understand what they’re hiring someone to do.
  - hotpotamus 3y ago
    I suppose I'm the guy you're talking about - I'm the lone ops guy on the one dev team for a small SaaS business. I try to write as little code as possible actually; which isn't to say I try to spend little time writing code, it's more that I try to rely on community maintained frameworks because I fear creating some complex duct-tape and zip-tie web of homebrew automation that becomes increasingly hard for myself (or god forbid another person) to maintain. I like the term platform engineer or maybe just infrastructure engineer, but I don't think they really add much specificity. For instance, we don't use any of the big three cloud providers, which I suspect would be an expectation for most of these titles. I think the possible diversity of modern tech stacks is just too much to pin these titles down, and mostly I just tell people I'm an "IT guy" ; it's only when you talk to someone in the industry that titles like Devops or Platform Engineer have any meaning, and as this thread shows, there's not even that much consensus within.
BFLpL0QNek 3y ago
I agree with all that. DevOps is similar to agile. Originally they both described a set of principles, an ethos, not a specific role. DevOps was your development team work closely with ops as one team, using software engineering practices, config as code etc etc. Then, it just came another name for sysadmin and later platform teams and back to where it all started, passing things over the wall from engineering to platforms.
- sam_lowry_ 3y ago
  > Originally they both described a set of principles, an ethos, not a specific role. The way I understood DevOps (back from the first DevOps Days in Ghent) is that clear separation of sysadmin and developer roles harms the process of software delivery and that developers should maintain their code up to production, and learn from the process. It was, in a way, a critique of the project-driven business where teams ship and jump over the board.
  - Joker_vD 3y ago
    The problem is, however, that those two roles require different knowledge, skills, attitudes and mindsets, and so many (most?) developers don't really like sysadminning: you know, messing with VLANs, DNS resolving, packet routing, gluing two networks (one in Sidney, second one in New York) into one via VPN, watching disk quotas and memory limits, working around bugs in Linux scheduler, pinning CPUs, all that mind-bogglingly boring (but needed for smooth operation) stuff.
    - sam_lowry_ 3y ago
      True. I enjoy making sure software works for people, the more people the better. If that involves all that sysadminning, I will enjoy that as well. But most of my peers are oblivious to the fact whether their code will be used at all and are having fun filling their heads with complex abstractions right before spitting them on screen.
  - sumtechguy 3y ago
    When it first started doing this I found it be to more tear down the moat many developers like to put between them and the rest of the business. Basically get up and go talk to QA, helpdesk, delivery, and marketing. Find out what is failing and fix that. Make sure those groups can use and maintain the code being delivered. THEN work on things that are new or champion those things that are broken to get fixed in the next cycle. That was devops to me. Then it turned into this guy who can not really do anything. Can not really check in code because the devs do that (separation of duties). So you never get any real experience in the codebase. Can not make any decisions because the marketing/product owners do that. Can not really make better test cases as that is for QA. You can however help on the helpdesk they are short 3 people today. Then the meetings. Endless meetings upon meetings. Because your 'devops' you need to know everything that is going on and better have those statuses on the high priority items to be fixed. Oh and you are 'on call' so you can never really 'go home'. Then if something breaks all you can do is just call in others and make them fix it. Making you little more than tier 3 helpdesk. Useless paper pusher job with zero authority.
gloryjulio 3y ago
In faanng componies I know, they r usually the infra engineer/reliability engineer/production engineer, etc. They are treated the same as typical software engineers just that their focus is not on the product.
JauntyHatAngle 3y ago
That was so chillingly accurate to my own experience I am actually baffled.
bob1029 3y ago
> Platform engineering is the right word. “devops” has literally no meaning. If you want to cut through all the BS, this is ultimately a debate about cloud vs on-prem relative to our ability to manage complexity across multiple vendors & styles of product. If you go to the cloud and actually do it entirely their way without antagonizing the process with 3rd parties and other ridiculous nonsense, you don't need leet hackers to figure things out for you and build elaborate processes that you'd require a "devops" team to support. In all 3 major clouds, there are well-documented happy paths that a junior engineer can follow to put a perfectly functional webapp live on a TLS-secured domain in about 1 business day. As far as I am aware, "getting the site live on public internet and making sure the certs are ok" is DevOps bread & butter. I am not going to pay for a dedicated team to do this kind of babysitting in 2023. The real tragedy is all of these companies who "went to the cloud" but just wound up in an even more complex & contorted hybrid stance with the cloud simply appended to their vendor list.
- dairem 3y ago
  I 100% agree with this. I don't see how the average internal platform-engineering team or "devops" team will be a better platform-engineering team than AWS/GCP etc.
  - bob1029 3y ago
    This is exactly how I've been selling the whole idea of hardcore, cloud-native to leadership & our customers. Most responses begin something like: "By leveraging the capabilities of a multi-trillion dollar Death Star ..." The objective is to get completely out of the game of managing VMs, certificates, authentication, MFA, et. al. There is no way in hell we can do a better job than any cloud provider when it comes to compliance, consistency, security, etc. Maybe when we have 10k+ employees we can reconsider the benefits of suffering additional complexity.
- JohnMakin 3y ago
  you’ve clearly never built a website of any scale or complexity on any of these clouds. they’re esoteric and fussy. the documentation is often poor and wrong. they trick organizations by providing tiers of expensive support but it’s not great, especially when you’re running into a novel problem. there are “easy” plug and play solutions offered but they cost a lot and come with tons of drawbacks. Building a functional, secure website at scale is a lot different than exposing some ports and loading certs.
datavirtue 3y ago
I'm more than happy to focus on "DevOps" (wretch) but not in the course of building an application. One or the other. If I'm building an app and working through all of the requirement gathering and development the last thing I want to do is fuck with containers, deployment, build scripts, or cloud services. I'm might just be an old grouch, but having this stuff foisted on me makes me want to quit everything. The AWS Lambda + Serverless framework shit trend has to go. There are no savings or agility. It never handles everything. The gaps are the painful parts, not the bits it gets right. I'll choose a simple nodejs app any day over that yaml trash fire.
acedTrex 3y ago
platform engineering is where all of the fun problems are too!
williamcotton 3y ago
For a brief moment about 15-20 years ago developers were actually doing operations work. The infrastructure was simple. Some monolithic app servers behind a load balancer, memcached, perhaps a master/slave SQL database but mainly big iron… and then someone coined the term DevOps to describe the process of automating the management of these services… One thing that was very different was provisioning new machines consisted of ssh’ing in and installing the required packages and binaries. The deployment was probably a simple script that used scp. However, once the term DevOps was coined almost immediately some people began to stop doing development work and just focused on automating operations work and constructing glorious crystalline castles in the clouds to handle traffic that could have been realistically served by 1U in a shelf somewhere. Complexity begets complexity.
- galleywest200 3y ago
  Having developers deploy the code to production is a security concern. You need permissions separated to ensure they do not sneak extra code into production after QA does their part. Our SOX audit was adamant about us changing this.
  - oldmanhorton 3y ago
    Is this sarcasm? What's stopping whoever does have the permissions from "sneaking code into production?" This seems like something pretty easily addressed by a combination of code signing and time-gated permissions. A small dev team may not have the time or energy to stand those systems up, but a blanket statement that it's a security concern seems like a bit much.
    - klooney 3y ago
      SOC II and other audits tend to come with the same baggage- they want strict roles and separation of duties between operations and engineering.
    - bob1029 3y ago
      > What's stopping whoever does have the permissions from "sneaking code into production?" In our arrangement, the ability to push code to production is gated by the GitHub/Azure integration path. The QA or project person who is rotating the production deployment slots (azure functions) is not granted access in GitHub to deploy to those same functions. So, the developers pushing code and those deploying code are mutually exclusive groups. You could still defeat this with collaboration between employees or screwing with AAD records, but that's why we have a ton of audit logging turned on too.
nvarsj 3y ago
What you say is very true and I have seen it myself a few times (even being said consultant :)). I’ll add that platform engineering is where much of the hardcore engineering happens nowadays in modern software stacks. In the olden days, SWEs had to know the hardware/low level software stack themselves. I think the SWE role is what is at risk of commodization. It’s something sufficiently good LLMs may even be able to do themselves at some point given the right instructions. Platform engineering will never go away - it’s close to the hardware and so many things can go wrong. You’re building the abstractions so SWEs can sit in their cozy IDEs debating about overly complex language features and libraries (said tongue in cheek as I’m guilty of this plenty).
CyberDildonics 3y ago
infra platform What is an "infra platform" ?
- yodon 3y ago
  Infra is short for Infrastructure
  - CyberDildonics 3y ago
    What is an 'infrastructure platform'?
vpastore 3y ago
[dead]
vaylian 3y ago
> Let’s get rid of devops and hire platform engineers. The problem is it’s a really weird and rare combination of skillsets, in a field no one really wants to do except complete weirdos (I am one) TIL about platform engineers. But I'm still unsure what kind of things they produce. I know that they basically create a self-service organisation-specific infrastructure that can be used by developers inside an organisation to deploy software. But how does this self-service look like? Do developers fill in a web form to automatically get a server with the right firewall rules?
- nunez 3y ago
  Explaining what platform engineers do requires explaining why platform engineering exists. The whole idea behind platform engineering is the mindset that the "platform", i.e. the stuff that runs the things that business cares about, is a product. Reframing the platform as a product instead of "IT" or "the place where apps go" introduces three concepts that were previously foreign to operations: 1. The platform has customers instead of consumers, and getting/acting upon their feedback is important, 2. The platform has some semblance of reliability guarantees, and copious observability is needed to uphold them, and 3. Like other products, the platform needs a dedicated, cross-functional team to maintain and improve on it. So basically platform engineers are engineers that focus on either the "developer experience" part of the platform (i.e. the platform's frontend) or the platforms back-end (servers, persistence, networking, etc.). > But how does this self-service look like? Do developers fill in a web form to automatically get a server with the right firewall rules? Kind-of. In an ideal platform, developers wouldn't even know or care that their stuff is on a server. It could be in a toaster, for all they care. So it's kind of just there. They write and maintain code locally, and their build and release pipelines stores and deploys into it.
- klooney 3y ago
  Yes. Honestly it generally looks like low quality wrappers around AWS that don't save anyone any time.
  - aprdm 3y ago
    You probably never worked on a complex system to think that
galleywest200 3y ago
I am not sure I agree. I am a DevOps by title and I am reading the flagship code before I try to get answers from development about why things do not work. I dig into stack traces, read legacy BS, "stare and compare" logs I had no idea even existed until this moment. I deploy code to production during the week after hours, I make SQL DB changes on the regular, both manually and via liquibase. I am expected to troubleshoot issues with kubernetes as well as standard applications. I do not think my organization would call this a platform engineer, as that title makes no sense for the work I do.
- aprdm 3y ago
  Imo platform engineering is basically that but putting web ui and apis on top so that it’s easier for devs to do it themselves
5350-uiop-1130 3y ago
> 6 years or so into my career, but I can count on a single hand the amount of devops engineers I’ve worked with that can actually “dev” - and by dev, I mean do stuff like dig into application code and suggest modifications for the infrastructure, or write their own full fledged libraries. damn, that sounds pretty good. need to look into some devops courses.
deleted 3y ago
[deleted]
nunez 3y ago
> Let’s get rid of devops and hire platform engineers. The problem is it’s a really weird and rare combination of skillsets, in a field no one really wants to do except complete weirdos (I am one). I’ve long said that DevOps isnt a career you choose, it just happens to you. It always has been. Always! This is why FAANGs paid their SREs top dollar. The problem, as you noted, always starts from the top. Cultural change needs to be championed by someone with power inside of the company. However, cultural change is slow (site starts to scale) and expensive ($150k hacker nerd) and many executives still think of tech as a cost center to be optimized (see also: ChatGPT). Platform Engineering now is where "DevOps" was in 2014 or so, but with more Kubernetes and Rust. But as long as executives who do not (or refuse to) understand the value that strong engineering teams bring forth, the Platform team will just be interchangeable with today's DevOps team.
0x445442 3y ago
Hmmm, Platform Engineer as you describe it kind of sounds like System Administrator.
MuffinFlavored 3y ago
> - site starts to scale, never had an ops team Can you be more specific? From what I understand, "DevOps" helps you get what you want into production. Whether that be a few Docker containers of your node.js or Rust or Go or Java or C# or whatever application (probably an HTTP microservice or a batch job) as well as your infrastructure (Redis, Postgres, a message queue maybe) Obviously scaling your microservice API from 2 to 4 isn't "all devops do". I also get that not everything is a Docker container. I'm just curious what I am missing. I get that for example, making Postgres "scalable" is much harder but... is that really a "devops" thing or is that a Postgres-specific thing? Same for Redis if you need multiple scalable instances. Scaling infrastructure isn't easy but I thought devops was just responsible basically for standing up the "pipelines" to deploy whatever you want. Why can't you just have a monorepo of application YAML that goes into ArgoCD or something like that? Where do you hit the "need devops/consulting" part specifically?
- JohnMakin 3y ago
  I usually come in after the fact to clean up whatever horrible thing the consulting firm did. I don’t want to malign them publicly but it’s the same one every single time. I think they might not “need” it as much as they think they do, but they want faster deployments, better automation, less downtime. Usually the devops team is the one that handles this for most organizations. ramming code into prod by scp’ing some files into a single monolithic host can get you a long way, but at a certain point, it doesn’t, IMHO.
  - MuffinFlavored 3y ago
    > but they want faster deployments, better automation, less downtime. Which part of that isn't covered by `kubectl apply / argocd sync` in a pipeline of some sort like a Jenkins or a TeamCity or a GitHub Actions?
klyonrad 3y ago
> “devops” has literally no meaning. I’m only 6 years or so into my career, but I can count on a single hand the amount of devops engineers I’ve worked with that can actually “dev” - and by dev, I mean do stuff like dig into application code and suggest modifications for the infrastructure, or write their own full fledged libraries. To me that is simply not DevOps. That's just Operations. Businesses are throwing around the title without a care for the word
moomoo11 3y ago
My title is platform engineer. In reality I spend my day doing everything from yes infra work and living in yaml, to working in Go/TS/ruby/elixir/etc repos and building those as well. We have dedicated infra team, and as platform engineer I have to know their job to an intermediate level in addition to having advanced level in shipping code across stacks. I love it but most other people seem to struggle with the constant context switching.
pk-protect-ai 3y ago
The shittiest thing is to think that "devops" is a team or profession or group of people or some practices for admins. DevOps is the paradigm which have to be followed by the company which generates revenue online.

dijit 3y ago

20 years ago, when I was trying to enter the field, sysadmins were almost all former developers who had a pretty fundamental understanding of operating systems. Somewhere along the way "sysadmin" became a euphemism for helpdesk, and some folks thought "devops" (as a job title) was new, but it was just the same old sysadmins really, or new devs becoming new sysadmins with the fresher title, they felt themselves the vanguard of a new paradigm, I'm sure. Now we're in the same situation again, where "devops" (as a job title) can have no development experience. So we need a new term, apparently. Maybe gatekeeping is useful to stop this job title treadmill.

JohnMakin 3y ago
I think part of the reason this trend has been going on, and I have observed a similar thing as you point out here, is that businesses have tried to get rid of this “role” for a long time now and by constantly reinventing and renaming it. They have failed miserably. When people ask what I do, I call myself a systems engineer or “backend engineer” if that makes more sense to them. it’s more apt for what is actually needed and asked of me.
- boffinAudio 3y ago
  Things went wrong when we started calling Computerization, "Information Technology". This was a totally misguided re-branding of the field. It attracted bureaucratic personalities and classist managerial types, and utterly destroyed the field. Like you, I couch my job description: I'm a "Systems Software Developer, I develop software systems" .. because its true, thats what I do, even if its a majority of the time working on embedded projects. And if someone then responds with "ooh, so you're in IT", I immediately go to another corner of the party and avoid them like the intolerable bores they are ..
  - flushit 3y ago
    > And if someone then responds with "ooh, so you're in IT", I immediately go to another corner of the party and avoid them like the intolerable bores they are .. No need to be arrogant towards people outside the field. My grandma doesn't know the difference between the random IT guy that updates your Windows printer driver and a Linux kernel dev, but I'd still not call her an "intolerable bore".
    - FrustratedMonky 3y ago
      Guess Problem is that a lot of managers also don't know, and have same understanding as your Grandma. I've seen plenty of companies take a very highly paid engineer and have them install printer drivers on peoples PC's.

jen20 3y ago

> we need a dedicated devops team Here’s where it went wrong. > “devops” has literally no meaning. No, it does, you’ve just chosen to let vendors, consultants and managers redefine it as “whatever we were already doing”.

hnthrowaway0315 3y ago
But if you don't have a dedicated ops team, some people get pigeon-holed for the role in the team. Not necessarily the best thing for them, because now they face pressures from both sides. I'd recommend the lead takes the role.
- jen20 3y ago
  This is a simple organizational problem with a simple solution: don’t do that. Instead, ensure everyone on the team has a production pager, and that the person responsible for work prioritization gets every single page in order that they feel the pain of that (this may well be the lead, like you say!). Align incentives to create simple, reliable software.
  - maayank 3y ago
    > and that the person responsible for work prioritization gets every single page in order that they feel the pain of that. This sounds like an incredibly toxic work environment
    - jen20 3y ago
      It’s substantially less toxic than the people who prioritize work only prioritizing feature delivery, and not operability.
    - ryukafalz 3y ago
      Having one single person get all the pages does sound like it's going a bit far. The point though is for there to be a tight feedback loop between problems in the infrastructure and the ability to solve them. If this isn't the case, someone else in the organization is feeling that pain, but often they aren't empowered to solve them.

datadeft 3y ago

> Platform engineering is the right word. First of all platform engineering is two words. second of all what is my understanding that devops = platform engineering? Do you have a dictionary definition for these? No, so it does not make sense to say that you should use X instead of Y. Btw. systems engineering and technical operations are the things we usually mean by devops, deploying, configuring and scaling applications in an IT infrastructure and dealing with different aspects of running these applications. DevOps was just an approach to remove the silos that some companies built between development and operations bringing good old tribalism to technology because we cannot exist without tribalism. > Let’s get rid of devops and hire platform engineers. I wish this problem was so simple.

JohnMakin 3y ago
Sorry, english is not my first language, but I’m glad you pointed out my confusion about how many words are in “platform engineering.” > DevOps was just an approach to remove the silos that some companies built between development and operations I know this is always the intention but it is never implemented this way, it becomes a “throw crap over the wall” situation again because now you’ve made a second team and management sees that stuff as “their” job and the silos develop. I don’t profess to know the correct way to do it, but if I was managing an IT team, I’d assign a dedicated “Ops” guy to every dev team who would occasionally also write code. I dont care what you call this person. I’m in favor of platform engineer because it’s a lot more defined than “DevOps.” People have wildly different ideas about what “senior” devops is too. With a term like platform engineer, my gauge for if a person is a senior at that role is by the quality of the platform they can deliver to me. it’s vastly better, but again, I don’t give a crap what you call it as long as organizations finally understand what they’re hiring someone to do.
- hotpotamus 3y ago
  I suppose I'm the guy you're talking about - I'm the lone ops guy on the one dev team for a small SaaS business. I try to write as little code as possible actually; which isn't to say I try to spend little time writing code, it's more that I try to rely on community maintained frameworks because I fear creating some complex duct-tape and zip-tie web of homebrew automation that becomes increasingly hard for myself (or god forbid another person) to maintain. I like the term platform engineer or maybe just infrastructure engineer, but I don't think they really add much specificity. For instance, we don't use any of the big three cloud providers, which I suspect would be an expectation for most of these titles. I think the possible diversity of modern tech stacks is just too much to pin these titles down, and mostly I just tell people I'm an "IT guy" ; it's only when you talk to someone in the industry that titles like Devops or Platform Engineer have any meaning, and as this thread shows, there's not even that much consensus within.

BFLpL0QNek 3y ago

I agree with all that. DevOps is similar to agile. Originally they both described a set of principles, an ethos, not a specific role. DevOps was your development team work closely with ops as one team, using software engineering practices, config as code etc etc. Then, it just came another name for sysadmin and later platform teams and back to where it all started, passing things over the wall from engineering to platforms.

sam_lowry_ 3y ago
> Originally they both described a set of principles, an ethos, not a specific role. The way I understood DevOps (back from the first DevOps Days in Ghent) is that clear separation of sysadmin and developer roles harms the process of software delivery and that developers should maintain their code up to production, and learn from the process. It was, in a way, a critique of the project-driven business where teams ship and jump over the board.
- Joker_vD 3y ago
  The problem is, however, that those two roles require different knowledge, skills, attitudes and mindsets, and so many (most?) developers don't really like sysadminning: you know, messing with VLANs, DNS resolving, packet routing, gluing two networks (one in Sidney, second one in New York) into one via VPN, watching disk quotas and memory limits, working around bugs in Linux scheduler, pinning CPUs, all that mind-bogglingly boring (but needed for smooth operation) stuff.
  - sam_lowry_ 3y ago
    True. I enjoy making sure software works for people, the more people the better. If that involves all that sysadminning, I will enjoy that as well. But most of my peers are oblivious to the fact whether their code will be used at all and are having fun filling their heads with complex abstractions right before spitting them on screen.
- sumtechguy 3y ago
  When it first started doing this I found it be to more tear down the moat many developers like to put between them and the rest of the business. Basically get up and go talk to QA, helpdesk, delivery, and marketing. Find out what is failing and fix that. Make sure those groups can use and maintain the code being delivered. THEN work on things that are new or champion those things that are broken to get fixed in the next cycle. That was devops to me. Then it turned into this guy who can not really do anything. Can not really check in code because the devs do that (separation of duties). So you never get any real experience in the codebase. Can not make any decisions because the marketing/product owners do that. Can not really make better test cases as that is for QA. You can however help on the helpdesk they are short 3 people today. Then the meetings. Endless meetings upon meetings. Because your 'devops' you need to know everything that is going on and better have those statuses on the high priority items to be fixed. Oh and you are 'on call' so you can never really 'go home'. Then if something breaks all you can do is just call in others and make them fix it. Making you little more than tier 3 helpdesk. Useless paper pusher job with zero authority.

gloryjulio 3y ago

In faanng componies I know, they r usually the infra engineer/reliability engineer/production engineer, etc. They are treated the same as typical software engineers just that their focus is not on the product.

JauntyHatAngle 3y ago

That was so chillingly accurate to my own experience I am actually baffled.

bob1029 3y ago

> Platform engineering is the right word. “devops” has literally no meaning. If you want to cut through all the BS, this is ultimately a debate about cloud vs on-prem relative to our ability to manage complexity across multiple vendors & styles of product. If you go to the cloud and actually do it entirely their way without antagonizing the process with 3rd parties and other ridiculous nonsense, you don't need leet hackers to figure things out for you and build elaborate processes that you'd require a "devops" team to support. In all 3 major clouds, there are well-documented happy paths that a junior engineer can follow to put a perfectly functional webapp live on a TLS-secured domain in about 1 business day. As far as I am aware, "getting the site live on public internet and making sure the certs are ok" is DevOps bread & butter. I am not going to pay for a dedicated team to do this kind of babysitting in 2023. The real tragedy is all of these companies who "went to the cloud" but just wound up in an even more complex & contorted hybrid stance with the cloud simply appended to their vendor list.

dairem 3y ago
I 100% agree with this. I don't see how the average internal platform-engineering team or "devops" team will be a better platform-engineering team than AWS/GCP etc.
- bob1029 3y ago
  This is exactly how I've been selling the whole idea of hardcore, cloud-native to leadership & our customers. Most responses begin something like: "By leveraging the capabilities of a multi-trillion dollar Death Star ..." The objective is to get completely out of the game of managing VMs, certificates, authentication, MFA, et. al. There is no way in hell we can do a better job than any cloud provider when it comes to compliance, consistency, security, etc. Maybe when we have 10k+ employees we can reconsider the benefits of suffering additional complexity.
JohnMakin 3y ago
you’ve clearly never built a website of any scale or complexity on any of these clouds. they’re esoteric and fussy. the documentation is often poor and wrong. they trick organizations by providing tiers of expensive support but it’s not great, especially when you’re running into a novel problem. there are “easy” plug and play solutions offered but they cost a lot and come with tons of drawbacks. Building a functional, secure website at scale is a lot different than exposing some ports and loading certs.

datavirtue 3y ago

I'm more than happy to focus on "DevOps" (wretch) but not in the course of building an application. One or the other. If I'm building an app and working through all of the requirement gathering and development the last thing I want to do is fuck with containers, deployment, build scripts, or cloud services. I'm might just be an old grouch, but having this stuff foisted on me makes me want to quit everything. The AWS Lambda + Serverless framework shit trend has to go. There are no savings or agility. It never handles everything. The gaps are the painful parts, not the bits it gets right. I'll choose a simple nodejs app any day over that yaml trash fire.

acedTrex 3y ago

platform engineering is where all of the fun problems are too!

williamcotton 3y ago

For a brief moment about 15-20 years ago developers were actually doing operations work. The infrastructure was simple. Some monolithic app servers behind a load balancer, memcached, perhaps a master/slave SQL database but mainly big iron… and then someone coined the term DevOps to describe the process of automating the management of these services… One thing that was very different was provisioning new machines consisted of ssh’ing in and installing the required packages and binaries. The deployment was probably a simple script that used scp. However, once the term DevOps was coined almost immediately some people began to stop doing development work and just focused on automating operations work and constructing glorious crystalline castles in the clouds to handle traffic that could have been realistically served by 1U in a shelf somewhere. Complexity begets complexity.

galleywest200 3y ago
Having developers deploy the code to production is a security concern. You need permissions separated to ensure they do not sneak extra code into production after QA does their part. Our SOX audit was adamant about us changing this.
- oldmanhorton 3y ago
  Is this sarcasm? What's stopping whoever does have the permissions from "sneaking code into production?" This seems like something pretty easily addressed by a combination of code signing and time-gated permissions. A small dev team may not have the time or energy to stand those systems up, but a blanket statement that it's a security concern seems like a bit much.
  - klooney 3y ago
    SOC II and other audits tend to come with the same baggage- they want strict roles and separation of duties between operations and engineering.
  - bob1029 3y ago
    > What's stopping whoever does have the permissions from "sneaking code into production?" In our arrangement, the ability to push code to production is gated by the GitHub/Azure integration path. The QA or project person who is rotating the production deployment slots (azure functions) is not granted access in GitHub to deploy to those same functions. So, the developers pushing code and those deploying code are mutually exclusive groups. You could still defeat this with collaboration between employees or screwing with AAD records, but that's why we have a ton of audit logging turned on too.

nvarsj 3y ago

What you say is very true and I have seen it myself a few times (even being said consultant :)). I’ll add that platform engineering is where much of the hardcore engineering happens nowadays in modern software stacks. In the olden days, SWEs had to know the hardware/low level software stack themselves. I think the SWE role is what is at risk of commodization. It’s something sufficiently good LLMs may even be able to do themselves at some point given the right instructions. Platform engineering will never go away - it’s close to the hardware and so many things can go wrong. You’re building the abstractions so SWEs can sit in their cozy IDEs debating about overly complex language features and libraries (said tongue in cheek as I’m guilty of this plenty).

CyberDildonics 3y ago

infra platform What is an "infra platform" ?

yodon 3y ago
Infra is short for Infrastructure
- CyberDildonics 3y ago
  What is an 'infrastructure platform'?

vpastore 3y ago

[dead]

vaylian 3y ago

> Let’s get rid of devops and hire platform engineers. The problem is it’s a really weird and rare combination of skillsets, in a field no one really wants to do except complete weirdos (I am one) TIL about platform engineers. But I'm still unsure what kind of things they produce. I know that they basically create a self-service organisation-specific infrastructure that can be used by developers inside an organisation to deploy software. But how does this self-service look like? Do developers fill in a web form to automatically get a server with the right firewall rules?

nunez 3y ago
Explaining what platform engineers do requires explaining why platform engineering exists. The whole idea behind platform engineering is the mindset that the "platform", i.e. the stuff that runs the things that business cares about, is a product. Reframing the platform as a product instead of "IT" or "the place where apps go" introduces three concepts that were previously foreign to operations: 1. The platform has customers instead of consumers, and getting/acting upon their feedback is important, 2. The platform has some semblance of reliability guarantees, and copious observability is needed to uphold them, and 3. Like other products, the platform needs a dedicated, cross-functional team to maintain and improve on it. So basically platform engineers are engineers that focus on either the "developer experience" part of the platform (i.e. the platform's frontend) or the platforms back-end (servers, persistence, networking, etc.). > But how does this self-service look like? Do developers fill in a web form to automatically get a server with the right firewall rules? Kind-of. In an ideal platform, developers wouldn't even know or care that their stuff is on a server. It could be in a toaster, for all they care. So it's kind of just there. They write and maintain code locally, and their build and release pipelines stores and deploys into it.
klooney 3y ago
Yes. Honestly it generally looks like low quality wrappers around AWS that don't save anyone any time.
- aprdm 3y ago
  You probably never worked on a complex system to think that

galleywest200 3y ago

I am not sure I agree. I am a DevOps by title and I am reading the flagship code before I try to get answers from development about why things do not work. I dig into stack traces, read legacy BS, "stare and compare" logs I had no idea even existed until this moment. I deploy code to production during the week after hours, I make SQL DB changes on the regular, both manually and via liquibase. I am expected to troubleshoot issues with kubernetes as well as standard applications. I do not think my organization would call this a platform engineer, as that title makes no sense for the work I do.

aprdm 3y ago
Imo platform engineering is basically that but putting web ui and apis on top so that it’s easier for devs to do it themselves

5350-uiop-1130 3y ago

> 6 years or so into my career, but I can count on a single hand the amount of devops engineers I’ve worked with that can actually “dev” - and by dev, I mean do stuff like dig into application code and suggest modifications for the infrastructure, or write their own full fledged libraries. damn, that sounds pretty good. need to look into some devops courses.

deleted 3y ago

[deleted]

nunez 3y ago

> Let’s get rid of devops and hire platform engineers. The problem is it’s a really weird and rare combination of skillsets, in a field no one really wants to do except complete weirdos (I am one). I’ve long said that DevOps isnt a career you choose, it just happens to you. It always has been. Always! This is why FAANGs paid their SREs top dollar. The problem, as you noted, always starts from the top. Cultural change needs to be championed by someone with power inside of the company. However, cultural change is slow (site starts to scale) and expensive ($150k hacker nerd) and many executives still think of tech as a cost center to be optimized (see also: ChatGPT). Platform Engineering now is where "DevOps" was in 2014 or so, but with more Kubernetes and Rust. But as long as executives who do not (or refuse to) understand the value that strong engineering teams bring forth, the Platform team will just be interchangeable with today's DevOps team.

0x445442 3y ago

Hmmm, Platform Engineer as you describe it kind of sounds like System Administrator.

MuffinFlavored 3y ago

> - site starts to scale, never had an ops team Can you be more specific? From what I understand, "DevOps" helps you get what you want into production. Whether that be a few Docker containers of your node.js or Rust or Go or Java or C# or whatever application (probably an HTTP microservice or a batch job) as well as your infrastructure (Redis, Postgres, a message queue maybe) Obviously scaling your microservice API from 2 to 4 isn't "all devops do". I also get that not everything is a Docker container. I'm just curious what I am missing. I get that for example, making Postgres "scalable" is much harder but... is that really a "devops" thing or is that a Postgres-specific thing? Same for Redis if you need multiple scalable instances. Scaling infrastructure isn't easy but I thought devops was just responsible basically for standing up the "pipelines" to deploy whatever you want. Why can't you just have a monorepo of application YAML that goes into ArgoCD or something like that? Where do you hit the "need devops/consulting" part specifically?

JohnMakin 3y ago
I usually come in after the fact to clean up whatever horrible thing the consulting firm did. I don’t want to malign them publicly but it’s the same one every single time. I think they might not “need” it as much as they think they do, but they want faster deployments, better automation, less downtime. Usually the devops team is the one that handles this for most organizations. ramming code into prod by scp’ing some files into a single monolithic host can get you a long way, but at a certain point, it doesn’t, IMHO.
- MuffinFlavored 3y ago
  > but they want faster deployments, better automation, less downtime. Which part of that isn't covered by `kubectl apply / argocd sync` in a pipeline of some sort like a Jenkins or a TeamCity or a GitHub Actions?

klyonrad 3y ago

> “devops” has literally no meaning. I’m only 6 years or so into my career, but I can count on a single hand the amount of devops engineers I’ve worked with that can actually “dev” - and by dev, I mean do stuff like dig into application code and suggest modifications for the infrastructure, or write their own full fledged libraries. To me that is simply not DevOps. That's just Operations. Businesses are throwing around the title without a care for the word

moomoo11 3y ago

My title is platform engineer. In reality I spend my day doing everything from yes infra work and living in yaml, to working in Go/TS/ruby/elixir/etc repos and building those as well. We have dedicated infra team, and as platform engineer I have to know their job to an intermediate level in addition to having advanced level in shipping code across stacks. I love it but most other people seem to struggle with the constant context switching.

pk-protect-ai 3y ago

The shittiest thing is to think that "devops" is a team or profession or group of people or some practices for admins. DevOps is the paradigm which have to be followed by the company which generates revenue online.