12 ms·

How platform engineering works

112 points by chadxz 3y ago

CSMastermind 3y ago
I really dislike the co-opting of the phrase "Platform Engineering" as a replacement for DevOps.
leetrout 3y ago
I really like that outcomes over outputs is called out. I think this is important whenever programming / development meets The Real World(tm) _ESPECIALLY_ with internal tooling work.
imwillofficial 3y ago
I despise this buzzword.
ofrzeta 3y ago
In my opinion a lot of Devops comes down to improving "developer experience" (DX - another buzzword, but still). It starts with project setup and choosing tools that enable this such as JS build tools that are fast. Then you have easy (and again: fast) means of running tests, continuous integration, deployment for staging (or sharing with less technical team members). This has to be as frictionless as possible so developers can actually enjoy their work. A "by-product" is that you can also more easily deploy for production.
oofnik 3y ago
Here's the thing about buzzwords - they are a useful abstraction for grouping together a set of concepts for non-tech people to make decisions with tech impact. The ideas presented here are a good summary of the day-to-day issues I experience as a "DevOps engineer." I wish I had the pull to instigate a shift in mindset towards a more platform-oriented mode of work, but unfortunately for me, that is not the case. Maybe it's wishful thinking, but forwarding articles and presentations like this to senior directors can be a way to influence the direction of their thinking to align more with what you believe to be in the best interest of the organization.
- deleted 3y ago
  [deleted]
matsemann 3y ago
This may be overly negative to a whole field, but I sometimes feek the platform teams add more hurdles than "stability and velocity". At places with basically no platform team, no advanced cloud setup etc, I as a dev could understand everything, and deploying mostly meant getting my jar file or whatever running on some ec2 instance. Now I need to write a dockerfile, loads of manifests, know about ingresses, use whatever custom tooling and abstractions they've built on top of the cloud provider. And what used to be a single guy is now a huge team just adding more complexity. And I'm having a hard time understanding if it's worth it, or just a cost sink.
- reillyse 3y ago
  Sounds a bit like those places aren’t at the scale where a platform team makes sense. What about when the one dev deploying their jar to an ec2 instance moves on to another company, how does the next dev even understand what this jar stuff is when they just want to push their SPA to vercel. Allowing devs to do whatever they want works at small new orgs but you need to put some kind of shape on it as they grow.
  - adrianmsmith 3y ago
    > What about when the one dev deploying their jar to an ec2 instance moves on to another company, how does the next dev even understand what this jar stuff is I think that's more a problem of one person vs a team, as opposed to deploying JARs on EC2s vs cloud and a bunch of custom tooling. If you have a team of devs and they all deploy JARs to EC2, one of them leaving won't be a problem, the rest will still know how to do it. If you were to have a single platform engineer who's built a bunch of custom tooling over a bunch of Kubernetes files, and nobody knows how it works or where the files are, and then they leave, you've got the same problem as the solo EC2 dev leaving.
    - re-thc 3y ago
      > I think that's more a problem of one person vs a team Not always. The "team" you imagine might also be 1 person deploying the jar and the rest copying them. When that person leaves suddenly know 1 knows what to do anymore either.
anotherhue 3y ago
You'd be amazed how little platform engineering is needed if you can violently constrain technology choices. Give me a blobstore and some VMs and I'll be happy, and if someone raises the slightest fuss I will point out that it's still years better than helm.
- hhh 3y ago
  What’s wrong with helm?
  - nforgerit 3y ago
    IMO lots of accidental complexity with no real value (in most use cases). I saw myself disagreeing frequently with pre-defined charts and creating an own chart just for one use case is better done with kustomize. It might make sense though for a bigger org with at least several dozens of engineers to ensure certain internal standards (in which platform engineering might make sense as well).
  - pirates 3y ago
    i think helm is “fine”, but the more i use it and get deep into some more advanced uses the more i dislike it. helm has a tendency to sprawl really fast. the ability to template any part of a manifest can lead to “if there is even a chance the default will ever be changed we should make it a template” (at my company at least). when your entire chart is “if values.thing X, else Y”, it makes it difficult to be confident about what exactly will be deployed. especially if you only have a chart and values can be inserted in many places (helmfile). without helm template it’s sometimes impossible to reason with all the meta-templates. speaking of debugging, helm template sometimes has very awful debug output and traces of where your error is. in more simple charts it’s not so bad. with library charts you can spend hours debugging, say, your deployment.yaml because it’s saying there’s an error on line 4 of the template, but line 4 is just some label or benign key, and oh finally i figured out that the deployment expects the configmap to be rendered, and the typo is in line 4 of the CM, not the deployment helm lint doesn’t catch duplicate keys? i found that one weird tools like argocd don’t like helm too much. yeah you can make it work, but flat yaml is so much simpler. and if you use helm + argo, it turns the helm into flat yaml anyway i have heavy dislike of the syntax of ‘and values.thing1 values.thing2’ in a templated line. (same with ‘or’, though this might just be a personal hangup) finally, to be positive, the one thing i really like about helm actually is the checksum function that changes a label on the deploy/sts/ds when you change config, so pods automatically roll over.
BlackFly 3y ago
What does it mean to be greater than the sum of your parts? It is more than just alignment, everybody moving in the same direction: that would just be a well directed sum.Once upon a time, management talked about alignment because just being the sum of your parts would be great if it was all in the same direction. Now, for everybody obsessed with being a force multiplier, it needs to be more than just that. You should find yourself more capable in such an organization, but that is a tall order in today's world. Almost every problem you will face has been solved and help is easy to find even on your own. So being in such an organization should make you more capable of solving problems than your base ability. Someone must have already solved that problem for you and the solution must be in easy reach. That's still a tall order, most solutions you find described on the internet are also in easy reach nowadays. Your platform can also make people less capable. It is easier to be a force multiplier with a coefficient less than one. Make it more difficult to solve problems. Require them to communicate with many teams in order to accomplish anything. Make the solution difficult to integrate and operate and highly non-standard. If the solution is non-standard and the documentation is anything short of impeccable, it will basicly be unlearnable. The book Accelerate and the research backing it indicates that the most productive teams are able to redesign their entire system without talking to anyone and to complete their work without communication and coordination to other teams. Effective platforms advance that. They make it easy for people to redesign their systems to implement good designs. Bad platforms reduce that capability beyond what a person would ordinarily have and user complaints are brushed off with a tautological, "You can't make everyone happy." Bad platforms narrow possibilities and then require product teams to beg platform teams to allow ordinary designs that were cut off. I'm on a platform team and I struggle sometimes trying to figure out if we are helping people or hurting them hence this philosophical rant.
- mkl95 3y ago
  > What does it mean to be greater than the sum of your parts? In my experience it's mostly automation and observability. Delegating as much stuff as possible on tools as opposed to humans, empowering the latter to fix things.
  - BlackFly 3y ago
deleted 3y ago
[deleted]
layer8 3y ago
After reading the article I still don’t know what “platform engineering” is meant to refer to.
- KineticLensman 3y ago
  > After reading the article I still don’t know what “platform engineering” is meant to refer to. It wasn't obvious to me either, so I found the following definitions. I still think that the article doesn't say 'how PE works' as opposed to providing a bunch of tips. "The engineering platform is created and maintained by a dedicated product team, designed to support the needs of software developers and others by providing common, reusable tools and capabilities, and interfacing to complex infrastructure." [0] "Platform engineering is the process of designing and implementing toolchains that improve the software delivery experience. Platform engineers set up automated infrastructure and self-service controls that allow developers to work more efficiently." [1] "Platform engineering is the “discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era. Platform engineers provide an integrated product most often referred to as an ‘Internal Developer Platform’ covering the operational necessities of the entire lifecycle of an application." [2] [0] https://www.gartner.com/en/articles/what-is-platform-engineering https://www.gartner.com/en/articles/what-is-platform-enginee... [1] https://www.howtogeek.com/devops/platform-engineering-vs-devops-what-you-need-to-know/ https://www.howtogeek.com/devops/platform-engineering-vs-dev... [2] https://humanitec.com/blog/sre-vs-devops-vs-platform-engineering https://humanitec.com/blog/sre-vs-devops-vs-platform-enginee...
dools 3y ago
[flagged]
techperson20007 3y ago
Ahh yes. The bi-monthly shit on ops comment threads. Ops folks can barely reverse a string. We should not exist as a field. Every company regardless of scale should just use 4 manually provisioned EC2 instances. If that doesn’t work your code is shit and you aren’t using Elixir / Phoenix! Anyone even attempting to use Kubernetes should be branded with the Hetzner logo and forced to work at McDonalds. Fear not HN. McDonalds here I come.
- matsemann 3y ago
  That's a lot of straw people.. There are lots of valid criticism, dismissing all of that is arrogant and counter productive.
  - slyall 3y ago
    The largest thread here is people saying "deploying mostly meant getting my jar file or whatever running on some ec2 instance" and talking about manually setting up ec2 instances and each team somehow copying them into production. Just last week there was a thread about a company with 250 Petabytes in s3 and every comment from a developer was how it should be moved in-house. People were linking to one piece of s3-compatible software that the only example of a live deploy was single-digit gigabytes.
- deterministic 3y ago
  Mostly justified in my experience. In my experience, every time an “ops” team gets involved, complexity and cost goes up, with zero or negative benefits. Your experience might be different of course, but that doesn’t invalidate the negative experiences of others.
swyx 3y ago
> Platform Engineering is the application of a Product Mindset to supporting your engineering organization's software delivery velocity and system stability. so… move fast and don’t break things? must be nice to never have tradeoffs
deleted 3y ago
[deleted]
deterministic 3y ago
The whole point of “DevOps” was to not have a separate platform/deployment/operations team. Each development team should handle everything needed to build, test, deploy, and monitor production code. Every time I have been on a team where that was not the case, the result was bloated complexity and more failures in production. Usually because the platform/deployment/operations team was composed of not very skilled developers trying to justify their salary to management. If you have worked with a platform/deployment/operations team that was competent and reduced complexity then great for you. I have yet to experience such a phenomenon. Please let me know in comments why you think your experience was different.

112 points by chadxz 3y ago

I really dislike the co-opting of the phrase "Platform Engineering" as a replacement for DevOps.

I really like that outcomes over outputs is called out. I think this is important whenever programming / development meets The Real World(tm) _ESPECIALLY_ with internal tooling work.

imwillofficial 3y ago

I despise this buzzword.

ofrzeta 3y ago

In my opinion a lot of Devops comes down to improving "developer experience" (DX - another buzzword, but still). It starts with project setup and choosing tools that enable this such as JS build tools that are fast. Then you have easy (and again: fast) means of running tests, continuous integration, deployment for staging (or sharing with less technical team members). This has to be as frictionless as possible so developers can actually enjoy their work. A "by-product" is that you can also more easily deploy for production.

oofnik 3y ago

Here's the thing about buzzwords - they are a useful abstraction for grouping together a set of concepts for non-tech people to make decisions with tech impact. The ideas presented here are a good summary of the day-to-day issues I experience as a "DevOps engineer." I wish I had the pull to instigate a shift in mindset towards a more platform-oriented mode of work, but unfortunately for me, that is not the case. Maybe it's wishful thinking, but forwarding articles and presentations like this to senior directors can be a way to influence the direction of their thinking to align more with what you believe to be in the best interest of the organization.

deleted 3y ago
[deleted]

matsemann 3y ago

This may be overly negative to a whole field, but I sometimes feek the platform teams add more hurdles than "stability and velocity". At places with basically no platform team, no advanced cloud setup etc, I as a dev could understand everything, and deploying mostly meant getting my jar file or whatever running on some ec2 instance. Now I need to write a dockerfile, loads of manifests, know about ingresses, use whatever custom tooling and abstractions they've built on top of the cloud provider. And what used to be a single guy is now a huge team just adding more complexity. And I'm having a hard time understanding if it's worth it, or just a cost sink.

reillyse 3y ago
Sounds a bit like those places aren’t at the scale where a platform team makes sense. What about when the one dev deploying their jar to an ec2 instance moves on to another company, how does the next dev even understand what this jar stuff is when they just want to push their SPA to vercel. Allowing devs to do whatever they want works at small new orgs but you need to put some kind of shape on it as they grow.
- adrianmsmith 3y ago
  > What about when the one dev deploying their jar to an ec2 instance moves on to another company, how does the next dev even understand what this jar stuff is I think that's more a problem of one person vs a team, as opposed to deploying JARs on EC2s vs cloud and a bunch of custom tooling. If you have a team of devs and they all deploy JARs to EC2, one of them leaving won't be a problem, the rest will still know how to do it. If you were to have a single platform engineer who's built a bunch of custom tooling over a bunch of Kubernetes files, and nobody knows how it works or where the files are, and then they leave, you've got the same problem as the solo EC2 dev leaving.
  - re-thc 3y ago
    > I think that's more a problem of one person vs a team Not always. The "team" you imagine might also be 1 person deploying the jar and the rest copying them. When that person leaves suddenly know 1 knows what to do anymore either.

anotherhue 3y ago

You'd be amazed how little platform engineering is needed if you can violently constrain technology choices. Give me a blobstore and some VMs and I'll be happy, and if someone raises the slightest fuss I will point out that it's still years better than helm.

hhh 3y ago
What’s wrong with helm?
- nforgerit 3y ago
  IMO lots of accidental complexity with no real value (in most use cases). I saw myself disagreeing frequently with pre-defined charts and creating an own chart just for one use case is better done with kustomize. It might make sense though for a bigger org with at least several dozens of engineers to ensure certain internal standards (in which platform engineering might make sense as well).
- pirates 3y ago
  i think helm is “fine”, but the more i use it and get deep into some more advanced uses the more i dislike it. helm has a tendency to sprawl really fast. the ability to template any part of a manifest can lead to “if there is even a chance the default will ever be changed we should make it a template” (at my company at least). when your entire chart is “if values.thing X, else Y”, it makes it difficult to be confident about what exactly will be deployed. especially if you only have a chart and values can be inserted in many places (helmfile). without helm template it’s sometimes impossible to reason with all the meta-templates. speaking of debugging, helm template sometimes has very awful debug output and traces of where your error is. in more simple charts it’s not so bad. with library charts you can spend hours debugging, say, your deployment.yaml because it’s saying there’s an error on line 4 of the template, but line 4 is just some label or benign key, and oh finally i figured out that the deployment expects the configmap to be rendered, and the typo is in line 4 of the CM, not the deployment helm lint doesn’t catch duplicate keys? i found that one weird tools like argocd don’t like helm too much. yeah you can make it work, but flat yaml is so much simpler. and if you use helm + argo, it turns the helm into flat yaml anyway i have heavy dislike of the syntax of ‘and values.thing1 values.thing2’ in a templated line. (same with ‘or’, though this might just be a personal hangup) finally, to be positive, the one thing i really like about helm actually is the checksum function that changes a label on the deploy/sts/ds when you change config, so pods automatically roll over.

BlackFly 3y ago

What does it mean to be greater than the sum of your parts? It is more than just alignment, everybody moving in the same direction: that would just be a well directed sum.Once upon a time, management talked about alignment because just being the sum of your parts would be great if it was all in the same direction. Now, for everybody obsessed with being a force multiplier, it needs to be more than just that. You should find yourself more capable in such an organization, but that is a tall order in today's world. Almost every problem you will face has been solved and help is easy to find even on your own. So being in such an organization should make you more capable of solving problems than your base ability. Someone must have already solved that problem for you and the solution must be in easy reach. That's still a tall order, most solutions you find described on the internet are also in easy reach nowadays. Your platform can also make people less capable. It is easier to be a force multiplier with a coefficient less than one. Make it more difficult to solve problems. Require them to communicate with many teams in order to accomplish anything. Make the solution difficult to integrate and operate and highly non-standard. If the solution is non-standard and the documentation is anything short of impeccable, it will basicly be unlearnable. The book Accelerate and the research backing it indicates that the most productive teams are able to redesign their entire system without talking to anyone and to complete their work without communication and coordination to other teams. Effective platforms advance that. They make it easy for people to redesign their systems to implement good designs. Bad platforms reduce that capability beyond what a person would ordinarily have and user complaints are brushed off with a tautological, "You can't make everyone happy." Bad platforms narrow possibilities and then require product teams to beg platform teams to allow ordinary designs that were cut off. I'm on a platform team and I struggle sometimes trying to figure out if we are helping people or hurting them hence this philosophical rant.

mkl95 3y ago
> What does it mean to be greater than the sum of your parts? In my experience it's mostly automation and observability. Delegating as much stuff as possible on tools as opposed to humans, empowering the latter to fix things.
- BlackFly 3y ago

deleted 3y ago

[deleted]

layer8 3y ago

After reading the article I still don’t know what “platform engineering” is meant to refer to.

KineticLensman 3y ago
> After reading the article I still don’t know what “platform engineering” is meant to refer to. It wasn't obvious to me either, so I found the following definitions. I still think that the article doesn't say 'how PE works' as opposed to providing a bunch of tips. "The engineering platform is created and maintained by a dedicated product team, designed to support the needs of software developers and others by providing common, reusable tools and capabilities, and interfacing to complex infrastructure." [0] "Platform engineering is the process of designing and implementing toolchains that improve the software delivery experience. Platform engineers set up automated infrastructure and self-service controls that allow developers to work more efficiently." [1] "Platform engineering is the “discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era. Platform engineers provide an integrated product most often referred to as an ‘Internal Developer Platform’ covering the operational necessities of the entire lifecycle of an application." [2] [0] https://www.gartner.com/en/articles/what-is-platform-engineering https://www.gartner.com/en/articles/what-is-platform-enginee... [1] https://www.howtogeek.com/devops/platform-engineering-vs-devops-what-you-need-to-know/ https://www.howtogeek.com/devops/platform-engineering-vs-dev... [2] https://humanitec.com/blog/sre-vs-devops-vs-platform-engineering https://humanitec.com/blog/sre-vs-devops-vs-platform-enginee...

dools 3y ago

[flagged]

techperson20007 3y ago

Ahh yes. The bi-monthly shit on ops comment threads. Ops folks can barely reverse a string. We should not exist as a field. Every company regardless of scale should just use 4 manually provisioned EC2 instances. If that doesn’t work your code is shit and you aren’t using Elixir / Phoenix! Anyone even attempting to use Kubernetes should be branded with the Hetzner logo and forced to work at McDonalds. Fear not HN. McDonalds here I come.

matsemann 3y ago
That's a lot of straw people.. There are lots of valid criticism, dismissing all of that is arrogant and counter productive.
- slyall 3y ago
  The largest thread here is people saying "deploying mostly meant getting my jar file or whatever running on some ec2 instance" and talking about manually setting up ec2 instances and each team somehow copying them into production. Just last week there was a thread about a company with 250 Petabytes in s3 and every comment from a developer was how it should be moved in-house. People were linking to one piece of s3-compatible software that the only example of a live deploy was single-digit gigabytes.
deterministic 3y ago
Mostly justified in my experience. In my experience, every time an “ops” team gets involved, complexity and cost goes up, with zero or negative benefits. Your experience might be different of course, but that doesn’t invalidate the negative experiences of others.

swyx 3y ago

> Platform Engineering is the application of a Product Mindset to supporting your engineering organization's software delivery velocity and system stability. so… move fast and don’t break things? must be nice to never have tradeoffs

deleted 3y ago

[deleted]

deterministic 3y ago

The whole point of “DevOps” was to not have a separate platform/deployment/operations team. Each development team should handle everything needed to build, test, deploy, and monitor production code. Every time I have been on a team where that was not the case, the result was bloated complexity and more failures in production. Usually because the platform/deployment/operations team was composed of not very skilled developers trying to justify their salary to management. If you have worked with a platform/deployment/operations team that was competent and reduced complexity then great for you. I have yet to experience such a phenomenon. Please let me know in comments why you think your experience was different.