4 ms·

We've gone full circle: 1. Pre-kub: orchestration is manual through AWS EC2 ui 2. Kub: orchestration is automated and configuration is treated as code (YAML,

We've gone full circle: 1. Pre-kub: orchestration is manual through AWS EC2 ui 2. Kub: orchestration is automated and configuration is treated as code (YAML, JSON, proto, whatever) and kept in revision control just like source code 3. Post-kub: orchestration is manual through a ui In all seriousness I'm sure this is useful in limited cases, but for most the right answer isn't some fancy UI, its middleware or some layer on top of kubernetes that allows you to automate away the toil-y parts of YAML, but keep the flexibility. EDIT: line breaks.

jchw 6y ago
What I like about Kubernetes is that it’s a platform you get things like exponential backoff, replication, liveness and health checking and so forth as part of the platform, and plugins for metric collection and monitoring are fairly standard. These are things I don’t even feel most PaaS have done a great job on across the board, so I can certainly see the appeal of having Kubernetes under the hood.
- stevepike 6y ago
  Yeah, there's a whole category of apps that run on heroku with maybe 10 dynos tops that could definitely be deployed on kubernetes with manual / web ui-based configuration.
- k__ 6y ago
  PaaS came before K8s, it's the minimum that K8s is better than PaaS.
  - jchw 6y ago
    I actually don’t see why a PaaS couldn’t do all of this better. Especially at larger companies, where containers and monitoring have been the norm internally forever, it is unclear why PaaS offerings still feel primitive... maybe there is some technology limitation here. Edit: disclaimer, I have not used a PaaS service in a couple years.
    - sneak 6y ago
      I think PaaS, autodeployed from a branch, is the best system for most orgs.
    - k__ 6y ago
      What I was getting at was, maybe K8s folks should look ahead (serverless) and not back (PaaS).
      - mplewis 6y ago
        Serverless is not a strict upgrade to containerized service deploys in any sense.
        k__ 6y ago
        Why not? It takes containers to their logical conclusion by generalizing them to a function while removing quite some operational overhead on the way.
- foxtr0t 6y ago
  Yeah that is understandable, k8s is a platform with a plethora of capabilities. On the other hand, the vast majority of organizations using k8s have multiple users that have multiple work streams and multiple applications (not true by definition, but close). This lends itself to a code as configuration approach, and makes managing orchestration through a UI risky and error prone. We review code, not clicks. Further, K8s exposes an YAML interface because it is relatively human readable and of course is machine readable. I'm sure the k8s community encourages users to build on top of this. Want to really do code as config? Write a nice DSL in your favorite scripting language to produce these YAMLs. Submit that as source. You've now automated away the vast majority of toil related to k8s YAML config, because your code can be: 1. modular by design 2. tested 3. tracked as code etc etc
  - palotasb 6y ago
    I've been experimenting on a personal project with https://www.pulumi.com/ https://www.pulumi.com/ where you can do the configuration and deployment in Python or JavasScript and I think I can recommend it.
    - verdverm 6y ago
      Do you have pointers on how to merge and manage Pulumi modules(?) when multiple languages are used?
    - throwayws 6y ago
      How does it compare to Terraform, which it is based on?
technics256 6y ago
IMHO I feel I get most of the way with operators and Kustomize.
throwaway894345 6y ago
> In all seriousness I'm sure this is useful in limited cases, but for most the right answer isn't some fancy UI, its middleware or some layer on top of kubernetes that allows you to automate away the toil-y parts of YAML, but keep the flexibility. I think Dhall is in the right category of solutions for this problem. YAML needs to be reusable and text templates a la Helm are an amusingly bad solution. Moreover, I think static typing is a very nice feature for things that generate YAML since it’s so easy to make mistakes with these kinds of tools. I’ve used Starlark for these kinds of tasks in the past and the lack of static typing has been a sore point.

jchw 6y ago

What I like about Kubernetes is that it’s a platform you get things like exponential backoff, replication, liveness and health checking and so forth as part of the platform, and plugins for metric collection and monitoring are fairly standard. These are things I don’t even feel most PaaS have done a great job on across the board, so I can certainly see the appeal of having Kubernetes under the hood.

stevepike 6y ago
Yeah, there's a whole category of apps that run on heroku with maybe 10 dynos tops that could definitely be deployed on kubernetes with manual / web ui-based configuration.
k__ 6y ago
PaaS came before K8s, it's the minimum that K8s is better than PaaS.
- jchw 6y ago
  I actually don’t see why a PaaS couldn’t do all of this better. Especially at larger companies, where containers and monitoring have been the norm internally forever, it is unclear why PaaS offerings still feel primitive... maybe there is some technology limitation here. Edit: disclaimer, I have not used a PaaS service in a couple years.
  - sneak 6y ago
    I think PaaS, autodeployed from a branch, is the best system for most orgs.
  - k__ 6y ago
    What I was getting at was, maybe K8s folks should look ahead (serverless) and not back (PaaS).
    - mplewis 6y ago
      Serverless is not a strict upgrade to containerized service deploys in any sense.
      - k__ 6y ago
        Why not? It takes containers to their logical conclusion by generalizing them to a function while removing quite some operational overhead on the way.
foxtr0t 6y ago
Yeah that is understandable, k8s is a platform with a plethora of capabilities. On the other hand, the vast majority of organizations using k8s have multiple users that have multiple work streams and multiple applications (not true by definition, but close). This lends itself to a code as configuration approach, and makes managing orchestration through a UI risky and error prone. We review code, not clicks. Further, K8s exposes an YAML interface because it is relatively human readable and of course is machine readable. I'm sure the k8s community encourages users to build on top of this. Want to really do code as config? Write a nice DSL in your favorite scripting language to produce these YAMLs. Submit that as source. You've now automated away the vast majority of toil related to k8s YAML config, because your code can be: 1. modular by design 2. tested 3. tracked as code etc etc
- palotasb 6y ago
  I've been experimenting on a personal project with https://www.pulumi.com/ https://www.pulumi.com/ where you can do the configuration and deployment in Python or JavasScript and I think I can recommend it.
  - verdverm 6y ago
    Do you have pointers on how to merge and manage Pulumi modules(?) when multiple languages are used?
  - throwayws 6y ago
    How does it compare to Terraform, which it is based on?

technics256 6y ago

IMHO I feel I get most of the way with operators and Kustomize.

throwaway894345 6y ago

> In all seriousness I'm sure this is useful in limited cases, but for most the right answer isn't some fancy UI, its middleware or some layer on top of kubernetes that allows you to automate away the toil-y parts of YAML, but keep the flexibility. I think Dhall is in the right category of solutions for this problem. YAML needs to be reusable and text templates a la Helm are an amusingly bad solution. Moreover, I think static typing is a very nice feature for things that generate YAML since it’s so easy to make mistakes with these kinds of tools. I’ve used Starlark for these kinds of tasks in the past and the lack of static typing has been a sore point.