3 ms·

Well docker based stuff seems to have a lot more of an ecosystem on the orchestration side of things. If I want to have a bunch of instances scaled up and wire

Well docker based stuff seems to have a lot more of an ecosystem on the orchestration side of things. If I want to have a bunch of instances scaled up and wired together network wise and defined in a nice yaml file, kubernetes makes this easy and replicable and I just have to push a container image somewhere like GHCR and have it all rolled out with CI. You can deploy almost the same system on top of the managed kubernetes offerings from any of the big cloud providers, so other than working out a few differences generally your config will work on anything. If I want to do this with unikernels I'm lost, would I do it with AWS services like EC2 and auto scaling groups or something? How do I get it all wired together and configured? Perhaps terraform but I'm not sure. It's all about the maturity of the glue and the architectures, not to mention training and familiarity. Does (mystery unikernels solution) have the equivalent of custom controllers somewhere in that stack? K8s provides an abstraction layer over resources and lets you define your system holistically. If this exists in a mature and flexible state for unikernels I'm all ears. What real advantage is there to using unikernels over alpine linux + k8s (or whatever) orchestration?

eyberg 3y ago
> If I want to do this with unikernels I'm lost, would I do it with AWS services like EC2 and auto scaling groups or something? This is a key point that many many people struggle with until they actually push their unikernels out to the cloud. It is one thing to run ops locally but try doing a 'ops image create -t aws' followed by 'ops instance create -t aws' and then you'll understand how we manage to push a lot of the scheduling work back onto the cloud provider itself.
- antihero 3y ago
  Yeah I've popped my image onto AWS and got it running in an instance, I figure the next port of call is the terraform part of the docs to see how you'd put something out in production. It would be cool to write some sort of book like, ops for k8s people, where it takes a lot of the common concepts in k8s and explains how you'd do them with this system. ops seems like decent a CLI to get things done but as far as I can see not really an orchestration system in the same way as k8s? Perhaps I misunderstood.
  - eyberg 3y ago
    You are correct that ops does not provide any sort of orchestration framework today, but it's also not necessarily needed in a lot of cases because the end artifacts are vms not containers. There is a terraform provider that we ended up writing for one company but only because they were a (large) hashicorp shop and nothing went to prod unless it was in tf: https://docs.ops.city/ops/terraform https://docs.ops.city/ops/terraform This doesn't really do much though. It's only there to check a box and fit into an existing paradigm. A lot of the orchestration bits that k8s provides is because when it was initially adopted there wasn't a great way to attach networks/storage/etc. to various containers like people had for on the clouds with vms. OPS just re-uses the same primitives you'l find on every single cloud out there. If you need more advanced functionality like auto-scaling that also can utilize the clouds ASGs.
    - antihero 3y ago
      Yeah I guess a lot of the orchestration can be effectively handled by terraform, I get what you mean regarding cloud primitives. This is my next step to seeing how this can work in a production environment. Instead of an ingress you can use ELB, for instance. I guess this means your environment is very "native" to a specific cloud however and not as easily transferrable. One thing I did like about k8s is that it's easy to self-host with microk8s but tbh, I think in production you'd be using a cloud provider's managed k8s (e.g. AKS) anyway. Looking at how to attach/configure a ElastiCache, RDS, or some EFS to a project also. What's the best way to runtime configure an image when you create it as an instance? I couldn't see any options on ops instance create for environment variables or anything like that but this also seems like, weird - IE you wouldn't give a VM environment variables, but with a normal EC2 instance there's an agent that handles this sort of thing. Would you have to bake the configuration into the image? Also how does one output the ELF for the image? Is it just the file of [image name] in ~/.ops/images ? Or is it the case that terraform should build a correctly pre-configured image as part of the apply process?w Always good to have other options, of course! I do appreciate the minimalism. This is exciting :) The terraform docs example uses GCP, perhaps if I figure out how to do it on AWS I could PR the docs with an example. At that stage where my questions lead to more questions.

eyberg 3y ago

> If I want to do this with unikernels I'm lost, would I do it with AWS services like EC2 and auto scaling groups or something? This is a key point that many many people struggle with until they actually push their unikernels out to the cloud. It is one thing to run ops locally but try doing a 'ops image create -t aws' followed by 'ops instance create -t aws' and then you'll understand how we manage to push a lot of the scheduling work back onto the cloud provider itself.

antihero 3y ago
Yeah I've popped my image onto AWS and got it running in an instance, I figure the next port of call is the terraform part of the docs to see how you'd put something out in production. It would be cool to write some sort of book like, ops for k8s people, where it takes a lot of the common concepts in k8s and explains how you'd do them with this system. ops seems like decent a CLI to get things done but as far as I can see not really an orchestration system in the same way as k8s? Perhaps I misunderstood.
- eyberg 3y ago
  You are correct that ops does not provide any sort of orchestration framework today, but it's also not necessarily needed in a lot of cases because the end artifacts are vms not containers. There is a terraform provider that we ended up writing for one company but only because they were a (large) hashicorp shop and nothing went to prod unless it was in tf: https://docs.ops.city/ops/terraform https://docs.ops.city/ops/terraform This doesn't really do much though. It's only there to check a box and fit into an existing paradigm. A lot of the orchestration bits that k8s provides is because when it was initially adopted there wasn't a great way to attach networks/storage/etc. to various containers like people had for on the clouds with vms. OPS just re-uses the same primitives you'l find on every single cloud out there. If you need more advanced functionality like auto-scaling that also can utilize the clouds ASGs.
  - antihero 3y ago
    Yeah I guess a lot of the orchestration can be effectively handled by terraform, I get what you mean regarding cloud primitives. This is my next step to seeing how this can work in a production environment. Instead of an ingress you can use ELB, for instance. I guess this means your environment is very "native" to a specific cloud however and not as easily transferrable. One thing I did like about k8s is that it's easy to self-host with microk8s but tbh, I think in production you'd be using a cloud provider's managed k8s (e.g. AKS) anyway. Looking at how to attach/configure a ElastiCache, RDS, or some EFS to a project also. What's the best way to runtime configure an image when you create it as an instance? I couldn't see any options on ops instance create for environment variables or anything like that but this also seems like, weird - IE you wouldn't give a VM environment variables, but with a normal EC2 instance there's an agent that handles this sort of thing. Would you have to bake the configuration into the image? Also how does one output the ELF for the image? Is it just the file of [image name] in ~/.ops/images ? Or is it the case that terraform should build a correctly pre-configured image as part of the apply process?w Always good to have other options, of course! I do appreciate the minimalism. This is exciting :) The terraform docs example uses GCP, perhaps if I figure out how to do it on AWS I could PR the docs with an example. At that stage where my questions lead to more questions.