5 ms·

Serious question: Is there an emerging cross platform workflow language to just write stuff to run on any cloud/container hosting setup? The idea would be to b

Serious question: Is there an emerging cross platform workflow language to just write stuff to run on any cloud/container hosting setup? The idea would be to be portable, avoid vendor lock-in and take advantage price differences or quickly route around a system failure in one of the providers.

JacobHenner 8y ago
https://www.terraform.io/ https://www.terraform.io/
- hinkley 8y ago
  Is there a good way to sandbox terraform configurations? I'm not directly involved (just hear the screaming) but everything I'm hearing is that making modifications is a test of willpower. For us it's been about as transparent as a brick wall and I'm not clear if that's down to our bureaucracy or built into the design. Both are anathema to the goal of making complex deployments straightforward and self-describing (you can't manage something this complicated unless big parts of it are as obvious as can be).
  - matwood 8y ago
    The recommended way, at least for AWS, is to have multiple accounts. One for production, and then however many more for test and development. Separate accounts let you run TF changes and know you will not impact production. TF can be tricky to grok at first especially if you don't have everything in TF. But, I couldn't imagine managing more than a server or 2 without it or something similar at this point. Once you get into VPCs, IAMs, etc..., some type of tool is really required. I'm also a little confused about your transparency comment. IME, tf is very clear what it is going to do in a plan. The current state files are also just json, and easy to read/search if you're not sure about something.
    - hinkley 8y ago
      Declarative syntax is notoriously hard to debug, especially for newbies. As a general rule, if you're giving someone a tool that uses declarative syntax, you also need to provide them a private (not shared) sandbox in which to test out theories, try new things, and reproduce errors seen in production. Since we don't have that, TF is pretty much the worst solution for our problems. Kube or even Docker Swarm would serve us much better.
- driverdan 8y ago
  Terraform requires separate config for each provider.
  - djsumdog 8y ago
    There's no way around that really. I wrote my own provisioner and each provider is so different: https://github.com/sumdog/bee2 https://github.com/sumdog/bee2 Vultr allows floating IPs for IPv4 and IPv6, but Digital Ocean only has floating IPv4. Vultr will start a machine with a floating v4, but you have to add a floating v6 address (giving you two v6 addresses). Digital Ocean does the same thing with v4 (giving you two v4 addresses). They both have different network adapter names, so you've got to configure those per provider as well. Terraform and my own thing help in easing the transition if you ever need to move, but modifications will still have to be made.
alexeldeib 8y ago
Some newer candidates besides Terraform are Portable Service Definitions and Crossplane. https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0032-portable-service-definitions.md https://github.com/kubernetes/enhancements/blob/master/keps/... https://crossplane.io https://crossplane.io
- djsumdog 8y ago
  Huh .. It only supports the big three providers. No Digital Ocean or Vultr.
segmondy 8y ago
Yes, kubernetes
- mindfulmonkey 8y ago
  yeah, but your configs aren't fully portable between cloud providers when you look at things like LBs, storage, etc..
  - movedx 8y ago
    I thought load balancers popped out the end of Services and it was plugins that handled the specific cloud environment? I'd say that still constitutes cross platform.
  - erulabs 8y ago
    Yes and No. For example, "type: LoadBalancer" works fine on almost every cloud, but various annotations need to be added for SSL termination on an AWS ALB, for example. The annotations don't collide tho, so you can have a load balancer with both AWS and Google Cloud annotations, and it will work fine on either cloud. Volume classes are probably the best example of being cloud-specific, but this problem is solved by having a different volume class for each cloud provider, named the same, such that the deployment can always grab a disk regardless of which cloud its living in.
  - praseodym 8y ago
    They are. Kubernetes has abstractions at exactly the right layers (e.g. Service to create a load balancer) so that you can exchange configs between cloud providers. There can of course be some difference in the capabilities that each cloud provider supports (e.g. not all load balancer implementations may support UDP) but the abstraction is definitely there.
autechr3 8y ago
https://cnab.io/ https://cnab.io/
mark-ruwt 8y ago
We rely on bash. Each machine that's spun up is built from scratch via one command-line call. The first half of the process interacts with each hosting API (we rely on DigitalOcean, Linode, and Vultr primarily), to build a clean slate machine with all of the packages and libraries that we expect. The second half of the process runs the actual build process, building the instance step-by-step on top of the clean slate, blissfully unaware of which hosting provider it lives on. This model allows us to be portable and avoid vendor-lock in, and a cross-provider infrastructure lets us gracefully handle system failures while keeping costs down.
- _kyran 8y ago
  Any chance you've got an open source version of this script that you could share?
  - mark-ruwt 8y ago
    No, unfortunately, open-sourcing it has been on my to-do list for an embarrassingly long time. But, building one is easier than it sounds! Think of the problem in two parts. First, find a distro used by multiple providers (we're on CentOS) and craft one script that uses each provider's API to spin up a clean machine. Once you have that done, it's a matter of understanding your own build process, writing a script that you'll pass into each instance on creation that will fetch your source control, install libraries, and put all the pieces together. Lots of if/else, lots of curl, lots of yum, lots of jq, but all of it is really straightforward.
    - lotyrin 8y ago
      Also if your providers + OS support cloud-init, then you can express a fleet of instances which run this sort of script at boot time in something like Terraform pretty easily. Switching clouds becomes "uh... what does <provider> call their <size> instance again?" Alternatively, pre-baking cloud images that have already run such a script and are ready to boot becomes pretty easy with a tool like Packer. Though, as the underlying OS changes, you'll need something to validate your scripts' functionality against, and a tool that's a little more declarative might make them less fragile to those changes.
fantispug 8y ago
Apache libcloud works well for more basic services (e.g. storand and compute) and is extending into container management services.
juddlyon 8y ago
Terraform/Packer/Ansible is a popular combo.

JacobHenner 8y ago

https://www.terraform.io/ https://www.terraform.io/

hinkley 8y ago
Is there a good way to sandbox terraform configurations? I'm not directly involved (just hear the screaming) but everything I'm hearing is that making modifications is a test of willpower. For us it's been about as transparent as a brick wall and I'm not clear if that's down to our bureaucracy or built into the design. Both are anathema to the goal of making complex deployments straightforward and self-describing (you can't manage something this complicated unless big parts of it are as obvious as can be).
- matwood 8y ago
  The recommended way, at least for AWS, is to have multiple accounts. One for production, and then however many more for test and development. Separate accounts let you run TF changes and know you will not impact production. TF can be tricky to grok at first especially if you don't have everything in TF. But, I couldn't imagine managing more than a server or 2 without it or something similar at this point. Once you get into VPCs, IAMs, etc..., some type of tool is really required. I'm also a little confused about your transparency comment. IME, tf is very clear what it is going to do in a plan. The current state files are also just json, and easy to read/search if you're not sure about something.
  - hinkley 8y ago
    Declarative syntax is notoriously hard to debug, especially for newbies. As a general rule, if you're giving someone a tool that uses declarative syntax, you also need to provide them a private (not shared) sandbox in which to test out theories, try new things, and reproduce errors seen in production. Since we don't have that, TF is pretty much the worst solution for our problems. Kube or even Docker Swarm would serve us much better.
driverdan 8y ago
Terraform requires separate config for each provider.
- djsumdog 8y ago
  There's no way around that really. I wrote my own provisioner and each provider is so different: https://github.com/sumdog/bee2 https://github.com/sumdog/bee2 Vultr allows floating IPs for IPv4 and IPv6, but Digital Ocean only has floating IPv4. Vultr will start a machine with a floating v4, but you have to add a floating v6 address (giving you two v6 addresses). Digital Ocean does the same thing with v4 (giving you two v4 addresses). They both have different network adapter names, so you've got to configure those per provider as well. Terraform and my own thing help in easing the transition if you ever need to move, but modifications will still have to be made.

alexeldeib 8y ago

Some newer candidates besides Terraform are Portable Service Definitions and Crossplane. https://github.com/kubernetes/enhancements/blob/master/keps/sig-apps/0032-portable-service-definitions.md https://github.com/kubernetes/enhancements/blob/master/keps/... https://crossplane.io https://crossplane.io

djsumdog 8y ago
Huh .. It only supports the big three providers. No Digital Ocean or Vultr.

segmondy 8y ago

Yes, kubernetes

mindfulmonkey 8y ago
yeah, but your configs aren't fully portable between cloud providers when you look at things like LBs, storage, etc..
- movedx 8y ago
  I thought load balancers popped out the end of Services and it was plugins that handled the specific cloud environment? I'd say that still constitutes cross platform.
- erulabs 8y ago
  Yes and No. For example, "type: LoadBalancer" works fine on almost every cloud, but various annotations need to be added for SSL termination on an AWS ALB, for example. The annotations don't collide tho, so you can have a load balancer with both AWS and Google Cloud annotations, and it will work fine on either cloud. Volume classes are probably the best example of being cloud-specific, but this problem is solved by having a different volume class for each cloud provider, named the same, such that the deployment can always grab a disk regardless of which cloud its living in.
- praseodym 8y ago
  They are. Kubernetes has abstractions at exactly the right layers (e.g. Service to create a load balancer) so that you can exchange configs between cloud providers. There can of course be some difference in the capabilities that each cloud provider supports (e.g. not all load balancer implementations may support UDP) but the abstraction is definitely there.

autechr3 8y ago

https://cnab.io/ https://cnab.io/

mark-ruwt 8y ago

We rely on bash. Each machine that's spun up is built from scratch via one command-line call. The first half of the process interacts with each hosting API (we rely on DigitalOcean, Linode, and Vultr primarily), to build a clean slate machine with all of the packages and libraries that we expect. The second half of the process runs the actual build process, building the instance step-by-step on top of the clean slate, blissfully unaware of which hosting provider it lives on. This model allows us to be portable and avoid vendor-lock in, and a cross-provider infrastructure lets us gracefully handle system failures while keeping costs down.

_kyran 8y ago
Any chance you've got an open source version of this script that you could share?
- mark-ruwt 8y ago
  No, unfortunately, open-sourcing it has been on my to-do list for an embarrassingly long time. But, building one is easier than it sounds! Think of the problem in two parts. First, find a distro used by multiple providers (we're on CentOS) and craft one script that uses each provider's API to spin up a clean machine. Once you have that done, it's a matter of understanding your own build process, writing a script that you'll pass into each instance on creation that will fetch your source control, install libraries, and put all the pieces together. Lots of if/else, lots of curl, lots of yum, lots of jq, but all of it is really straightforward.
  - lotyrin 8y ago
    Also if your providers + OS support cloud-init, then you can express a fleet of instances which run this sort of script at boot time in something like Terraform pretty easily. Switching clouds becomes "uh... what does <provider> call their <size> instance again?" Alternatively, pre-baking cloud images that have already run such a script and are ready to boot becomes pretty easy with a tool like Packer. Though, as the underlying OS changes, you'll need something to validate your scripts' functionality against, and a tool that's a little more declarative might make them less fragile to those changes.

fantispug 8y ago

Apache libcloud works well for more basic services (e.g. storand and compute) and is extending into container management services.

juddlyon 8y ago

Terraform/Packer/Ansible is a popular combo.