3 ms·

For a upcoming project we're using AWS Marketplace + Cloudformation in combination with the community edition of Chef Habitat to try to solve this problem. Once

by gfodor 7y ago

For a upcoming project we're using AWS Marketplace + Cloudformation in combination with the community edition of Chef Habitat to try to solve this problem. Once someone deploys the cloudformation template, it sets up: - Email (SES) - SSL certificates + renewal (ACM + lets encrypt) - Backup + Restore (RDS snapshots + AWS Backup for EFS) - ALB (if desired) - CDN (Cloudfront) - Firewall - Auto scaling database (RDS pgsql serverless) with automatic pausing - Auto scaling storage (EFS) You can adjust capacity by just dialing up or down the ASG, and our Elixir app auto-clusters using the Habitat ring for service discovery. Packages are upgraded when new versions are pushed to our package repository. All binaries run in jailed process environments scoped to Habitat packages, with configuration management and supervision handled by Habitat. This is a non-container approach, focusing on VMs. However, the theory is that this will be pretty much turnkey for a scalable self-hosted product on AWS, including software updates. It's hard to say how well the theory will fall out in practice, but I'm optimistic. Avoiding a mess of microservices was fairly important in making this kind of thing possible, we have a few services, with a dominant monolith in Elixir.

mwcampbell 7y ago
This is the first I've heard about Chef Habitat in a few years. I'm curious about why you chose it. Also, how do you configure new VMs? Do you have a custom machine image, or do you use a stock image and have a startup script (via EC2 user data) that sets it up? I've spent a little time looking at Habitat docs, and I find it unfortunate that they're trying so hard to jump on the container bandwagon when apparently it also works well on bare metal or VMs.
- gfodor 7y ago
  Our project was greenfield 2.5 years ago, and in evaluating the ecosystem we ran into the work going on w Habitat after trying to figure out if there was a VM-analog to the stuff going on with k8s and containers (which was a very scary prospect to dive into.) Habitat seems to be a good tech stack for solving a variety of operational problems that you typically pull different products off the shelf and duct tape together -- particularly if you are focused on VMs. It was a bit of a bumpy ride but the product has stabilized enough we have confidence in it and it turns out to be quite a good fit for the self hosting use-case. Basically, it's an all-in-one solution for packaging, isolation, service discovery, configuration management, and deploys. So it was relatively easy to transition our production deployment bits to a packaged up deployed solution to 3rd party servers, with all of the bells and whistles. Our setup is a fairly minimal custom AMI that just installs the basic package dependencies for Habitat and configures the supervisor. Everything else is bootstrapped within Habitat, including our own proprietary configuration management service which allows a nice web-based GUI for configuring the ring state. The user data script just does some basic UNIX setup and DNS initialization and then does a bunch of Habitat service loads. By abstracting over all configuration via Habitat, a unified interface exists for configuring all the services across all the machines, regardless of their underlying configuration management approach, programming language, conventions, etc.

by gfodor 7y ago

mwcampbell 7y ago

This is the first I've heard about Chef Habitat in a few years. I'm curious about why you chose it. Also, how do you configure new VMs? Do you have a custom machine image, or do you use a stock image and have a startup script (via EC2 user data) that sets it up? I've spent a little time looking at Habitat docs, and I find it unfortunate that they're trying so hard to jump on the container bandwagon when apparently it also works well on bare metal or VMs.

gfodor 7y ago
Our project was greenfield 2.5 years ago, and in evaluating the ecosystem we ran into the work going on w Habitat after trying to figure out if there was a VM-analog to the stuff going on with k8s and containers (which was a very scary prospect to dive into.) Habitat seems to be a good tech stack for solving a variety of operational problems that you typically pull different products off the shelf and duct tape together -- particularly if you are focused on VMs. It was a bit of a bumpy ride but the product has stabilized enough we have confidence in it and it turns out to be quite a good fit for the self hosting use-case. Basically, it's an all-in-one solution for packaging, isolation, service discovery, configuration management, and deploys. So it was relatively easy to transition our production deployment bits to a packaged up deployed solution to 3rd party servers, with all of the bells and whistles. Our setup is a fairly minimal custom AMI that just installs the basic package dependencies for Habitat and configures the supervisor. Everything else is bootstrapped within Habitat, including our own proprietary configuration management service which allows a nice web-based GUI for configuring the ring state. The user data script just does some basic UNIX setup and DNS initialization and then does a bunch of Habitat service loads. By abstracting over all configuration via Habitat, a unified interface exists for configuring all the services across all the machines, regardless of their underlying configuration management approach, programming language, conventions, etc.