4 ms·

We rely on bash. Each machine that's spun up is built from scratch via one command-line call. The first half of the process interacts with each hosting API (w

We rely on bash. Each machine that's spun up is built from scratch via one command-line call. The first half of the process interacts with each hosting API (we rely on DigitalOcean, Linode, and Vultr primarily), to build a clean slate machine with all of the packages and libraries that we expect. The second half of the process runs the actual build process, building the instance step-by-step on top of the clean slate, blissfully unaware of which hosting provider it lives on. This model allows us to be portable and avoid vendor-lock in, and a cross-provider infrastructure lets us gracefully handle system failures while keeping costs down.

_kyran 8y ago
Any chance you've got an open source version of this script that you could share?
- mark-ruwt 8y ago
  No, unfortunately, open-sourcing it has been on my to-do list for an embarrassingly long time. But, building one is easier than it sounds! Think of the problem in two parts. First, find a distro used by multiple providers (we're on CentOS) and craft one script that uses each provider's API to spin up a clean machine. Once you have that done, it's a matter of understanding your own build process, writing a script that you'll pass into each instance on creation that will fetch your source control, install libraries, and put all the pieces together. Lots of if/else, lots of curl, lots of yum, lots of jq, but all of it is really straightforward.
  - lotyrin 8y ago
    Also if your providers + OS support cloud-init, then you can express a fleet of instances which run this sort of script at boot time in something like Terraform pretty easily. Switching clouds becomes "uh... what does <provider> call their <size> instance again?" Alternatively, pre-baking cloud images that have already run such a script and are ready to boot becomes pretty easy with a tool like Packer. Though, as the underlying OS changes, you'll need something to validate your scripts' functionality against, and a tool that's a little more declarative might make them less fragile to those changes.
    - mark-ruwt 8y ago
      I started using this model in '15 and it's not fragile. At all. The less I'm relying on outside frameworks, the better I sleep at night.
      - lotyrin 8y ago
        Certainly understandable -- I'd prefer to keep things simple and just have some kind of validation in place rather than rely on an abstraction if I can get away with it. Having maintained various automation over the course of the past decade and a half, I can say things do change around. Over the course of only a few years though, obviously you can stick to some LTS release of whatever you're using and be pretty confident that e.g. "some-package" does not get renamed to "some-package-version" or split into "some-core" and "some-utils", or have a package get upgraded to a version with some less-than-backward-compatible configuration options, etc.
  - lugg 8y ago
    Is this a cookie cutter stack or do you have different models? Be curious to hear more about your application stack.
    - mark-ruwt 8y ago
      There's nothing special about our stack. We have four different instance types (static, api, db, proxy), and we rely on a lot of the usual suspects: Apache, Tomcat, MySQL, Varnish, and HAProxy.
wise_young_man 8y ago
I made something similar and turned it into a service [1] focused on WordPress, but unfortunately there hasn't been much interest from people as I thought there would be, though that could be due to my lack of marketing. My goal was the same, to make hosting more portable with features like snapshotting and restoration of WP sites across servers and to even eventually expand beyond just servers, to bring in domain registrars and cloud storage to be able to move things around easier. For example: you have a site hosted on AWS EC2 with DNS at Namecheap and nightly backups at Dropbox and let's say the AWS Virginia region goes down. You create a new server in Digital Ocean and restore the snapshot from Dropbox and the linked DNS at Namecheap is auto updated. The more I thought about this though, I began to realize that maybe these features wouldn't be useful to the audience I wanted to target, which was people who wanted to grow from shared hosting and have something reliable and less noisy neighbors, but still more affordable than managed WP hosts and lastly more control (bring your cloud/server provider). [1]: http://pagefog.com http://pagefog.com
- kkarakk 8y ago
  some unsolicited feedback: your name is terrible(unrelated to your product in any way) and your website doesn't communicate the problem you say you're solving all i get from your sites landing page is "wordpress hosting" which is not exactly uncommon scrolling to the bottom shows me some cloud providers. makes me think you just help people host wordpress in the cloud
MuffinFlavored 8y ago
Why/what benefit?
suls 8y ago
Wondering how what you did compares to cloudinit? https://cloudinit.readthedocs.io/en/latest/ https://cloudinit.readthedocs.io/en/latest/

_kyran 8y ago

Any chance you've got an open source version of this script that you could share?

mark-ruwt 8y ago
No, unfortunately, open-sourcing it has been on my to-do list for an embarrassingly long time. But, building one is easier than it sounds! Think of the problem in two parts. First, find a distro used by multiple providers (we're on CentOS) and craft one script that uses each provider's API to spin up a clean machine. Once you have that done, it's a matter of understanding your own build process, writing a script that you'll pass into each instance on creation that will fetch your source control, install libraries, and put all the pieces together. Lots of if/else, lots of curl, lots of yum, lots of jq, but all of it is really straightforward.
- lotyrin 8y ago
  Also if your providers + OS support cloud-init, then you can express a fleet of instances which run this sort of script at boot time in something like Terraform pretty easily. Switching clouds becomes "uh... what does <provider> call their <size> instance again?" Alternatively, pre-baking cloud images that have already run such a script and are ready to boot becomes pretty easy with a tool like Packer. Though, as the underlying OS changes, you'll need something to validate your scripts' functionality against, and a tool that's a little more declarative might make them less fragile to those changes.
  - mark-ruwt 8y ago
    I started using this model in '15 and it's not fragile. At all. The less I'm relying on outside frameworks, the better I sleep at night.
    - lotyrin 8y ago
      Certainly understandable -- I'd prefer to keep things simple and just have some kind of validation in place rather than rely on an abstraction if I can get away with it. Having maintained various automation over the course of the past decade and a half, I can say things do change around. Over the course of only a few years though, obviously you can stick to some LTS release of whatever you're using and be pretty confident that e.g. "some-package" does not get renamed to "some-package-version" or split into "some-core" and "some-utils", or have a package get upgraded to a version with some less-than-backward-compatible configuration options, etc.
- lugg 8y ago
  Is this a cookie cutter stack or do you have different models? Be curious to hear more about your application stack.
  - mark-ruwt 8y ago
    There's nothing special about our stack. We have four different instance types (static, api, db, proxy), and we rely on a lot of the usual suspects: Apache, Tomcat, MySQL, Varnish, and HAProxy.

wise_young_man 8y ago

I made something similar and turned it into a service [1] focused on WordPress, but unfortunately there hasn't been much interest from people as I thought there would be, though that could be due to my lack of marketing. My goal was the same, to make hosting more portable with features like snapshotting and restoration of WP sites across servers and to even eventually expand beyond just servers, to bring in domain registrars and cloud storage to be able to move things around easier. For example: you have a site hosted on AWS EC2 with DNS at Namecheap and nightly backups at Dropbox and let's say the AWS Virginia region goes down. You create a new server in Digital Ocean and restore the snapshot from Dropbox and the linked DNS at Namecheap is auto updated. The more I thought about this though, I began to realize that maybe these features wouldn't be useful to the audience I wanted to target, which was people who wanted to grow from shared hosting and have something reliable and less noisy neighbors, but still more affordable than managed WP hosts and lastly more control (bring your cloud/server provider). [1]: http://pagefog.com http://pagefog.com

kkarakk 8y ago
some unsolicited feedback: your name is terrible(unrelated to your product in any way) and your website doesn't communicate the problem you say you're solving all i get from your sites landing page is "wordpress hosting" which is not exactly uncommon scrolling to the bottom shows me some cloud providers. makes me think you just help people host wordpress in the cloud

MuffinFlavored 8y ago

Why/what benefit?

suls 8y ago

Wondering how what you did compares to cloudinit? https://cloudinit.readthedocs.io/en/latest/ https://cloudinit.readthedocs.io/en/latest/