5 ms·

Hey HN, We're Lyn & Colin and we built Layer - It creates a unique staging server for every commit. Before Layer, I was CTO of a startup with a 10-person deve

Hey HN, We're Lyn & Colin and we built Layer - It creates a unique staging server for every commit. Before Layer, I was CTO of a startup with a 10-person developer team, and dealing with staging servers (and end-to-end tests) was one of the most annoying parts of our workflow. Layer lets you define staging servers the same way you define Docker containers, and get a unique one for every commit going back a week. We run them in the cloud, so there's no need to pay for AWS servers you're not using. Also, if you're running a webserver, you get a unique URL for each unique server automatically so that you can post it to a slack channel and get feedback immediately. When we're setting up the staging server, we use the same sort of cache as Docker so that you can skip repetitive tasks like setting up a database or running database migrations if they haven't changed. Would love to get your feedback, it only takes five minutes to do the onboarding tutorial.

Lyn_layerci 6y ago
Hey HN! I'm Lyn, co-founder of Layer. Here's a quick 5 min demo video in case you don't want to do the sign up tutorial (even though it takes 5 mins toooo): https://www.youtube.com/watch?v=PH-n70gPQgA https://www.youtube.com/watch?v=PH-n70gPQgA Happy to answer any questions here.
lasdfas 6y ago
How is this even possible for $35? 50 staging servers per seat ($1450 Value) 11GB RAM per server 8 CPUs per server
- colinchartier 6y ago
  We spent six months developing a hypervisor that lets us treat any linux VM as a lambda - We pass on the efficiency to our users as a value add.
  - lasdfas 6y ago
    You should probably be selling that if it works well. That is something I have not seen.
    - deadbunny 6y ago
      <mostly sarcastic> Bakevm with tooling > create userdata script for cloud init to copy payload to server > create systemd oneshot service to run payload && shutdown
      - colinchartier 6y ago
        You should try it! Our baking interface looks more like Docker than a traditional VM provision (ala Packer) Also, we keep the baked images around for a week and trigger them whenever they are requested (with, say, HTTP, or terminal access)
      - lasdfas 6y ago
        I may be misunderstanding the product, but it seems to be able to run a webserver on each "server".
  - lykr0n 6y ago
    More info on that would be cool. Something like Amazon's FireCracker?
    - colinchartier 6y ago
      Sort of - FireCracker is less suited for testing use-cases because it still cold-boots. We use the same kernel features that let laptops resume when the lid is opened instead, they're less mature for production but much faster for staging and testing.
      - lykr0n 6y ago
        Ah. VMware does something similar for hot provisioning VMs from a point in time memory snapshot. So you guys boot the image in the background, then capture a memory dump of it so you can quickly launch VMs from that snapshot? Or you boot the VMs the traditional way the first time, and then just suspend them when not in use?
        colinchartier 6y ago
        We do even more, we take snapshots as the "Layerfile" runs - this basically lets you merge the memory dump semantics you mention with the interface of Docker. Also, we monitor which files are read by any step in the process, and let you skip setup chunks that don't need to run (i.e., installing libraries) without needing to micromanage what files you copy into the staging server.
        lykr0n 6y ago
        Man that sounds cool. I don't have any use for something like that, but I would love to see it opensourced to play with it (which I assume you guys won't be doing anytime soon)
      - TechBro8615 6y ago
        Are you using socket activation or something similar? I think there are really interesting use cases around that paradigm for packing services onto one host. Some blog posts I remember from when I was exploring this idea a few years ago: http://0pointer.de/blog/projects/socket-activated-containers.html http://0pointer.de/blog/projects/socket-activated-containers... https://www.ctl.io/developers/blog/post/running-drupal-in-linux-containers-but-not-docker-on-pantheon https://www.ctl.io/developers/blog/post/running-drupal-in-li...
  - mike_d 6y ago
    Why are you trying to tackle some small little market of staging servers for the handful of companies that actually care about this? Take your Hypervisor and sell it to companies that want to efficiently lift-and-shift legacy systems in to the cloud and make your investors super happy.
- deleted 6y ago
  [deleted]
defen 6y ago
How would this handle external state? I realize in general that's an impossible task, but specifically I'm thinking of something like a database. Is there a way for each "server" to get a private copy of the database? Otherwise it seems like if I create a non-backwards-compatible migration (let's say I drop a table that's no longer necessary) I could break servers that were spawned for other commits.
- colinchartier 6y ago
  Glad you asked! If you do the Django tutorial after clicking "try now", you can see an example of a postgres database that gets set up for every run. Since we give full VMs instead of just containers, you can keep the database running in the background. Because we do memory snapshotting, you could turn a 5 minute database start + migrate + populate into a 1 second "restore from memory snapshot"
  - ignoramous 6y ago
    > Since we give full VMs instead of just containers, you can keep the database running in the background. Interesting contrast to flockport that runs multiple processes in a LXC [0] (treats it like a lightweight VM). [0] https://news.ycombinator.com/item?id=17478736 https://news.ycombinator.com/item?id=17478736
lazyant 6y ago
First of all, this looks awesome and go Canada! Random comments: Is there a way to try without giving access to my github account? Less importantly: Can you put your email in your HN profile? Can you add LayerCI Twitter handle to your site?
- colinchartier 6y ago
  1. Not yet unfortunately, we didn't want to waste time creating a separate auth system when we only integrate with GitHub for now anyways. To see a short demo though, here's an earlier one we made: https://www.youtube.com/watch?v=PH-n70gPQgA https://www.youtube.com/watch?v=PH-n70gPQgA 2. My email (for Layer) is colin@layerci.com 3. Done! Thanks for your feedback.
gravypod 6y ago
How do you handle very complex and integrated architectures like a monorepo with many microservices? How do you run your layer file locally? Can you describe automated tests in a layer file? I built something similar[0] to this for integration testing and those were my biggest hurdles. At a previous job where all of the code was python/node/java I was also able to generate cross-service end to end code coverage from these tests and that feature was extremely helpful as it helps you remove redundant tests. Your service looks very polished and I'm extremely impressed someone is taking this idea to market. 0- bunt.build
- colinchartier 6y ago
  > monorepo with many microservices We scan the entire repository for Layerfiles, so you could, say, create a layerfile per service for unit tests, and a bigger Layerfile in the repo root that does the whole deployment and runs a few E2E tests. > How do you run your layer file locally? They can be approximated by running a docker container and just copy/pasting the commands. We haven't built this yet because I know it will be a huge time sink to make it reliable (see, e.g., vagrant) > Can you describe automated tests in a layer file? We tell you if any step fails (e.g., returns non-zero exit code) so you can just as easily run "./deploy.sh" as "./test.sh" - you can also start a webserver and run E2E or acceptance tests against it, for example. > generate cross-service end to end code coverage We are trying to avoid anything that isn't very general - it's possible to run https://codecov.io/ https://codecov.io/ within Layer if you want functionality like this.

Lyn_layerci 6y ago

Hey HN! I'm Lyn, co-founder of Layer. Here's a quick 5 min demo video in case you don't want to do the sign up tutorial (even though it takes 5 mins toooo): https://www.youtube.com/watch?v=PH-n70gPQgA https://www.youtube.com/watch?v=PH-n70gPQgA Happy to answer any questions here.

lasdfas 6y ago

How is this even possible for $35? 50 staging servers per seat ($1450 Value) 11GB RAM per server 8 CPUs per server

colinchartier 6y ago
We spent six months developing a hypervisor that lets us treat any linux VM as a lambda - We pass on the efficiency to our users as a value add.
- lasdfas 6y ago
  You should probably be selling that if it works well. That is something I have not seen.
  - deadbunny 6y ago
    <mostly sarcastic> Bakevm with tooling > create userdata script for cloud init to copy payload to server > create systemd oneshot service to run payload && shutdown
    - colinchartier 6y ago
      You should try it! Our baking interface looks more like Docker than a traditional VM provision (ala Packer) Also, we keep the baked images around for a week and trigger them whenever they are requested (with, say, HTTP, or terminal access)
    - lasdfas 6y ago
      I may be misunderstanding the product, but it seems to be able to run a webserver on each "server".
- lykr0n 6y ago
  More info on that would be cool. Something like Amazon's FireCracker?
  - colinchartier 6y ago
    Sort of - FireCracker is less suited for testing use-cases because it still cold-boots. We use the same kernel features that let laptops resume when the lid is opened instead, they're less mature for production but much faster for staging and testing.
    - lykr0n 6y ago
      Ah. VMware does something similar for hot provisioning VMs from a point in time memory snapshot. So you guys boot the image in the background, then capture a memory dump of it so you can quickly launch VMs from that snapshot? Or you boot the VMs the traditional way the first time, and then just suspend them when not in use?
      - colinchartier 6y ago
        We do even more, we take snapshots as the "Layerfile" runs - this basically lets you merge the memory dump semantics you mention with the interface of Docker. Also, we monitor which files are read by any step in the process, and let you skip setup chunks that don't need to run (i.e., installing libraries) without needing to micromanage what files you copy into the staging server.
        lykr0n 6y ago
        Man that sounds cool. I don't have any use for something like that, but I would love to see it opensourced to play with it (which I assume you guys won't be doing anytime soon)
    - TechBro8615 6y ago
      Are you using socket activation or something similar? I think there are really interesting use cases around that paradigm for packing services onto one host. Some blog posts I remember from when I was exploring this idea a few years ago: http://0pointer.de/blog/projects/socket-activated-containers.html http://0pointer.de/blog/projects/socket-activated-containers... https://www.ctl.io/developers/blog/post/running-drupal-in-linux-containers-but-not-docker-on-pantheon https://www.ctl.io/developers/blog/post/running-drupal-in-li...
- mike_d 6y ago
  Why are you trying to tackle some small little market of staging servers for the handful of companies that actually care about this? Take your Hypervisor and sell it to companies that want to efficiently lift-and-shift legacy systems in to the cloud and make your investors super happy.
deleted 6y ago
[deleted]

defen 6y ago

How would this handle external state? I realize in general that's an impossible task, but specifically I'm thinking of something like a database. Is there a way for each "server" to get a private copy of the database? Otherwise it seems like if I create a non-backwards-compatible migration (let's say I drop a table that's no longer necessary) I could break servers that were spawned for other commits.

colinchartier 6y ago
Glad you asked! If you do the Django tutorial after clicking "try now", you can see an example of a postgres database that gets set up for every run. Since we give full VMs instead of just containers, you can keep the database running in the background. Because we do memory snapshotting, you could turn a 5 minute database start + migrate + populate into a 1 second "restore from memory snapshot"
- ignoramous 6y ago
  > Since we give full VMs instead of just containers, you can keep the database running in the background. Interesting contrast to flockport that runs multiple processes in a LXC [0] (treats it like a lightweight VM). [0] https://news.ycombinator.com/item?id=17478736 https://news.ycombinator.com/item?id=17478736

lazyant 6y ago

First of all, this looks awesome and go Canada! Random comments: Is there a way to try without giving access to my github account? Less importantly: Can you put your email in your HN profile? Can you add LayerCI Twitter handle to your site?

colinchartier 6y ago
1. Not yet unfortunately, we didn't want to waste time creating a separate auth system when we only integrate with GitHub for now anyways. To see a short demo though, here's an earlier one we made: https://www.youtube.com/watch?v=PH-n70gPQgA https://www.youtube.com/watch?v=PH-n70gPQgA 2. My email (for Layer) is colin@layerci.com 3. Done! Thanks for your feedback.

gravypod 6y ago

How do you handle very complex and integrated architectures like a monorepo with many microservices? How do you run your layer file locally? Can you describe automated tests in a layer file? I built something similar[0] to this for integration testing and those were my biggest hurdles. At a previous job where all of the code was python/node/java I was also able to generate cross-service end to end code coverage from these tests and that feature was extremely helpful as it helps you remove redundant tests. Your service looks very polished and I'm extremely impressed someone is taking this idea to market. 0- bunt.build

colinchartier 6y ago
> monorepo with many microservices We scan the entire repository for Layerfiles, so you could, say, create a layerfile per service for unit tests, and a bigger Layerfile in the repo root that does the whole deployment and runs a few E2E tests. > How do you run your layer file locally? They can be approximated by running a docker container and just copy/pasting the commands. We haven't built this yet because I know it will be a huge time sink to make it reliable (see, e.g., vagrant) > Can you describe automated tests in a layer file? We tell you if any step fails (e.g., returns non-zero exit code) so you can just as easily run "./deploy.sh" as "./test.sh" - you can also start a webserver and run E2E or acceptance tests against it, for example. > generate cross-service end to end code coverage We are trying to avoid anything that isn't very general - it's possible to run https://codecov.io/ https://codecov.io/ within Layer if you want functionality like this.