8 ms·

Ephemeral Environments for DevOps: Why, What and How? (2017)

40 points by royosherove 8y ago

Nursie 8y ago
We need this so much. The environment is part of the product. With software, if you can't easily and automatically recreate your builds from source, you have a problem. This ought to apply to cloud environments too.
alexandercrohde 8y ago
So, as a dev-ops engineer, I'm trying to decide if I'm onboard with this idea. I still don't see the justification though. The idea of provisioning an environment to test a branch seems like a lot of overhead. Concretely, how long does it take to spin up an environment, install the containers/app, and run tests? I imagine the answer is going to be over an hour. Definitely feels like a lot of overhead, particularly when most branches change ~.1% of an application's code and so theoretically the full-diff is kilobytes worth of data. - The alternative of course is just to have a pre-set number of build-servers (e.g. 5) and then have jenkins dynamically run jobs against them. Seems simpler, I assume faster, and just less risky.
- chupasaurus 8y ago
  > The idea of provisioning an environment to test a branch seems like a lot of overhead. The idea of running integration tests on a non-complete environment seems like a lot of fun at the release. > Concretely, how long does it take to spin up an environment, install the containers/app, and run tests? I work currently at the community-sized online bank, total time of our pipeline to spin up a new "bank" is 22 minutes, running multiple of these envs on 4 HDDs in RAID 10. We'd move the whole thing on flash array in 2 weeks and my estimation is that this metric will go down to 7-8 minutes. > The alternative of course is just to have a pre-set number of build-servers (e.g. 5) and then have jenkins dynamically run jobs against them. You have to run these builds somewhere, if changes would run on a single staging env, it would be down most of the time. If it would be a handful of stagings, it might be still not enough because either you wouldn't have enough of them and it'll ended up to the single env situation multiplied, or you would burn cash for idle servers.
thinkingkong 8y ago
Wow is this ever overkill. This needs to be coupled with a lot more “how and we we picked this methodology” advise or information. For example, if the goal is to reduce risk then is this the best way to approach that? Are all these components black boxes which cannot be stubbed out? Are there any other end to end or integration tests that can be ran in a canary environment that are way faster which will cover the most obvious use cases? Id really think a few times before adopting this practice.
- argd678 8y ago
  Currently DevOps tools do almost no testing or validation at all, it’s like time traveling back to the 80s.
  - chupasaurus 8y ago
    Internal validation of a backend system could be automated but that would be heck of a job, anything beyond that requires human to interact and report.
andrethegiant 8y ago
Shameless plug: I'm currently working on a service that'll spin up / tear down environments automatically for each branch in your repo. We're focusing on the manual testing use case between designers and front-end engineers, so prod deployment is omitted from our featureset. But if you're looking to spin up ephemeral environments without building something yourself, check us out: https://featurepeek.com https://featurepeek.com
- org3432 8y ago
  Great stuff, I like the idea of bringing the same level of reasoning that's been hard learned over the past 50 years of software development to devops too. Currently the state of devops tools includes omitting debuggers when a company makes a new DSL, poor if any linting, low to non-existent test coverage so obvious bugs and regressions are normal, very poor IDE integration if any, often no real way to test locally, and not surprisingly the tools are bug riddled and at best scratch a single itch in an eco system w/o directly addressing the overall business problem.
- deleted 8y ago
  [deleted]
- danenania 8y ago
  Our product EnvKey[1] can also help to secure and simplify configuration of ephemeral environments—all you do is set a single ENVKEY environment variable, include an envkey library, and then all the secrets/credentials/config you need will be made available and kept in sync on each load. You also get a nice ui for editing variables and managing access. Ah, and all this is done without trusting a central server through the magic of end-to-end encryption. I think it could potentially jive well with a tool like yours given that proper secrets management can be a tricky piece of this puzzle. 1 - https://www.envkey.com https://www.envkey.com
  - andrethegiant 8y ago
    Stumbled upon this the other day! Will take a look
  - 8y ago

40 points by royosherove 8y ago

We need this so much. The environment is part of the product. With software, if you can't easily and automatically recreate your builds from source, you have a problem. This ought to apply to cloud environments too.

alexandercrohde 8y ago

So, as a dev-ops engineer, I'm trying to decide if I'm onboard with this idea. I still don't see the justification though. The idea of provisioning an environment to test a branch seems like a lot of overhead. Concretely, how long does it take to spin up an environment, install the containers/app, and run tests? I imagine the answer is going to be over an hour. Definitely feels like a lot of overhead, particularly when most branches change ~.1% of an application's code and so theoretically the full-diff is kilobytes worth of data. - The alternative of course is just to have a pre-set number of build-servers (e.g. 5) and then have jenkins dynamically run jobs against them. Seems simpler, I assume faster, and just less risky.

chupasaurus 8y ago
> The idea of provisioning an environment to test a branch seems like a lot of overhead. The idea of running integration tests on a non-complete environment seems like a lot of fun at the release. > Concretely, how long does it take to spin up an environment, install the containers/app, and run tests? I work currently at the community-sized online bank, total time of our pipeline to spin up a new "bank" is 22 minutes, running multiple of these envs on 4 HDDs in RAID 10. We'd move the whole thing on flash array in 2 weeks and my estimation is that this metric will go down to 7-8 minutes. > The alternative of course is just to have a pre-set number of build-servers (e.g. 5) and then have jenkins dynamically run jobs against them. You have to run these builds somewhere, if changes would run on a single staging env, it would be down most of the time. If it would be a handful of stagings, it might be still not enough because either you wouldn't have enough of them and it'll ended up to the single env situation multiplied, or you would burn cash for idle servers.

thinkingkong 8y ago

Wow is this ever overkill. This needs to be coupled with a lot more “how and we we picked this methodology” advise or information. For example, if the goal is to reduce risk then is this the best way to approach that? Are all these components black boxes which cannot be stubbed out? Are there any other end to end or integration tests that can be ran in a canary environment that are way faster which will cover the most obvious use cases? Id really think a few times before adopting this practice.

argd678 8y ago
Currently DevOps tools do almost no testing or validation at all, it’s like time traveling back to the 80s.
- chupasaurus 8y ago
  Internal validation of a backend system could be automated but that would be heck of a job, anything beyond that requires human to interact and report.

andrethegiant 8y ago

Shameless plug: I'm currently working on a service that'll spin up / tear down environments automatically for each branch in your repo. We're focusing on the manual testing use case between designers and front-end engineers, so prod deployment is omitted from our featureset. But if you're looking to spin up ephemeral environments without building something yourself, check us out: https://featurepeek.com https://featurepeek.com

org3432 8y ago
Great stuff, I like the idea of bringing the same level of reasoning that's been hard learned over the past 50 years of software development to devops too. Currently the state of devops tools includes omitting debuggers when a company makes a new DSL, poor if any linting, low to non-existent test coverage so obvious bugs and regressions are normal, very poor IDE integration if any, often no real way to test locally, and not surprisingly the tools are bug riddled and at best scratch a single itch in an eco system w/o directly addressing the overall business problem.
deleted 8y ago
[deleted]
danenania 8y ago
Our product EnvKey[1] can also help to secure and simplify configuration of ephemeral environments—all you do is set a single ENVKEY environment variable, include an envkey library, and then all the secrets/credentials/config you need will be made available and kept in sync on each load. You also get a nice ui for editing variables and managing access. Ah, and all this is done without trusting a central server through the magic of end-to-end encryption. I think it could potentially jive well with a tool like yours given that proper secrets management can be a tricky piece of this puzzle. 1 - https://www.envkey.com https://www.envkey.com
- andrethegiant 8y ago
  Stumbled upon this the other day! Will take a look
- 8y ago