4 ms·

My team has been developing against a fully remote environment (K8s cluster) for some years now and it makes for a really powerful DevEx. Code sits on our lapt

by eysi 3y ago

My team has been developing against a fully remote environment (K8s cluster) for some years now and it makes for a really powerful DevEx. Code sits on our laptops but live syncs to the remote services without requiring a Docker build or K8s deploy. It really does feel like local. In particular it lets us do away with the commit-push-pray cycle because we can run integ tests and beyond as we code as opposed to waiting for CI. We use Garden, (https://docs.garden.io https://docs.garden.io) for this. (And yes I am afilliated :)). But whether you use Garden or not, leveraging the power of the cloud for “inner loop” dev can be pretty amazing with right tooling. I wrote a bit more about our experience here: https://thenewstack.io/one-year-of-remote-kubernetes-development-lessons-learned/ https://thenewstack.io/one-year-of-remote-kubernetes-develop...

jayd16 3y ago
Kind of interesting to think that CI is significantly slower in practice and both systems need to be maintained. Is it just the overhead of pushing through git or are there other reasons as well?
- mewpmewp2 3y ago
  You would need a very perfect and flexible CI system in place that wouldn't need to rebuild anything it doesn't need and only run the tests you want or only recently failed tests etc. Many CI systems would spin up a new box instead of using persistent so likely have to rebuild if no cache, etc. So basically I would say most of the overhead is in not having a persistent box with knowledge of last build or ability to choose what to run in there, which pretty much just equals to local capabilities.
  - peanball 3y ago
    Often you also have the CI system designed in a way to verify a “from scratch” build that avoids any issues with “works on my machine” scenarios due to things still being cached that shouldn’t be there anymore.
  - jayd16 3y ago
    Having persistent boxes with sticky sessions seems seems pretty achievable.
- eysi 3y ago
  The way we do things is that we build everything in the cloud and store in a central container registry. So if I trigger a build during dev, the CI runner can re-use that, e.g. if it’s needed before running a test or creating a preview env. Similarly if another dev (or a CI runner) triggers a build of one of our services, I won’t have to next time I start my dev environment. And because it’s built in the cloud there’s no “works on my machine”. Same applies to tests actually. They run in the cloud in an independent and trusted environment and the results are cached and stored centrally. Garden knows all the files and config that belong to a given test suite. So the very first CI run may run tests for service A, service B, and service C. I then write code that only changes service B, open a PR and only the relevant tests get run in CI. And because it’s all in prod-like environments, I can run integ and e2e tests from my laptop as I code, instead of only having that set up for CI.
8n4vidtmkvmk 3y ago
I tried Garden briefly but didn't like it for some reason. DevSpace was simpler to set up and works quite reliably. The sync feature where they automatically inject something into the pod works really well.
- eysi 3y ago
  DevSpace is a great tool but it’s bummer you didn’t like Garden. Admittedly, documentation and stability weren’t quite what we’d like and we’ve done a massive overhaul of the foundational pieces in the past 12 months. If you want to share feedback I’m all ears, my email is in my profile.

by eysi 3y ago

jayd16 3y ago

Kind of interesting to think that CI is significantly slower in practice and both systems need to be maintained. Is it just the overhead of pushing through git or are there other reasons as well?

mewpmewp2 3y ago
You would need a very perfect and flexible CI system in place that wouldn't need to rebuild anything it doesn't need and only run the tests you want or only recently failed tests etc. Many CI systems would spin up a new box instead of using persistent so likely have to rebuild if no cache, etc. So basically I would say most of the overhead is in not having a persistent box with knowledge of last build or ability to choose what to run in there, which pretty much just equals to local capabilities.
- peanball 3y ago
  Often you also have the CI system designed in a way to verify a “from scratch” build that avoids any issues with “works on my machine” scenarios due to things still being cached that shouldn’t be there anymore.
- jayd16 3y ago
  Having persistent boxes with sticky sessions seems seems pretty achievable.
eysi 3y ago
The way we do things is that we build everything in the cloud and store in a central container registry. So if I trigger a build during dev, the CI runner can re-use that, e.g. if it’s needed before running a test or creating a preview env. Similarly if another dev (or a CI runner) triggers a build of one of our services, I won’t have to next time I start my dev environment. And because it’s built in the cloud there’s no “works on my machine”. Same applies to tests actually. They run in the cloud in an independent and trusted environment and the results are cached and stored centrally. Garden knows all the files and config that belong to a given test suite. So the very first CI run may run tests for service A, service B, and service C. I then write code that only changes service B, open a PR and only the relevant tests get run in CI. And because it’s all in prod-like environments, I can run integ and e2e tests from my laptop as I code, instead of only having that set up for CI.

8n4vidtmkvmk 3y ago

I tried Garden briefly but didn't like it for some reason. DevSpace was simpler to set up and works quite reliably. The sync feature where they automatically inject something into the pod works really well.

eysi 3y ago
DevSpace is a great tool but it’s bummer you didn’t like Garden. Admittedly, documentation and stability weren’t quite what we’d like and we’ve done a massive overhaul of the foundational pieces in the past 12 months. If you want to share feedback I’m all ears, my email is in my profile.