3 ms·

At my current company we started moving our setup to k8s... And using docker-compose required another deployment platform. So end 2019 we started using k3s...

by TeeWEE 5y ago

At my current company we started moving our setup to k8s... And using docker-compose required another deployment platform. So end 2019 we started using k3s... But developers complained about heavy resource usage, and slow startup time. So in the end we moved back to docker-compose... This means our local dev environments are NOT the same as our k8s prod clusters. But if this works well, and k3s is more lean. It could unify our infra stack across developer machines, to production. Same tooling everywhere... Love it

midrus 5y ago
From my experience, trying to boot up the entire platform on developer's laptops (or remote cloud servers) for each developer is something doomed to fail and cause pain, whatever you use. My best experience so far, has been to start services individually, and work with mocks and tests. And do integration on a shared staging environment or, even better, have a way to deploy multiple versions to the same env (like you can do with App Engine, Vercel, etc). I hope I never ever again have to work at a place where I need to replicate the entire platform for my development environment.
pqb 5y ago
Recently, I found Tilt [0] to be a good partner of mine to run "all services locally". It can be compared to "webpack for backend" (live-reloading, a lot of configuration possibilities [1]). Tilt uses Titlfiles for configuration, which are written Pythonish Starlark language and you use them to write any specific logic there. You want to run a bunch of services directly? Use local_resource()/local(). You have Procfile? There is procfile() function. You have a docker-compose.yml with services like databases? You can run it too with docker_compose(). You want to separate your Tiltfile on multiple files and include them all-together on tilt-up command? There is load()/load_dynamic(). You need some web-ui for frontend devs and a nice log browser? It is there too. You want to update your local cluster with newly built image on file save? No problem, tilt will do that with k8s_yaml() function. You need to do some extra steps before running a service (e.g. data seeding)? Use aforementioned local()/local_resource(). You need a sample project that uses tilt? Sure, there is some tutorial for C# [2]. Do you use jib instead of docker-build? Here it is [3]. Also, I am not very lucky in having resemble 1:1 k8s cluster locally. You could be close, but as long as you run in the cloud you will have different configuration on local (additional annotations, various quirks that do not exist in kind/k3s but they are on GCP). However, making dedicated dev environments in the cloud might be very costly and incur a lot of additional tinkering. [0]: https://tilt.dev/ https://tilt.dev/ [1]: https://docs.tilt.dev/api.html https://docs.tilt.dev/api.html [2]: https://docs.tilt.dev/example_csharp.html https://docs.tilt.dev/example_csharp.html [3]: https://github.com/tilt-dev/tilt-example-java/blob/master/101-jib/Tiltfile https://github.com/tilt-dev/tilt-example-java/blob/master/10...

by TeeWEE 5y ago

midrus 5y ago

From my experience, trying to boot up the entire platform on developer's laptops (or remote cloud servers) for each developer is something doomed to fail and cause pain, whatever you use. My best experience so far, has been to start services individually, and work with mocks and tests. And do integration on a shared staging environment or, even better, have a way to deploy multiple versions to the same env (like you can do with App Engine, Vercel, etc). I hope I never ever again have to work at a place where I need to replicate the entire platform for my development environment.

pqb 5y ago

Recently, I found Tilt [0] to be a good partner of mine to run "all services locally". It can be compared to "webpack for backend" (live-reloading, a lot of configuration possibilities [1]). Tilt uses Titlfiles for configuration, which are written Pythonish Starlark language and you use them to write any specific logic there. You want to run a bunch of services directly? Use local_resource()/local(). You have Procfile? There is procfile() function. You have a docker-compose.yml with services like databases? You can run it too with docker_compose(). You want to separate your Tiltfile on multiple files and include them all-together on tilt-up command? There is load()/load_dynamic(). You need some web-ui for frontend devs and a nice log browser? It is there too. You want to update your local cluster with newly built image on file save? No problem, tilt will do that with k8s_yaml() function. You need to do some extra steps before running a service (e.g. data seeding)? Use aforementioned local()/local_resource(). You need a sample project that uses tilt? Sure, there is some tutorial for C# [2]. Do you use jib instead of docker-build? Here it is [3]. Also, I am not very lucky in having resemble 1:1 k8s cluster locally. You could be close, but as long as you run in the cloud you will have different configuration on local (additional annotations, various quirks that do not exist in kind/k3s but they are on GCP). However, making dedicated dev environments in the cloud might be very costly and incur a lot of additional tinkering. [0]: https://tilt.dev/ https://tilt.dev/ [1]: https://docs.tilt.dev/api.html https://docs.tilt.dev/api.html [2]: https://docs.tilt.dev/example_csharp.html https://docs.tilt.dev/example_csharp.html [3]: https://github.com/tilt-dev/tilt-example-java/blob/master/101-jib/Tiltfile https://github.com/tilt-dev/tilt-example-java/blob/master/10...