4 ms·
as someone late to the party, what's the local development auto-reload, auto-deployment, etc. story in k8s? i can set up a k8s instance with that or k3s or micr
by baq 4y ago
as someone late to the party, what's the local development auto-reload, auto-deployment, etc. story in k8s? i can set up a k8s instance with that or k3s or microk8s or whatever relatively easily, but how do i get my code into there without having to build and push images after the smallest change? short googling points to skaffold[0], but is that state of the art?
[0] https://skaffold.dev/ https://skaffold.dev/
- yrro 4y agoodo, perhaps? https://developers.redhat.com/products/odo/overview https://developers.redhat.com/products/odo/overview
- mrj 4y agoI'm currently working on creating a local dev with https://tilt.dev/ https://tilt.dev/ and having a good time
- godtoldmetodoit 4y ago+1 on Tilt. It just works.
- ClumsyPilot 4y agoTilt is excellent, and unlike other fools it does not ise YAML, it uses python - real programming language- so you can deal with any strange scoping, pathing,or arbitrary logic like 'look for this file in this folder, but only on second tuesday after full moon' And it is declarative. Try it, and you will never look back
- sound1 4y agoenjoyed your comment!
- ranger207 4y agoFor webdev, my team has the build output directory mounted as hostPath volumes, so you change the files on your disk and the changes are reflected in the pod
- skinnyarms 4y agoSkaffold is great, I'm hearing good things about Tilt though so I was planning on giving it a whirl soon.
- scottm01 4y agoCheck out Devspace: https://devspace.sh/ https://devspace.sh/ Gives you a fairly slick "hot reload" experience in k8s (minikube-created or not).
- stusmall 4y agoI appreciate the comments on this because its confirming my own experience with k8s local dev best practices. As I write this there are 5 comments with 4 different technologies. For a less cheeky answer, I'm currently using skaffold but I'm not in love with it. It does the job but has some rough edges
- rdli 4y ago(Note: I work on Telepresence) There's a bunch of semi-opinionated tools like Skaffold: Garden, Tilt (just bought by Docker), Docker Compose, and of course, the trusty ol' shell script powered by kustomize. We wrote Telepresence because we found that everyone has their own snowflake-ish workflow once you get beyond the base case of "build container, kubectl apply", so we decided to focus just on the inner dev loop, and then integrate with your preferred workflow. YMMV.
- qbasic_forever 4y agoSkaffold is in my experience the best tool in this space, but there are ton of promising options. There's a really nice VS Code plugin from google that basically adds a whole workflow based on skaffold to the IDE: https://cloud.google.com/code/docs/vscode https://cloud.google.com/code/docs/vscode or https://marketplace.visualstudio.com/items?itemName=GoogleCloudTools.cloudcode https://marketplace.visualstudio.com/items?itemName=GoogleCl... Despite a lot of focus on Google's hosted k8s offersings, the cloud code extension happily works with minikube and other local clusters.
- russellwolf 4y agoGoogle PM for Skaffold & Cloud Code here. Glad to hear Skaffold & Cloud Code are working well for you. I wanted to note for others that there is a Cloud Code JetBrains plugin that also adds a workflow based on Skaffold to IDEs such as IntelliJ, PyCharm, etc. as well. Just like the Skaffold and the VS Code plugin, it also works with any K8s cluster. Documentation: https://cloud.google.com/code/docs/intellij https://cloud.google.com/code/docs/intellij JetBrains Marketplace listing: https://plugins.jetbrains.com/plugin/8079-cloud-code https://plugins.jetbrains.com/plugin/8079-cloud-code
- oofbey 4y agoMy standard trick here is to mount your code directory into the container. Pretty easy with k8s using `hostPath`. You can still build the container with the production-worthy process, but whatever directory your code ends up in gets overridden with the live directory you're editing from the host machine. Then if you're using an application server that can run in a dev mode and notice when file changes, changes show up instantly just as if you're not using containers at all.
- physicles 4y agoHere's a way to avoid having to push the image (but you do still have to build it). If you're using k3s and containerd for the container runtime, you can build the image with docker and then load it directly to the place containerd expects it to be stored. Then if the pod's imagePullPolicy is IfNotPresent, simply deleting/restarting the pod will be enough to get your new code running. The magic incantation is: docker save the-image-name:latest | sudo ctr -a /var/run/k3s/containerd/containerd.sock -n k8s.io image import - But for front end development where auto-reload is super useful, I don't use containers at all. Just expose whatever services you need outside the cluster using NodePort or host mode networking.
- russellwolf 4y agoGoogle PM for Skaffold here. As others have mentioned, there are many great solutions in the market for Kubernetes local iterative development. If you do decide to give Skaffold a try, I recommend checking out the File Sync feature (https://skaffold.dev/docs/pipeline-stages/filesync/ https://skaffold.dev/docs/pipeline-stages/filesync/) which enables you to continue making changes during a Skaffold iterative development session and have that code synced over to your running Kubernetes instance without having to rebuild or push images. When using Jib or Buildpacks to containerize your application, File Sync just works for Go, NodeJS, and Java but requires a little configuration for Dockerfiles. FWIW, Skaffold is under active development and we're continuing to expand support for iterative development, CI and CD use cases as well as simplifying configs and general ease of use. We also have a bunch of ways you can reach out to the team (https://skaffold.dev/docs/resources/ https://skaffold.dev/docs/resources/) if you have any questions. We'd be happy to hear from you.