4 ms·

There are a lot of projects these days that aim to use Kubernetes as a docker-compose environment. I personally use http://skaffold.dev/ http://skaffold.dev/ wi

There are a lot of projects these days that aim to use Kubernetes as a docker-compose environment. I personally use http://skaffold.dev/ http://skaffold.dev/ with either a local Kubernetes cluster built with https://github.com/rancher/k3d https://github.com/rancher/k3d or https://github.com/kubernetes-sigs/kind https://github.com/kubernetes-sigs/kind. I think there's a very easy argument to be made that says that running K8s locally is overkill, but what I will say is that if you run your applications locally in K8s, that's one step closer to having your local environment mirror the production environment. Couple that with things like running https://github.com/localstack/localstack https://github.com/localstack/localstack locally and you get even closer.

gravypod 6y ago
`kind` and `k3d` are great but I don't think their aimed at being a developer friendly cluster from what I understand. They both require a lot of tweaking and setup to make them look like their a managed kube cluster (LB, storage, ingresses, etc). I wish there was something that could create a kube cluster that: 1. sets up automatic port forwards to my host system 2. handles ingresses by binding port :80 on my system 3. provisions PVs for you when you create PVCs and respects retention policies 4. Allows you to select from a monitoring stack and only require that you supply configurations for dashboards/alerts that can be 1:1 applicable to production so devs can debug dashboards/alerts locally. `kind` is targeting itself as being "a real kubernetes implementation" `k3d` is targeting itself as being Rancher's kubernetes implementation so if you buy into all of the Rancher tooling it'd be a great option.
- MichaelMoser123 6y ago
  I have a script that simplifies creation of kind clusters with local registry and ingress https://github.com/MoserMichael/kind-helper https://github.com/MoserMichael/kind-helper (to make it more developer friendly that is) but i guess kind is more for automation testing purposes as it runs all modes in docker containers on the same machine.
  - gravypod 6y ago
    Looks like a nice tool. If it could watch for changes, rebuild, repush, and restart pods it would be the perfect dev environment. Even still this looks like a great start. Have you thought about polishing it a little more and doing a Show HN?
    - MichaelMoser123 6y ago
      Thanks, i think that building images and restarting pods would be a bit out of scope for this script, but you have examples in the tests for this project. For kubernetes cluster administration i have this project; https://github.com/MoserMichael/s9k https://github.com/MoserMichael/s9k
tilolebo 6y ago
Talking about kind and k3s, does anyone know how to load a local docker image into a k3d cluster? The command is supposed to be: 'k3d image import --cluster myCluster image:tag', but it fails with "Failed to start exec process in node 'k3d-myCluster-tools'" On kind it works fine with 'kind load docker-image --name myCluster image:tag'. I'm using WSL2.
- MichaelMoser123 6y ago
  You might take a look at my script https://github.com/MoserMichael/kind-helper https://github.com/MoserMichael/kind-helper , sets up local regustry with kind so you can do without image import. (Comes with test case/example))
watermelon0 6y ago
Neither k3d (k3s) nor kind support building of images since they use containerd directly, which [building] seems to be a requirement for development environment. This means that you need to build your app using Docker, and then import it to k3d/kind environment, thus needing local registry (not enabled by default), which means that images are duplicated.
- gravypod 6y ago
  Yea, exactly, this. I see all sorts of people making build systems and tools and they say you just do "my_command build my_target && docker-compose up -d" and I can't help but wonder: are these people ONLY developing 1 service? For me, that would be multiplied by 20x and we're not even at the number of services we're expecting. We do a lot of integration with third parties which is essentially pull down some data and write it into a microservice responsible for keeping that data. Each of these integrations is owned by a separate engineer because it takes up a lot of time to deal with integration calls, planning, etc and we need to be language flexible (some people only give you a .dll or .jar, no choices). I'm assuming people building these tools don't use a local development workflow because: their systems are too big to run on a single machine (Googlers) or don't work on large projects (open source side projects). The clostest thing to what you and I are looking for, though, is skaffold and tilt.dev.
- moondev 6y ago
  Have you tried building images from inside a pod via buildah?
- oldman_peter 6y ago
  I use a `buildah` to build images in a k3s container. You'll have to enable user_namespace at boottime on your k3s host though.

gravypod 6y ago

`kind` and `k3d` are great but I don't think their aimed at being a developer friendly cluster from what I understand. They both require a lot of tweaking and setup to make them look like their a managed kube cluster (LB, storage, ingresses, etc). I wish there was something that could create a kube cluster that: 1. sets up automatic port forwards to my host system 2. handles ingresses by binding port :80 on my system 3. provisions PVs for you when you create PVCs and respects retention policies 4. Allows you to select from a monitoring stack and only require that you supply configurations for dashboards/alerts that can be 1:1 applicable to production so devs can debug dashboards/alerts locally. `kind` is targeting itself as being "a real kubernetes implementation" `k3d` is targeting itself as being Rancher's kubernetes implementation so if you buy into all of the Rancher tooling it'd be a great option.

MichaelMoser123 6y ago
I have a script that simplifies creation of kind clusters with local registry and ingress https://github.com/MoserMichael/kind-helper https://github.com/MoserMichael/kind-helper (to make it more developer friendly that is) but i guess kind is more for automation testing purposes as it runs all modes in docker containers on the same machine.
- gravypod 6y ago
  Looks like a nice tool. If it could watch for changes, rebuild, repush, and restart pods it would be the perfect dev environment. Even still this looks like a great start. Have you thought about polishing it a little more and doing a Show HN?
  - MichaelMoser123 6y ago
    Thanks, i think that building images and restarting pods would be a bit out of scope for this script, but you have examples in the tests for this project. For kubernetes cluster administration i have this project; https://github.com/MoserMichael/s9k https://github.com/MoserMichael/s9k

tilolebo 6y ago

Talking about kind and k3s, does anyone know how to load a local docker image into a k3d cluster? The command is supposed to be: 'k3d image import --cluster myCluster image:tag', but it fails with "Failed to start exec process in node 'k3d-myCluster-tools'" On kind it works fine with 'kind load docker-image --name myCluster image:tag'. I'm using WSL2.

MichaelMoser123 6y ago
You might take a look at my script https://github.com/MoserMichael/kind-helper https://github.com/MoserMichael/kind-helper , sets up local regustry with kind so you can do without image import. (Comes with test case/example))

watermelon0 6y ago

Neither k3d (k3s) nor kind support building of images since they use containerd directly, which [building] seems to be a requirement for development environment. This means that you need to build your app using Docker, and then import it to k3d/kind environment, thus needing local registry (not enabled by default), which means that images are duplicated.

gravypod 6y ago
Yea, exactly, this. I see all sorts of people making build systems and tools and they say you just do "my_command build my_target && docker-compose up -d" and I can't help but wonder: are these people ONLY developing 1 service? For me, that would be multiplied by 20x and we're not even at the number of services we're expecting. We do a lot of integration with third parties which is essentially pull down some data and write it into a microservice responsible for keeping that data. Each of these integrations is owned by a separate engineer because it takes up a lot of time to deal with integration calls, planning, etc and we need to be language flexible (some people only give you a .dll or .jar, no choices). I'm assuming people building these tools don't use a local development workflow because: their systems are too big to run on a single machine (Googlers) or don't work on large projects (open source side projects). The clostest thing to what you and I are looking for, though, is skaffold and tilt.dev.
moondev 6y ago
Have you tried building images from inside a pod via buildah?
oldman_peter 6y ago
I use a `buildah` to build images in a k3s container. You'll have to enable user_namespace at boottime on your k3s host though.