4 ms·
I've started using K8s and so far so good. Minikube makes local dev easy. However, I use K8s with ansible. So now I can: * Build a fully baked container for lo
by brango 9y ago
I've started using K8s and so far so good. Minikube makes local dev easy. However, I use K8s with ansible. So now I can:
* Build a fully baked container for local use
* Build a container that mounts my local filesystem for the code directory so I can have a sane dev-test cycle with hot-code reloading
* Mount a directory when running locally containing credentials for accessing Google Cloud services
* Spin up a K8s cluster in arbitrary GKE accounts, template and deploy my deployment and service accounts, pin some with external IPs, etc
* Push containers to GCR
* Deploy my containers wherever - shared directory ones can only run locally.
This is for an architecture with about 4 microservices and that will probably grow.
Now I'm working on an ansible playbook to zip the code directory of one of my microservices, upload it to GCS, then run a build container on my cluster to build the docker image for my microservice and push to GCR from there so I don't have to waste time pushing large containers up to GCR. Once this is done I'll look at promoting containers through dev/test/prod environments since all config will be done with env vars.
I've never read about using ansible with K8s, but to me it's a no-brainer. Most people seem to cobble together bash scripts, but using the best of both has really led to a good experience.
The benefit we hope to get is isolation and cost efficiency.
- scaryclam 9y agoThat's interesting. I'm using docker-compose for local development at the moment. Do you have any experience with it and would you choose Minikube over docker-compose?
- cryptarch 9y agoYou didn't ask me, but here you are: I'm using docker-compose as well but I'm looking at moving to Minikube, so I can more realistically test my K8s clusters before sending them down to CI/staging. I also don't want to maintain both docker-compose and K8s as I basically am ops by myself and I have NaN tasks in my backlog.
- brango 9y agoI haven't used docker-compose. I'd been holding back with docker until all the stuff like service discovery, scaling, secrets, sane local dev and deployment experiences were available. So far K8s seems to tick all those boxes.
- dlor 9y agoSince you're already on GCP/GKE, have you considered using Cloud Container Builder? https://cloud.google.com/container-builder/docs/ https://cloud.google.com/container-builder/docs/ It provides an API for what you're already doing with uploading zips to GCS, doing your build, and having the resulting container end up in GCR. Disclaimer: I work at Google on GCP.
- brango 9y agoThanks. I'd missed that. It looks pretty close to what I want. Just one suggestion - I'm uploading python files. It'd be nice to be able to exclude my `venv` virtual env directory from being zipped and uploaded since that's strictly for local use. From the docs I'm not sure if that's possible (I've not gone into detail with them yet). If I can also run tests as part of the build process then I'd be sold.