17 ms·
Tilt – Local Kubernetes development with no stress
- linkmotif 8y agoMy biggest problem with local k8s dev is building containers. I’m on macOS. Does this solution address that?
- yebyen 8y agoIs the problem with building containers that you don't have a Dockerfile and you don't want to spend time making one? Draft does that pretty well with Draftpacks, in my experience if there is a Draftpack for what you're using then the Dockerfile that draft spits out will get you pretty close to a workable container image with basically no effort, or sometimes even all the way there. I'm trying to understand what you mean that building containers is a problem. Is it the procedural act of actually building and rebuilding the container that is the problem? (Draft handles that too...) Edit: According to the Tilt tutorial, yes, tilt will build your containers. Eg: https://docs.tilt.build/first_config.html https://docs.tilt.build/first_config.html
- linkmotif 8y agoI’m all aboard Docker, but I don’t use Linux natively, so I’d need to run a VM inside macOS, or have another box to build containers. Even if I did, I use GCR in production, so I’d need to template my container images in helm to use GCR only in production versus whatever I use locally for development. And what would that even be? I haven’t explored minikube to know what it does for local dev container registry. Anyway, thanks for the link. Missed that. EDIT: Looks like tilt is using GCR in those examples. I’m not trying to traffic container images off of google cloud for local development. That requires (fast) internet just for local development, and outgoing traffic really adds up quickly in terms of cost. PSS: never heard of Draft. Checking it out now. Not using azure though, maybe that’s not relevant.
- yebyen 8y agoAzure team open source products can almost all be used without Azure cloud.[1] I do use Azure but not for everything, you can definitely use Draft without Azure, it's Kube-native software so you can use it anywhere you have Kubernetes. (Like Tilt, from what I understand about it so far... Tilt and Draft look to be very similar tools, with different approaches.) I helped someone on Twitter with this question last week. I have a (* single-node) Kube somewhere and I want to build images in it, and not have to pull them from a repository. The parlance in Kubernetes for this is, an imagePullPolicy of Never. (you actually do need a repository if you don't have a single-node Kubernetes, full-stop.) Docker does actually run a VM with Linux in it, doesn't it? The modern version of Docker for MacOS also offers a Kubernetes checkbox, so if you're already using Docker, you needn't have another VM just for running Kubernetes. This is certainly a problem that could use more clarity and for users, maybe a nice howto blog post for clarification, but it's absolutely possible to do this without a registry. I think Tilt mentioning "gcr.io" in the image tag is not necessarily a signal that you should actually push your image to the gcr.io registry. It's therefore possible to build and execute your containers while never pushing or pulling. If that's what you wanted to do, then that's exactly what you may do by setting your imagePullPolicy to Never. (I have no idea how this works with Tilt, but I'd assume it's possible to use a similar approach since it's also on Kubernetes.) The tweet in question: https://twitter.com/yebyen/status/1080534315157635072 https://twitter.com/yebyen/status/1080534315157635072 [1]: the exception is things like aks-engine. You can only use aks-engine with Azure as I understand it, because Azure implements their own API for creating virtual machines, and nobody else implements that particular API. But I can't say how unique the Azure API actually is, or if it's also open source, could you build your own Azure in a rack, from all open-source components? That would be pretty cool, if you could...
- mnutt 8y agoIf you tried something like docker-machine on Mac a while back and were disappointed with abysmal performance, it's worth another look. They've replaced VirtualBox with HyperKit, which uses the Apple-supported Hypervisor.framework and works much, much better.
- nicksantos 8y agoLocal kubernetes solutions (like docker-for-mac and minkube) usually let you push your image directly to the local docker daemon. No need to upload to a remote registry only to re-download it. A lot of tools in this space are pretty good about detecting this optimization, including Tilt. (disclaimer: I work on Tilt)
- johnmarcus 8y agoTry working with Elixer and Phoenix and you will start to appreciate how builds can be a nightmare.
- yebyen 8y agoNot trying to be obstinate, but I'm a Rails dev and probably won't do that There is a Draftpack for elixir: https://github.com/technosophos/draft-elixir/tree/master/packs/elixir https://github.com/technosophos/draft-elixir/tree/master/pac... And here's an Alpine-based dockerfile: https://github.com/bitwalker/alpine-elixir-phoenix https://github.com/bitwalker/alpine-elixir-phoenix What kind of problems would I expect to run into?
- rcarmo 8y agoI have no problems building containers on macOS. Can you elaborate?
- linkmotif 8y agoBuilding containers requires Linux, so I need to run a VM or build on another box/the cloud.
- NickBusey 8y agoI built a container a few minutes ago on macOS, not sure why you believe this to be the case.
- linkmotif 8y agoDocker containers/lxc require Linux. They are a Linux technology.
- aaachilless 8y agoThis is technically true but in practice Docker for Mac hides this from you and does not require a prohibitive amount of memory. 99% of the developers at my company have 16gb MBPs and we all build multiple Docker containers daily.
- linkmotif 8y agoThanks. I didn’t know the visualization system changed. I will check this out again.
- chupasaurus 8y agoAs weird as it sounds, Microsoft had implemented the full namespaces stack in Server 2016 (unavailable in Win10 as the base for native containers is Server where kernel differs a bit), which Docker supports (and k8s in alpha/early beta). Docker switched from LXC to containerd in 2016.
- figpope 8y agoIs your issue with building containers and getting them into the minikube VM? If so, you can use "eval $(minikube docker-env)" to make your shell's docker commands talk to the docker daemon in the VM.
- linkmotif 8y agoRunning a VM is a total drag. I only have 16GB RAM. Then even if I run a VM, the docker registry is not the same as my production docker registry (GCR). I guess I can account for this with a variable in my helm templates, but then it’s just more complexity. Would you recommend minikube even with 16GB RAM? How much do you allot to the VM?
- gkop 8y agoI'm a long-time minikube fan, I like the isolation (totally worth whatever minor VM memory overhead), reliability, ease-of-use; it feels like an appliance in a good way, which is something I traditionally associate with VMs. minikube contributed strongly to me personally getting excited about k8s in 2016. The local docker registry is an asset because it gives me flexible when offline.
- linkmotif 8y agoThank you for your report! I will look into this.
- flurdy 8y agoUuuuh not using YAML. Controversial. (in the docker/k8s ecosphere, where YAML seems ubiquitous) I don't love YAML though I am not sure Starlark looks any more readable. YACL => Yet another config language...?
- Proven 8y ago> Uuuuh not using YAML That's not what it says. Maybe next time you can try to understand before you comment. And YAML sucks, by the way.
- sb8244 8y agoThis would be a stopper for me simply because all k8s config my team does is in yaml
- q3k 8y agoYAML, especially how it's used in k8s, is terrible. Writing it by hand is mind-numbingly toilful and extremely error prone, and templating YAML with Go (like Helm does) is a travesty. The most sensible approach I've seen is to use jsonnet+kubecfg. This lets me use a somewhat sane, composable, turing complete language that spits out a tree of k8s objects. I can use as much or little repeated definitions (or abstractions) as I like. Then kubecfg performs all the kubectl invocations to actually make the cluster state match my calculated definitions.
- fermuch 8y agoI'd also point out pulumi. It is a service, but it solves a lot of trouble when dealing with k8s configuration, and you have a real language to to create your definitions...
- q3k 8y agoBeing written in Node.js and running in an aaS model (unless you go for the call-us enterprise pricing), Pulumi unfortunately gets scratched off my list of software I want to use for provisioning production.
- 8y ago
- maxdo 8y agois there anything working with helm in the similar way ?
- GarMan 8y agoThere are at least two options that work with helm, google's skaffold (https://github.com/GoogleContainerTools/skaffold https://github.com/GoogleContainerTools/skaffold) and microsoft's draft (https://github.com/Azure/draft https://github.com/Azure/draft). The former supports helm along with kustomize and just plain old kubectl yaml files, draft seems to be tied to helm. Personally I have used skaffold with success, and draft paniced on me so I stopped playing with it.
- kkapelon 8y agothere is also garden.io
- dexcs 8y agoTell me you don't work for them...
- metzby 8y agoTilt can use helm! (Disclaimer: Tilt CEO here) You can just call k8s_yaml(local('helm template <args>')). That tells Tilt to parse the yaml generated by running (locally) helm template. Feel free to reach out if you'd like to pair to try it.
- rlamana 8y agoAt Okteto we had similar issues so we built our own tool and open-sourced it as CND (https://github.com/okteto/cnd https://github.com/okteto/cnd). Our approach is to be decoupled from how your pod is deployed so it works with Helm, CNAB, kubectl, etc…
- rcarmo 8y agoI love the name, especially considering that the GitHub username is 'windmilleng'. (if you don't get it, look up Dom Quixote)
- FrankPetrilli 8y agoHey, awesome! I met the Windmill Eng team a few weeks ago at Kubecon in Seattle, awesome to see tilt here. It's definitely a young product, but it's a good start to exactly the experience I want. By experience I mean that when I do local dev and edit code live while reloading or restarting the service, I see changes reflected immediately in 2-3 commands that take seconds. In moving software towards Kubernetes, I come to rely on the service abstractions available therein. So it's sensical to develop in the same kind of environment. Tilt gives me the ability to get the save => build => package => push => reload loop into Kubernetes and without the manual hassle thereof. I've used it in minikube, on-prem, and GKE. Running a dev box in a GCE instance and running tilt against GKE is a super nice environment. (Fair notice: I'm a Googler who focuses on GKE, so bias and all, my opinions are not Google's, etc.)
- metzby 8y agoThanks Frank! It was great to chat at Kubecon. (I'm Dan, the Windmill CEO, goofy big-haired guy)
- karakanb 8y agoI think the docs kind of misses what problem it solves and how it solves it. I do all the local stuff with docker-compose while using the same images for k8s deployments, how does this help me? Does this aim to replace docker-compose with k8s, or is this just for testing k8s resource changes in realtime, or is it testing your local k8s resource definitions against a remote cluster? Obviously this is useful for some people, I would like to understand what problems this project adresses and how it adresses them. Also, doesn't not reusing the k8s definitions and re-defining them in a tiltfile cause yet-another-config-to-manage, which can quite easily digress from the production resource definitions, which is one of the problems using docker-compose for local development and k8s for deployment? I would really like to read the problem and proposed solutions in a detailed documentation page. I might have missed it if there is already one, please correct me if so.
- metzby 8y agoHey, I'm CEO of Tilt. Thanks for the feedback. The problem this solves is that distsys devs don't get enough feedback. You say you use docker-compose. Compared to docker-compose, Tilt has two advantages: 1) It updates the services as you edit them 2) Its UI makes it easy to see where errors are occurring, without you having to play 20 questions on the command-line or worry about things scrolling off-screen. The Tiltfile doesn't redefine your k8s definitions; it reuses your existing k8s definitions. The Tiltfile just tells Tilt how to get them. (Many teams don't just use yaml files, but generate them, so Tilt has to be able to do that, too.) There is a landing page at https://tilt.build https://tilt.build which has more of the "why Tilt". Also, we're adding native support for docker-compose next week. I'd love to understand your case more, because at a high level you sound like exactly the kind of dev we want to help.
- hodgesrm 8y agoHi! Thanks for checking in on HN. I'm very interested in k8s productivity but had some trouble understanding what Tilt is actually doing--is it a replacement for minikube, hosted kubernetes, etc.? Is there an architecture diagram that shows what runs where?
- kkapelon 8y agoHow does this compare with garden.io, skaffold, draft, microk8S etc???
- metzby 8y agoHey, Tilt CEO here, so I'm certainly biased, but also semi-informed. We think you deserve three properties: 1) it's easy to start your whole app 2) it's fast to update your app as you edit 3) common problems can't be missed. docker-compose gives you 1. (some people hack in mount points and file watching to get 2, but it's hacky) skaffold and garden.io give you 1 and some of 2. (Tilt has fast_build, which can update pods in-place, without rebuilding the whole image, which I don't believe either skaffold or garden.io do) Our big value-add is the UI. Which may sound weird for a Terminal UI, but if you look at the demo video on https://tilt.build https://tilt.build , I think you'll get a sense for how Tilt is working to keep the problem that's blocking you in your face, so you don't waste time playing 20 questions with kubectl.
- deleted 8y ago[deleted]
- eysi 8y agoHi all, Garden CTO here. First of all, great job on Tilt! A lot of interesting stuff happening in terms of DevEx in the multi-service realm. I just wanted to chime in on the points above. Regarding 2), Garden does indeed support updating pods without re-building and re-deploying via our hot-reload feature (https://docs.garden.io/using-garden/hot-reload https://docs.garden.io/using-garden/hot-reload) which essentially copies source files into the running container on file save (works best for dynamic languages). Regarding 3), Garden has a terminal UI that shows the status of individual services and updates as changes are made to the codebase. It will for example print error messages for failed container builds and failed deployments. If configured so, Garden can also run tests on code changes and will print the error output if tests fail. Our next release will also contain the first version of a dashboard which displays service statuses and dependency graphs and updates in real time. However, our terminal UI is not interactive like Tilt’s—which looks really nice!
- deleted 8y ago
- jaytaylor 8y agoIn the readme there's a link to kubernetes slack. Anyone know how can I get an invite to join the workspace?
- metzby 8y agohttp://slack.k8s.io/ http://slack.k8s.io/
- leandot 8y agoIt actually uses your existing Dockerfile and existing k8s yaml definitions contrary to what some people are saying in the comments,the Tiltfile seems to be just a handful of glue logic. The logs ui is pretty nice too, I've definitely looked for an easier way to check logs from multiple running pods.
- tirumaraiselvan 8y agoSeems like this is the main product of a full-time startup. In that case, I am curious how are you thinking about the future and business model considering it is 1) open source and 2) many competitors.
- phentz 8y agoHow does it compare with Telepresence + skaffold ? I like the console UI idea. I dislike the server components (the same way I dislike draft and tiller).