4 ms·
I've spent days trying all of these solution at my company. All of these solutions suck, they are slow and only successful builds get their layers cached. This
by boronine 2y ago
I've spent days trying all of these solution at my company. All of these solutions suck, they are slow and only successful builds get their layers cached. This is a dead end. The only workable solution is to have a self-hosted runner with a big disk.
- aayushshah15 2y agohow do you ensure isolation between runs on a self hosted runner that way?
- boronine 2y agoWhat kind of isolation do you need? We are building our own code so I don't see the need for isolation beyond a clear directory.
- airspeedjangle 2y agoShared runner infrastructure in a big company. It's pretty common to treat these situations as multi-tenant low trust environments.
- damianh 2y agoPlenty of marketplace actions will install things and/or mutate the runner. It's a matter of time before someone does something or there's a build that doesn't cleannup after itself (e.g. leaving test processes running) that ruins the day for everyone else.
- damianh 2y agoSelh-hosted runners can be ephemeral too. With such either mount the cache as a disk or bake docker layers/images into the runner image.
- remdoWater 2y agoThis requires a lot of work from a dev inf team, though. Not as straightforward for an average team.
- damianh 2y agoI won't disagree. It should be easier imo. I guess this is why a cottage industry has sprung up addressing such e.g. https://news.ycombinator.com/item?id=39930908 https://news.ycombinator.com/item?id=39930908
- intelVISA 2y agoNow I've seen everything...
- boundlessdreamz 2y agoruns-ons supports custom images - https://runs-on.com/features/byoi/ https://runs-on.com/features/byoi/ and caching to S3 - https://runs-on.com/reference/caching/ https://runs-on.com/reference/caching/ I haven't used it yet but these two features make it the clear favourite for me in alternate github action runners
- ownagefool 2y agoIt's actually pretty easy. Setup GitHub app. Install the arc helm charts. Install a buildkitd statefulset. Update parans on build to use buildkitd. That's not to say there aren't better caching strategies, but a really basic ephemeral setup is right there.
- withinboredom 2y agoIf you are in k8s, you can use the default chart provided by GitHub and get 90% of the way there.
- teaearlgraycold 2y agoI use namespace’s action runners for this (just a customer, not affiliated in any way). They’re a company with a pretty good product stack. Although the web UI is annoyingly barebones.
- 20thr 2y agoHi -- Namespace's CEO here; if you have a chance, please drop me a note at hugo-at-namespacelabs.com; I'd love to hear what we could be doing better in the UI, and product overall. Thank you! Hugo @ Namespace (https://namespace.so https://namespace.so)
- kylegalbraith 2y agoThis is definitely a direction to try. But if its faster Docker image builds and a layer caching system that actually works, you should definitely try out Depot. We automatically persist layer cache to persistent NVMe devices and orchestrate that to be immediately available across builds.
- paholg 2y agoIf you have the ability to choose the tooling, I recommend looking into building docker images with nix. Yes, nix is complex. But its caching story is soooo much better than docker's, and all the other docker issues just disappear. https://nix.dev/tutorials/nixos/building-and-running-docker-images.html https://nix.dev/tutorials/nixos/building-and-running-docker-...
- mysza 2y agoCan you share example of github actions? When i use docker/setup-buildx-action and local runner i can't make it use the cache. I think it's the "docker-container" runner's fault