5 ms·

This is absolutely bananas; for my own CI workflow I'll have to pay $140+/month now just to run my own hardware.

by featherless 10mo ago

hedgehog 10mo ago
I'm curious, what are you doing that has over 1000 hours a month of action runtime?
- featherless 10mo ago
  I run a local Valhalla build cluster to power the https://sidecar.clutch.engineering https://sidecar.clutch.engineering routing engine. The cluster runs daily and takes a significant amount of wall-clock time to build the entire planet. That's about 50% of my CI time; the other 50% is presubmits + App Store builds for Sidecar + CANStudio / ELMCheck. Using GitHub actions to coordinate the Valhalla builds was a nice-to-have, but this is a deal-breaker for my pull request workflows.
  - hedgehog 10mo ago
    Cool, that looks a lot nicer than the OBD scanner app I've been using.
- Eikon 10mo ago
  On ZeroFS [0] I am doing around 80 000 minutes a month. A lot of it is wasted in build time though, due to a lack of appropriate caching facilities with GitHub actions. [0] https://github.com/Barre/ZeroFS/tree/main/.github/workflows https://github.com/Barre/ZeroFS/tree/main/.github/workflows
  - featherless 10mo ago
    I found that implementing a local cache on the runners has been helpful. Ingress/egress on local network is hella slow, especially when each build has ~10-20GB of artifacts to manage.
    - esafak 10mo ago
      What do you use for the local cache?
      - featherless 10mo ago
        Just wrote about my approach yesterday: https://jeffverkoeyen.com/blog/2025/12/15/SlotWarmedCaching/ https://jeffverkoeyen.com/blog/2025/12/15/SlotWarmedCaching/ tl;dr uses a local slot-based cache that is pre-warmed after every merge to main, taking Sidecar builds from ~10-15 minutes to <60 seconds.
  - theLiminator 10mo ago
    Wow, that's a very cool project.
    - Eikon 10mo ago
      Thank you!
  - hedgehog 10mo ago
    ZeroFS looks really good. I know a bit about this design space but hadn't run across ZeroFS yet. Do you do testing of the error recovery behavior (connectivity etc)?
    - Eikon 10mo ago
      This has been mostly manual testing for now. ZeroFS currently lacks automatic fault injection and proper crash tests, and it’s an area I plan to focus on. SlateDB, the lower layer, already does DST as well as fault injection though.
- duped 10mo ago
  1 hour build/test time, 20 devs, that's 50 runs a month. Totally possible.
  - gheltllck 10mo ago
    GH actions templates don’t build all branches by default. I guess it’s due to them not wanting the free tier to use to much resources. But I consider it an anti-pattern to not build everything at each push.
    - sunnyday_002 10mo ago
      That is because GH Actions is event based, that is more powerful and flexible than just triggering on every push and not letting it be configured. ``` on: push ``` is the event trigger to act on every push. You'll waste a lot of CI on building everything in my opinion, I only really care about the pull request.
hinkley 10mo ago
Am I right in assuming it’s not the amount of payment but the transition from $0 to paying a bill at all? I’m definitely sure it’s saving me more than $140 a month to have CI/CD running and I’m also sure I’d never break even on the opportunity cost of having someone write or set one up internally if someone else’s works - and this is the key - just as well. But investment in CI/CD is investing in future velocity. The hours invested are paid for by hours saved. So if the outcome is brittle and requires oversight that savings drops or disappears.
- newsoftheday 10mo ago
  You're sounding a lot like a Microsoft damage control artist.
  - PeterHolzwarth 10mo ago
    Keep this kind of comment on reddit, not here.
    - newsoftheday 10mo ago
      I'll keep it where I like actually, thanks.
  - hinkley 10mo ago
    The only company I’ve held a grudge against longer than MS is McDonalds and they are sort of cut from the same cloth. I’m also someone who paid for JetBrains when everyone still thought it wasn’t worth money to pay for a code editor. Though I guess that’s again now. And everyone is using an MS product instead.
- featherless 10mo ago
  This is not investment in CI/CD. I already did that, by buying and investing in my own hardware, my own workflows, my own caching solution. This is like if Dropbox started charging you money for the files you have stored on your backup hard drives.

This is absolutely bananas; for my own CI workflow I'll have to pay $140+/month now just to run my own hardware.

hedgehog 10mo ago

I'm curious, what are you doing that has over 1000 hours a month of action runtime?

featherless 10mo ago
I run a local Valhalla build cluster to power the https://sidecar.clutch.engineering https://sidecar.clutch.engineering routing engine. The cluster runs daily and takes a significant amount of wall-clock time to build the entire planet. That's about 50% of my CI time; the other 50% is presubmits + App Store builds for Sidecar + CANStudio / ELMCheck. Using GitHub actions to coordinate the Valhalla builds was a nice-to-have, but this is a deal-breaker for my pull request workflows.
- hedgehog 10mo ago
  Cool, that looks a lot nicer than the OBD scanner app I've been using.
Eikon 10mo ago
On ZeroFS [0] I am doing around 80 000 minutes a month. A lot of it is wasted in build time though, due to a lack of appropriate caching facilities with GitHub actions. [0] https://github.com/Barre/ZeroFS/tree/main/.github/workflows https://github.com/Barre/ZeroFS/tree/main/.github/workflows
- featherless 10mo ago
  I found that implementing a local cache on the runners has been helpful. Ingress/egress on local network is hella slow, especially when each build has ~10-20GB of artifacts to manage.
  - esafak 10mo ago
    What do you use for the local cache?
    - featherless 10mo ago
      Just wrote about my approach yesterday: https://jeffverkoeyen.com/blog/2025/12/15/SlotWarmedCaching/ https://jeffverkoeyen.com/blog/2025/12/15/SlotWarmedCaching/ tl;dr uses a local slot-based cache that is pre-warmed after every merge to main, taking Sidecar builds from ~10-15 minutes to <60 seconds.
- theLiminator 10mo ago
  Wow, that's a very cool project.
  - Eikon 10mo ago
    Thank you!
- hedgehog 10mo ago
  ZeroFS looks really good. I know a bit about this design space but hadn't run across ZeroFS yet. Do you do testing of the error recovery behavior (connectivity etc)?
  - Eikon 10mo ago
    This has been mostly manual testing for now. ZeroFS currently lacks automatic fault injection and proper crash tests, and it’s an area I plan to focus on. SlateDB, the lower layer, already does DST as well as fault injection though.
duped 10mo ago
1 hour build/test time, 20 devs, that's 50 runs a month. Totally possible.
- gheltllck 10mo ago
  GH actions templates don’t build all branches by default. I guess it’s due to them not wanting the free tier to use to much resources. But I consider it an anti-pattern to not build everything at each push.
  - sunnyday_002 10mo ago
    That is because GH Actions is event based, that is more powerful and flexible than just triggering on every push and not letting it be configured. ``` on: push ``` is the event trigger to act on every push. You'll waste a lot of CI on building everything in my opinion, I only really care about the pull request.

hinkley 10mo ago

Am I right in assuming it’s not the amount of payment but the transition from $0 to paying a bill at all? I’m definitely sure it’s saving me more than $140 a month to have CI/CD running and I’m also sure I’d never break even on the opportunity cost of having someone write or set one up internally if someone else’s works - and this is the key - just as well. But investment in CI/CD is investing in future velocity. The hours invested are paid for by hours saved. So if the outcome is brittle and requires oversight that savings drops or disappears.

newsoftheday 10mo ago
You're sounding a lot like a Microsoft damage control artist.
- PeterHolzwarth 10mo ago
  Keep this kind of comment on reddit, not here.
  - newsoftheday 10mo ago
    I'll keep it where I like actually, thanks.
- hinkley 10mo ago
  The only company I’ve held a grudge against longer than MS is McDonalds and they are sort of cut from the same cloth. I’m also someone who paid for JetBrains when everyone still thought it wasn’t worth money to pay for a code editor. Though I guess that’s again now. And everyone is using an MS product instead.
featherless 10mo ago
This is not investment in CI/CD. I already did that, by buying and investing in my own hardware, my own workflows, my own caching solution. This is like if Dropbox started charging you money for the files you have stored on your backup hard drives.