5 ms·

I think this post accurately isolates the single main issue with GitHub Actions, i.e. the lack of a tight feedback loop. Pushing and waiting for completion on w

I think this post accurately isolates the single main issue with GitHub Actions, i.e. the lack of a tight feedback loop. Pushing and waiting for completion on what's often a very simple failure mode is frustrating. Others have pointed out that there are architectural steps you can take to minimize this pain, like keeping all CI operations isolated within scripts that can be run locally (and treating GitHub Actions features purely as progressive enhancements, e.g. only using `GITHUB_STEP_SUMMARY` if actually present). Another thing that works pretty well to address the feedback loop pain is `workflow_dispatch` + `gh workflow run`: you still need to go through a push cycle, but `gh workflow run` lets you stay in development flow until you actually need to go look at the logs. (One frustrating limitation with that is that `gh workflow run` doesn't actually spit out the URL of the workflow run it triggers. GitHub claims this is because it's an async dispatch, but I don't see how there can possibly be no context for GitHub to provide here, given that they clearly obtain it later in the web UI.)

echelon 10mo ago
We need SSH access to the failed instances so we can poke around and iterate from any step in the workflow. Production runs should be immutable, but we should be able to get in to diagnose, edit, and retry. It'd lead to faster diagnosis, resolution, and fixing. The logs and everything should be there for us. And speaking of the logs situation, the GHA logs are really buggy sometimes. They don't load about half of the time I need them to.
- dweekly 10mo ago
  Are there solutions to this like https://github.com/marketplace/actions/ssh-to-github-action-runner https://github.com/marketplace/actions/ssh-to-github-action-... ?
- lawrencegripper 10mo ago
  I wrote something recently with webrtc to get terminal on failure: https://blog.gripdev.xyz/2026/01/10/actions-terminal-on-failure-for-debugging/ https://blog.gripdev.xyz/2026/01/10/actions-terminal-on-fail...
maliker 10mo ago
I've standardized on getting github actions to create/pull a docker image and run build/test inside that. So if something goes wrong I have a decent live debug environment that's very similar to what github actions is running. For what it's worth.
- Storment33 10mo ago
  I was doing something similar when moving from Earthly. But I have since moved to Nix to manage the environment. It is a lot better of a developer experience and faster! I would checkout an environment manager like Nix/Mise etc so you can have the same tools etc locally and on CI.
- woodruffw 10mo ago
  Yeah, images seem to work very well as an abstraction layer for most CI/CD users. It's kind of unfortunate that they don't (can't) fully generalize across Windows and macOS runners as well, though, since in practice that's where a lot of people start to get snagged by needing to do things in GitHub Actions versus using GitHub Actions as an execution layer.
- steeleduncan 10mo ago
  I do the same with Nix as it works for macOS builds as well It has the massive benefit of solving the lock-in problem. Your workflow is generally very short so it is easy to move to an alternative CI if (for example) Github were to jack up their prices for self hosted runners... That said, when using it in this way I personally love Github actions
  - maliker 10mo ago
    I'm scared by all these references to nix in the replies here. Sounds like I'm going to have learn nix. Sounds hard.
    - steeleduncan 10mo ago
      Gemini/ChatGPT help (a lot) when getting going. They make up for the poor documentation
anttiharju 10mo ago
> i.e. the lack of a tight feedback loop. Lefthook helps a lot https://anttiharju.dev/a/1#pre-commit-hooks-are-useful https://anttiharju.dev/a/1#pre-commit-hooks-are-useful Thing is that people are not willing to invest in it due to bad experiences with various git hooks, but there are ways to have it be excellent
- woodruffw 10mo ago
  Yeah, I'm one of those people who seems to consistently have middling-to-bad experiences with Git hooks (and Git hook managers). I think the bigger issue is that even with consistent developer tooling between both developer machines and CI, you still have the issue where CI needs to do a lot more stuff that local machines just can't/won't do (like matrix builds). Those things are fundamentally remote and kind of annoying to debug, but GitHub could invest a lot more in reducing the frustration involved in getting a fast remote cycle set up.
  - anttiharju 10mo ago
    GitHub could invest a lot more in actions for sure. Even just in basic stuff like actions/checkout@v6 being broken for self-hosted runners.
cedws 10mo ago
I’ve never used Nix and frankly am a sceptic, but can it solve this problem by efficiently caching steps?
tagraves 10mo ago
This is one of the big problems we solved with the RWX CI platform (RWX.com). You can use ‘rwx run’ and it automatically syncs your local changes, so no need to push — and with our automated caching, steps like setting up the environment cache hit so you don’t have to execute the same stuff over and over again while writing your workflow. Plus with our API or MCP server you can get the results directly in your terminal so no need to open the UI at all unless you want to do some in-depth spelunking.
iguessthislldo 10mo ago
I've never used gh workflow run, but I have used the GitHub API to run workflows and wanted to show the URL. I had to have it make another call to get the workflow runs and assume the last run is the one with the correct URL. This would obviously not work correctly if there were multiple run requests at the same time. Maybe some more checking could detect that, but it works for my purposes so far.
- ezekiel68 10mo ago
  Does the metadata in the further call not identify the branch/start time/some other useful info that could help disambiguate this? (honest question)
hinkley 10mo ago
I’ve contemplated building my own CI tool (with a local runner) and the thing is if you assume “write a pipeline that runs locally but also on push”, then the feature depth is mostly about queuing, analyzing output, and often left off, but IMO important, charting telemetry about the build history. Most of these are off the shelf, at least in some programming languages. It’s the integrations and the overmanagement where a lot of the weight is.
- irishcoffee 10mo ago
  I think you described Jenkins, which is infinitely better than GitHub runners.
  - mlrtime 10mo ago
    Jenkins has it's own set of issues. The theory behind GHA is you *should* be able to keep everything in git and not need a another service with it's own abstractions. But Actions just screws this up. I consider them both to be equally bad. I'm almost considering n8n to replace both but that will just lead to more problems.
    - irishcoffee 10mo ago
      All my Jenkins builds are between 1 and 3 lines depending on readability: Clone Cd $clonedFolder ./build.sh Add an archive step, all set. I will say even building and running unit testing in Jenkins isn’t bad, but if you aren’t careful it can get messy. Jenkins can be a super simple build tool. Strong-arming it into doing all the things starts to get messy quickly. Build, run unit tests, archive. I’m not even big on like, building on push. Nothing wrong with it, I’m just big on keeping things as simple as possible. GHA is just a ball of complexity.
modeless 10mo ago
It's insane to me that being able to run CI steps locally is not the first priority of every CI system. It ought to be a basic requirement.
- duped 10mo ago
  This goes against every incentive for the CI service provider
  - anon7000 10mo ago
    Not necessarily. For example, Buildkite lets you host your own runners.
    - deleted 10mo ago
      [deleted]
- mrinterweb 10mo ago
  I've often thought about this. There are times I would rather have CI run locally, and use my PGP signature to add a git note to the commit. Something like: ``` echo "CI passed" | gpg2 --clearsign --output=- | git notes add -F- ``` Then CI could check git notes and check the dev signature, and skip the workflow/pipeline if correctly signed. With more local CI, the incentive may shift to buying devs fancier machines instead of spending that money on cloud CI. I bet most devs have extra cores to spare and would not mind having a beefier dev machine.
  - teaearlgraycold 10mo ago
    "Works on my machine!"
  - akoboldfrying 10mo ago
    I think this is a sound approach, but I do see one legitimate reason to keep using a third-party CI service: reducing the chance of a software supply chain attack by building in a hardened environment that has (presumably) had attention from security people. I'd say the importance of this is increasing.
mmcnl 10mo ago
But very often the CI operations _are_ the problem. It's just YAML files with unlimited configuration options that have very limited documentation, without any type of LSP.
smw 10mo ago
https://github.com/nektos/act https://github.com/nektos/act Lets you run your actions locally. I've had significant success with it for fast local feedback.
- sestep 10mo ago
  I tried this five years ago back when I was an engineer on the PyTorch project, and it didn't work well enough to be worth it. Has it improved since then?
  - tonymet 10mo ago
    it has. it's improved to work with ~ 75% of steps . fast enough to worth trying before push
  - conception 10mo ago
    It works well enough that I didn’t realize this wasn’t first party till right now.
    - manquer 10mo ago
      It works, but there are fair amount of caveats, especially for someone working on things like Pytorch, the runtime is close but not the same, and its support of certain architectures etc can create annoying bugs.
  - mlrtime 10mo ago
    For me, no. Spend days trying to get it to recreate a production environment workflow. It is too different than production.
- danpalmer 10mo ago
  I tried this recently and it seems like you have to make a lot of decisions to support Act. It in no way "just works", but instead requires writing actions knowing that they'll run on Act.
- mlrtime 10mo ago
nine_k 10mo ago
I wonder what prevents a GH action from connecting to your VPN (Wireguard is fine) and post tons of diagnostics right onto your screen, and then, when something goes badly wrong, or when a certain point is reached, to just keep polling an HTTP endpoint for shell commands to execute. I mean. I understand, it would time out eventually. But it may be enough time to interactively check a few things right inside the running task's process. Of course this should only happen if the PR contains a file that says where to connect and when to stop for interactive input. You would only push such a file when an action is misbehaving, and you want to debug it. I understand that it's a band-aid, but a band-aid is better than the nothing which is available right now.

echelon 10mo ago

We need SSH access to the failed instances so we can poke around and iterate from any step in the workflow. Production runs should be immutable, but we should be able to get in to diagnose, edit, and retry. It'd lead to faster diagnosis, resolution, and fixing. The logs and everything should be there for us. And speaking of the logs situation, the GHA logs are really buggy sometimes. They don't load about half of the time I need them to.

dweekly 10mo ago
Are there solutions to this like https://github.com/marketplace/actions/ssh-to-github-action-runner https://github.com/marketplace/actions/ssh-to-github-action-... ?
lawrencegripper 10mo ago
I wrote something recently with webrtc to get terminal on failure: https://blog.gripdev.xyz/2026/01/10/actions-terminal-on-failure-for-debugging/ https://blog.gripdev.xyz/2026/01/10/actions-terminal-on-fail...

maliker 10mo ago

I've standardized on getting github actions to create/pull a docker image and run build/test inside that. So if something goes wrong I have a decent live debug environment that's very similar to what github actions is running. For what it's worth.

Storment33 10mo ago
I was doing something similar when moving from Earthly. But I have since moved to Nix to manage the environment. It is a lot better of a developer experience and faster! I would checkout an environment manager like Nix/Mise etc so you can have the same tools etc locally and on CI.
woodruffw 10mo ago
Yeah, images seem to work very well as an abstraction layer for most CI/CD users. It's kind of unfortunate that they don't (can't) fully generalize across Windows and macOS runners as well, though, since in practice that's where a lot of people start to get snagged by needing to do things in GitHub Actions versus using GitHub Actions as an execution layer.
steeleduncan 10mo ago
I do the same with Nix as it works for macOS builds as well It has the massive benefit of solving the lock-in problem. Your workflow is generally very short so it is easy to move to an alternative CI if (for example) Github were to jack up their prices for self hosted runners... That said, when using it in this way I personally love Github actions
- maliker 10mo ago
  I'm scared by all these references to nix in the replies here. Sounds like I'm going to have learn nix. Sounds hard.
  - steeleduncan 10mo ago
    Gemini/ChatGPT help (a lot) when getting going. They make up for the poor documentation

anttiharju 10mo ago

> i.e. the lack of a tight feedback loop. Lefthook helps a lot https://anttiharju.dev/a/1#pre-commit-hooks-are-useful https://anttiharju.dev/a/1#pre-commit-hooks-are-useful Thing is that people are not willing to invest in it due to bad experiences with various git hooks, but there are ways to have it be excellent

woodruffw 10mo ago
Yeah, I'm one of those people who seems to consistently have middling-to-bad experiences with Git hooks (and Git hook managers). I think the bigger issue is that even with consistent developer tooling between both developer machines and CI, you still have the issue where CI needs to do a lot more stuff that local machines just can't/won't do (like matrix builds). Those things are fundamentally remote and kind of annoying to debug, but GitHub could invest a lot more in reducing the frustration involved in getting a fast remote cycle set up.
- anttiharju 10mo ago
  GitHub could invest a lot more in actions for sure. Even just in basic stuff like actions/checkout@v6 being broken for self-hosted runners.

cedws 10mo ago

I’ve never used Nix and frankly am a sceptic, but can it solve this problem by efficiently caching steps?

tagraves 10mo ago

This is one of the big problems we solved with the RWX CI platform (RWX.com). You can use ‘rwx run’ and it automatically syncs your local changes, so no need to push — and with our automated caching, steps like setting up the environment cache hit so you don’t have to execute the same stuff over and over again while writing your workflow. Plus with our API or MCP server you can get the results directly in your terminal so no need to open the UI at all unless you want to do some in-depth spelunking.

iguessthislldo 10mo ago

I've never used gh workflow run, but I have used the GitHub API to run workflows and wanted to show the URL. I had to have it make another call to get the workflow runs and assume the last run is the one with the correct URL. This would obviously not work correctly if there were multiple run requests at the same time. Maybe some more checking could detect that, but it works for my purposes so far.

ezekiel68 10mo ago
Does the metadata in the further call not identify the branch/start time/some other useful info that could help disambiguate this? (honest question)

hinkley 10mo ago

I’ve contemplated building my own CI tool (with a local runner) and the thing is if you assume “write a pipeline that runs locally but also on push”, then the feature depth is mostly about queuing, analyzing output, and often left off, but IMO important, charting telemetry about the build history. Most of these are off the shelf, at least in some programming languages. It’s the integrations and the overmanagement where a lot of the weight is.

irishcoffee 10mo ago
I think you described Jenkins, which is infinitely better than GitHub runners.
- mlrtime 10mo ago
  Jenkins has it's own set of issues. The theory behind GHA is you *should* be able to keep everything in git and not need a another service with it's own abstractions. But Actions just screws this up. I consider them both to be equally bad. I'm almost considering n8n to replace both but that will just lead to more problems.
  - irishcoffee 10mo ago
    All my Jenkins builds are between 1 and 3 lines depending on readability: Clone Cd $clonedFolder ./build.sh Add an archive step, all set. I will say even building and running unit testing in Jenkins isn’t bad, but if you aren’t careful it can get messy. Jenkins can be a super simple build tool. Strong-arming it into doing all the things starts to get messy quickly. Build, run unit tests, archive. I’m not even big on like, building on push. Nothing wrong with it, I’m just big on keeping things as simple as possible. GHA is just a ball of complexity.

modeless 10mo ago

It's insane to me that being able to run CI steps locally is not the first priority of every CI system. It ought to be a basic requirement.

duped 10mo ago
This goes against every incentive for the CI service provider
- anon7000 10mo ago
  Not necessarily. For example, Buildkite lets you host your own runners.
  - deleted 10mo ago
    [deleted]
mrinterweb 10mo ago
I've often thought about this. There are times I would rather have CI run locally, and use my PGP signature to add a git note to the commit. Something like: ``` echo "CI passed" | gpg2 --clearsign --output=- | git notes add -F- ``` Then CI could check git notes and check the dev signature, and skip the workflow/pipeline if correctly signed. With more local CI, the incentive may shift to buying devs fancier machines instead of spending that money on cloud CI. I bet most devs have extra cores to spare and would not mind having a beefier dev machine.
- teaearlgraycold 10mo ago
  "Works on my machine!"
- akoboldfrying 10mo ago
  I think this is a sound approach, but I do see one legitimate reason to keep using a third-party CI service: reducing the chance of a software supply chain attack by building in a hardened environment that has (presumably) had attention from security people. I'd say the importance of this is increasing.

mmcnl 10mo ago

But very often the CI operations _are_ the problem. It's just YAML files with unlimited configuration options that have very limited documentation, without any type of LSP.

smw 10mo ago

https://github.com/nektos/act https://github.com/nektos/act Lets you run your actions locally. I've had significant success with it for fast local feedback.

sestep 10mo ago
I tried this five years ago back when I was an engineer on the PyTorch project, and it didn't work well enough to be worth it. Has it improved since then?
- tonymet 10mo ago
  it has. it's improved to work with ~ 75% of steps . fast enough to worth trying before push
- conception 10mo ago
  It works well enough that I didn’t realize this wasn’t first party till right now.
  - manquer 10mo ago
    It works, but there are fair amount of caveats, especially for someone working on things like Pytorch, the runtime is close but not the same, and its support of certain architectures etc can create annoying bugs.
- mlrtime 10mo ago
  For me, no. Spend days trying to get it to recreate a production environment workflow. It is too different than production.
danpalmer 10mo ago
I tried this recently and it seems like you have to make a lot of decisions to support Act. It in no way "just works", but instead requires writing actions knowing that they'll run on Act.
mlrtime 10mo ago

nine_k 10mo ago

I wonder what prevents a GH action from connecting to your VPN (Wireguard is fine) and post tons of diagnostics right onto your screen, and then, when something goes badly wrong, or when a certain point is reached, to just keep polling an HTTP endpoint for shell commands to execute. I mean. I understand, it would time out eventually. But it may be enough time to interactively check a few things right inside the running task's process. Of course this should only happen if the PR contains a file that says where to connect and when to stop for interactive input. You would only push such a file when an action is misbehaving, and you want to debug it. I understand that it's a band-aid, but a band-aid is better than the nothing which is available right now.