5 ms·

The biggest problem with Actions that I see a lot of people struggle with is that you can’t easily debug things when they aren’t going well. I think they need

The biggest problem with Actions that I see a lot of people struggle with is that you can’t easily debug things when they aren’t going well. I think they need to add some way for users to be able to ssh into the worker. Maybe each time an action step fails you have 5 mins to ssh into the container and inspect what’s going on.

deleted 5y ago
[deleted]
trinovantes 5y ago
Yea setting up Actions is a pretty awful experience. I had to constantly git add, commit --amend, force push and retrigger the run and look at print statements.
milliams 5y ago
I really like that with GitLab you can run the CI script in a local container to debug things and then only commit and push once it's working.
- petethepig 5y ago
  That's an even better solution.
- derefr 5y ago
  GitHub doesn't have exactly that, but it does have self-hosted workflow runners (which GitLab also has, as it happens.) If you run a local self-hosted runner, point your workflow at it, and trigger it (push/etc), then the job bounces from GitHub's backend over to your local runner, where you can then poke / prod / trace its execution. It's not quite as "safe" — you don't get to test the workflow in advance of deploying it, but rather have to test it by deploying it. But it's still pretty useful for debugging. You can e.g. set environment variables on the runner itself, that then propagate into the job, to see how they affect the job. Or change stuff in the runner's user's home directory until the job works, to figure out what the job needs there, rather than deploying over and over with different "tweak this file or that" steps.
rvieira 5y ago
I'm not that familiar with GH actions, but Sourcehut has a couple of pretty nice solutions for troubleshooting: (as you mentioned) time-limited SSH into the build server and a page to edit the manifest online and re-run without having to push anything.
degraafc 5y ago
I use tmate[1] to get a shell on the worker when I need to debug things interactively. Also act[2] lets you run a decent approximation of your actions locally. Still agree that it should be easier and not require third-party tools/actions, though. [1]: https://github.com/mxschmitt/action-tmate https://github.com/mxschmitt/action-tmate [2]: https://github.com/nektos/act https://github.com/nektos/act
- jrochkind1 5y ago
  action-tmate... woah!
- karlicoss 5y ago
  Yeah, the only downside of tmate is that you need to add/uncomment the line with it, push, then remove the commit again. It's much nicer in circleci, for example where you can simply rerun wish SSH in a button click. Even better would be if the VM was around for a bit after failing, so I could connect and inspect the state without rerunning at all.
  - degraafc 5y ago
    Yep, definitely agree. I feel like there are a lot of areas that GitHub Actions needs improvements to catch up to other CI providers, really basic-seeming stuff like "allow failure" support and so on.
  - horse666 5y ago
    One approach to avoid this karlicoss is to add a “workflow dispatch” section to the “on” events in the workflow: workflow_dispatch: inputs: debug_enabled: description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)' required: false default: false Then under the job steps, make the debug step conditional on that optional flag: # Enable tmate debugging of manually-triggered workflows if the input option was provided - name: Setup tmate session uses: mxschmitt/action-tmate@v3 if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.debug_enabled }} You then can just trigger the build on the required branch, setting debug_enabled to true and voila. Edit: but agree the CircleCI interface is nicer where you can just run a debug build with ssh access directly.
deleted 5y ago
[deleted]
crazysim 5y ago
https://tunshell.com/ https://tunshell.com/ works great for *nix runners. I made https://github.com/nelsonjchen/reverse-rdp-windows-github-actions https://github.com/nelsonjchen/reverse-rdp-windows-github-ac... for the Windows runners. Just add continue on error to the failing step and use temporarily use these kinds of tools in the next step(s).
- kevincox 5y ago
  > https://tunshell.com/ https://tunshell.com/ works great for nix runners. Oh and their interface is a `curl | sh` that execs whatever happens to be lying around in `/tmp/tunshell/client`. I guess* that is probably fine on a single-user system but I prefer not to run binaries that just about every process on my system can write to. https://lets.tunshell.com/init.sh https://lets.tunshell.com/init.sh Edit: I filed a bug https://github.com/TimeToogo/tunshell/issues/19 https://github.com/TimeToogo/tunshell/issues/19
  - derefr 5y ago
    > I guess that is probably fine on a single-user system Most of these services run your workload inside a container, so it's not even a single-user system; it's an ephemeral system. It's like anything's going to be there other than the base image + what you put there.
    - kevincox 5y ago
      This same script is used for the server and client. So you are running this on your workstation.
kody 5y ago
CircleCI lets you rerun a pipeline with ssh enabled so you can debug. It's helpful unless your failed step is at the end of a complex 20 minute deployment. I'll usually run my docker image locally, copy over the source, and run the pipeline by hand to debug, but then I have to set up all the environment variables that I get from CircleCI/Bitbucket Pipelines, and even then there's no guarantee that my local tests are identical to what I'm seeing in the pipeline (just today I hit a memory limit in Bitbucket pipelines that I couldn't recreate locally).
Waterluvian 5y ago
The ability to ssh into the Travis VM and investigate the terminal state on failure is just amazing.
seniorThrowaway 5y ago
You could create a self-hosted runner to test things out on. I'm not sure how hard it is to match Github's VM configuration but I don't think they veer far from the stock ubuntu config for instance.
sbaildon 5y ago
Post-fail ssh access is one of the headline features of https://builds.sr.ht https://builds.sr.ht
ollien 5y ago
I've used https://github.com/nektos/act https://github.com/nektos/act in the past. It works ok as long as you don't need artifacting.
OJFord 5y ago
I mostly struggle with the docs to be honest. But yeah, SSHing in is great. I miss that from Circle using GL & GH.

deleted 5y ago

[deleted]

trinovantes 5y ago

Yea setting up Actions is a pretty awful experience. I had to constantly git add, commit --amend, force push and retrigger the run and look at print statements.

milliams 5y ago

I really like that with GitLab you can run the CI script in a local container to debug things and then only commit and push once it's working.

petethepig 5y ago
That's an even better solution.
derefr 5y ago
GitHub doesn't have exactly that, but it does have self-hosted workflow runners (which GitLab also has, as it happens.) If you run a local self-hosted runner, point your workflow at it, and trigger it (push/etc), then the job bounces from GitHub's backend over to your local runner, where you can then poke / prod / trace its execution. It's not quite as "safe" — you don't get to test the workflow in advance of deploying it, but rather have to test it by deploying it. But it's still pretty useful for debugging. You can e.g. set environment variables on the runner itself, that then propagate into the job, to see how they affect the job. Or change stuff in the runner's user's home directory until the job works, to figure out what the job needs there, rather than deploying over and over with different "tweak this file or that" steps.

rvieira 5y ago

I'm not that familiar with GH actions, but Sourcehut has a couple of pretty nice solutions for troubleshooting: (as you mentioned) time-limited SSH into the build server and a page to edit the manifest online and re-run without having to push anything.

degraafc 5y ago

I use tmate[1] to get a shell on the worker when I need to debug things interactively. Also act[2] lets you run a decent approximation of your actions locally. Still agree that it should be easier and not require third-party tools/actions, though. [1]: https://github.com/mxschmitt/action-tmate https://github.com/mxschmitt/action-tmate [2]: https://github.com/nektos/act https://github.com/nektos/act

jrochkind1 5y ago
action-tmate... woah!
karlicoss 5y ago
Yeah, the only downside of tmate is that you need to add/uncomment the line with it, push, then remove the commit again. It's much nicer in circleci, for example where you can simply rerun wish SSH in a button click. Even better would be if the VM was around for a bit after failing, so I could connect and inspect the state without rerunning at all.
- degraafc 5y ago
  Yep, definitely agree. I feel like there are a lot of areas that GitHub Actions needs improvements to catch up to other CI providers, really basic-seeming stuff like "allow failure" support and so on.
- horse666 5y ago
  One approach to avoid this karlicoss is to add a “workflow dispatch” section to the “on” events in the workflow: workflow_dispatch: inputs: debug_enabled: description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)' required: false default: false Then under the job steps, make the debug step conditional on that optional flag: # Enable tmate debugging of manually-triggered workflows if the input option was provided - name: Setup tmate session uses: mxschmitt/action-tmate@v3 if: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.debug_enabled }} You then can just trigger the build on the required branch, setting debug_enabled to true and voila. Edit: but agree the CircleCI interface is nicer where you can just run a debug build with ssh access directly.

deleted 5y ago

[deleted]

crazysim 5y ago

https://tunshell.com/ https://tunshell.com/ works great for *nix runners. I made https://github.com/nelsonjchen/reverse-rdp-windows-github-actions https://github.com/nelsonjchen/reverse-rdp-windows-github-ac... for the Windows runners. Just add continue on error to the failing step and use temporarily use these kinds of tools in the next step(s).

kevincox 5y ago
> https://tunshell.com/ https://tunshell.com/ works great for nix runners. Oh and their interface is a `curl | sh` that execs whatever happens to be lying around in `/tmp/tunshell/client`. I guess* that is probably fine on a single-user system but I prefer not to run binaries that just about every process on my system can write to. https://lets.tunshell.com/init.sh https://lets.tunshell.com/init.sh Edit: I filed a bug https://github.com/TimeToogo/tunshell/issues/19 https://github.com/TimeToogo/tunshell/issues/19
- derefr 5y ago
  > I guess that is probably fine on a single-user system Most of these services run your workload inside a container, so it's not even a single-user system; it's an ephemeral system. It's like anything's going to be there other than the base image + what you put there.
  - kevincox 5y ago
    This same script is used for the server and client. So you are running this on your workstation.

kody 5y ago

CircleCI lets you rerun a pipeline with ssh enabled so you can debug. It's helpful unless your failed step is at the end of a complex 20 minute deployment. I'll usually run my docker image locally, copy over the source, and run the pipeline by hand to debug, but then I have to set up all the environment variables that I get from CircleCI/Bitbucket Pipelines, and even then there's no guarantee that my local tests are identical to what I'm seeing in the pipeline (just today I hit a memory limit in Bitbucket pipelines that I couldn't recreate locally).

Waterluvian 5y ago

The ability to ssh into the Travis VM and investigate the terminal state on failure is just amazing.

seniorThrowaway 5y ago

You could create a self-hosted runner to test things out on. I'm not sure how hard it is to match Github's VM configuration but I don't think they veer far from the stock ubuntu config for instance.

sbaildon 5y ago

Post-fail ssh access is one of the headline features of https://builds.sr.ht https://builds.sr.ht

ollien 5y ago

I've used https://github.com/nektos/act https://github.com/nektos/act in the past. It works ok as long as you don't need artifacting.

OJFord 5y ago

I mostly struggle with the docs to be honest. But yeah, SSHing in is great. I miss that from Circle using GL & GH.