4 ms·

We provision our self-hosted runners via Terraform and GitLab's Helm chart. It's a relatively painless setup, but obviously not a few button clicks and commands

mdaniel 4y ago
> a local execution story I can see how "local" can have multiple meanings, but here I meant "as a developer on my laptop, can I have local docker run things the way GL is going to run things?" I hear people say a lot "oh, I just use shell scripts, NBD" but as I said, I'm sure for hello-world setups which don't have any includes or take advantage of GLCI constructs that can work fine, but what rubs me the wrong way is that "gitlab-runner exec" doesn't say "just use shell scripts," it says "gitlab-runner exec - execute a build locally" and it for sure does not do that
- Volundr 4y ago
  Yeah this is a pain point for me as well, especially when developing the pipelines themselves. There's no good way to run it on my own machine, so I'm stuck pushing new commits to Gitlab over and over again and waiting for it to complete, or rather fail so I can troubleshoot the new error.
  - mdaniel 4y ago
    Depending on one's definition of "good," running GL locally in docker is (in my experience) painless, given sufficient disk space and probably RAM But yes, your observation is my whole complaint: it is not _reasonable_ to ask a GLCI developer to run a local copy of GL, complete with any shared GLCI template repos, in a local docker container just to have local execution. Maybe I wouldn't complain about it so much had I not started with circleci so long ago and had such a "wow, this is amazing" followed by gitlab-runner's :troll_face: -- to say nothing of GitHub Actions just straight up ignoring that whole demographic and hoping https://github.com/nektos/act https://github.com/nektos/act emulates enough to have people not notice the massive feature gap
    - neoromantique 4y ago
      unfortunately `act` is pretty bad :( My go to workflow for workflows (ha) is forking a repo as a private fork and toying on it until I'm happy with the results
  - eins1234 4y ago
    This thread makes me wonder if the root of the problem is actually the lack of a local runner. I hypothesize that it's the length of the feedback loop that's the real issue. Even if we had a local runner, if it takes a ton of time to start and complete every step, it'd be almost as painful to debug as a remote runner taking the same time. On the other hand, if the cloud runner is ridiculously fast, and completes steps on the order of single digit seconds, it would be fairly painless to debug.
    - Volundr 4y ago
      Not really. I mean yes my dev machine is faster than the CI runner, so things do run faster, but even assuming the CI runner was just as beefy and picked up the job immediately instead of taking 10ish seconds it's still the difference between When running remotely: edit file > commit > push > switch to browser > browse to pipeline > repeat Vs locally: edit file > run pipeline > repeat.
  - rmetzler 4y ago
    If you have access to the Kubernetes cluster or wherever your runner is running: put a sleep right before the error, execute into the running pod and debug it. I like to do gitlab-runner exec docker … locally, but it doesn’t work with includes and you need to set up your own variables.
- goodoldneon 4y ago
  Ah, I see what you're saying. I've had to do some debugging in GitLab instead of local and it wasn't ideal. To prevent mucking with important stuff, I have a scratch project that's dedicated to testing things. It isn't perfect, but I've been able to replicate nearly every problem in the scratch project

mdaniel 4y ago

> a local execution story I can see how "local" can have multiple meanings, but here I meant "as a developer on my laptop, can I have local docker run things the way GL is going to run things?" I hear people say a lot "oh, I just use shell scripts, NBD" but as I said, I'm sure for hello-world setups which don't have any includes or take advantage of GLCI constructs that can work fine, but what rubs me the wrong way is that "gitlab-runner exec" doesn't say "just use shell scripts," it says "gitlab-runner exec - execute a build locally" and it for sure does not do that

Volundr 4y ago
Yeah this is a pain point for me as well, especially when developing the pipelines themselves. There's no good way to run it on my own machine, so I'm stuck pushing new commits to Gitlab over and over again and waiting for it to complete, or rather fail so I can troubleshoot the new error.
- mdaniel 4y ago
  Depending on one's definition of "good," running GL locally in docker is (in my experience) painless, given sufficient disk space and probably RAM But yes, your observation is my whole complaint: it is not _reasonable_ to ask a GLCI developer to run a local copy of GL, complete with any shared GLCI template repos, in a local docker container just to have local execution. Maybe I wouldn't complain about it so much had I not started with circleci so long ago and had such a "wow, this is amazing" followed by gitlab-runner's :troll_face: -- to say nothing of GitHub Actions just straight up ignoring that whole demographic and hoping https://github.com/nektos/act https://github.com/nektos/act emulates enough to have people not notice the massive feature gap
  - neoromantique 4y ago
    unfortunately `act` is pretty bad :( My go to workflow for workflows (ha) is forking a repo as a private fork and toying on it until I'm happy with the results
- eins1234 4y ago
  This thread makes me wonder if the root of the problem is actually the lack of a local runner. I hypothesize that it's the length of the feedback loop that's the real issue. Even if we had a local runner, if it takes a ton of time to start and complete every step, it'd be almost as painful to debug as a remote runner taking the same time. On the other hand, if the cloud runner is ridiculously fast, and completes steps on the order of single digit seconds, it would be fairly painless to debug.
  - Volundr 4y ago
    Not really. I mean yes my dev machine is faster than the CI runner, so things do run faster, but even assuming the CI runner was just as beefy and picked up the job immediately instead of taking 10ish seconds it's still the difference between When running remotely: edit file > commit > push > switch to browser > browse to pipeline > repeat Vs locally: edit file > run pipeline > repeat.
- rmetzler 4y ago
  If you have access to the Kubernetes cluster or wherever your runner is running: put a sleep right before the error, execute into the running pod and debug it. I like to do gitlab-runner exec docker … locally, but it doesn’t work with includes and you need to set up your own variables.
goodoldneon 4y ago
Ah, I see what you're saying. I've had to do some debugging in GitLab instead of local and it wasn't ideal. To prevent mucking with important stuff, I have a scratch project that's dedicated to testing things. It isn't perfect, but I've been able to replicate nearly every problem in the scratch project