6 ms·

So, I'm not interested in the debate about the correctness (or otherwise) of yaml as a declarative programming language, but I will say this... iterating a Git

by cmsj 2y ago

So, I'm not interested in the debate about the correctness (or otherwise) of yaml as a declarative programming language, but I will say this... iterating a GitHub Actions workflow is a gigantic pain in the ass. Capturing all of the important logic in a script/makefile/whatever means I can iterate it locally way faster and then all I need github to do is provision an environment and call my scripts in the order I require.

motorest 2y ago
> iterating a GitHub Actions workflow is a gigantic pain in the ass. Capturing all of the important logic in a script/makefile/whatever means I can iterate it locally way faster and then all I need github to do is provision an environment and call my scripts in the order I require. What's wrong with this? https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/adding-scripts-to-your-workflow https://docs.github.com/en/actions/writing-workflows/choosin...
- kbolino 2y ago
  When it gets realistic, with conditions, variable substitutions, etc., it ends up being 20 steps in a language that isn't shell but is calling shell over and over again, and can't be run outside of CI. Whereas, if you just wrote one shell script, it could've done all of those things in one language and been runnable locally too.
  - motorest 2y ago
    > When it gets realistic, with conditions, variable substitutions, etc., What exactly do you find hard in writing your own scripts with a scripting language? Surely you are not a software developer who feels conditionals and variable substitutions are hard. > it ends up being 20 steps in a language that isn't shell but is calling shell over and over again, and can't be run outside of CI. Why are you writing your CICD scripts in a way that you cannot run them outside of a CICD pipeline? I mean, you're writing them yourself, aren't you? Why are you failing to meet your own requirements? If you have a requirement to run your own scripts outside of a pipeline, how come you're not writing them like that? It's CICD 101 that those scripts should be runnable outside of the pipeline. From your description, you're failing to even follow the most basic recommendations and best practices. Why? That doesn't sound like a YAML problem, does it?
    - kbolino 2y ago
      This is not about YAML in some general or abstract sense, it is about a YAML-based domain-specific language. If you think this is just about YAML, you are hyper-focused on the wrong detail. In order to use this domain-specific language properly, you first must learn it, and learning YAML is but a small part of that. Moreover, it is not immediately obvious that, once you know it, you actually want to avoid it. But you can't avoid it entirely, because it is the core language of the CI/CD platform. And you can't know how to avoid it effectively until you have spent some time just using it directly. Simplicity comes from tearing away what is unnecessary, but to discern necessary from unnecessary requires judgment gained by experience. There is no world in which this knowledge transfers immediately, frictionlessly, and losslessly. Furthermore, there is a lot that GitHub (replace with platform of choice) could have done to make this better. They largely have no incentive to do so, because platform lock-in isn't a bad thing to the platform owner, and it's a nontrivial amount of work on their part, just as it is a nontrivial amount of work on your part to learn and use their platform in a way that doesn't lock you into it.
- tom_ 2y ago
  Nothing significant on the face of it and I think that's pretty much exactly what's being suggested: don't have anything particularly interesting in the .yml file, just the bare minimum plus some small number of uncomplicated script invocations to install dependencies and actually do the build. (Iterating even on this stuff by waiting for the runner is still annoying though. You need to commit to the repo, push, and wait. Hence the suggestion of having scripts that you can also run locally, so you can test changes locally when you're iterating on them. This isn't any kind of guarantee, but it's far less annoying to do (say) 15 iterations locally followed by the inevitable extra 3 remotely than it is having to do all 18 remotely, waiting for the runner each time then debugging it by staring at the output logs. Even assuming you'd be able to get away with as few as 15 given that you don't have proper access to the machine.)
  - bastardoperator 2y ago
    But GitHub recommends that, so if people don't follow best practices, and then complain when the docs are clear, who's at fault? The person writing against a system they don't understand because they haven't read the docs or the people who recommend what you're professing in the docs?
    - kbolino 2y ago
      > But GitHub recommends that Where?
      - bastardoperator 2y ago
        Here: https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/adding-scripts-to-your-workflow https://docs.github.com/en/actions/writing-workflows/choosin...
        kbolino 2y ago
        Nothing in that document states anything remotely like the antecedent of "that", which was: > don't have anything particularly interesting in the .yml file, just the bare minimum plus some small number of uncomplicated script invocations to install dependencies and actually do the build It is a very basic "how to" with no recommendations. Moreover, they directly illustrate a bad practice: - name: Run the scripts run: | ./my-script.sh ./my-other-script.sh This is not running two scripts, this is running a shell command that invokes two scripts, and has no error handling if the first one fails. If that's the behavior you want, fine, but then put it in one shell script, not two. What am I supposed to do with this locally? If the first shell script fails, do I need to fix it, or do I just proceed on to the second one?
- maratc 2y ago
  Q: How do you determine what date it was 180 days ago? A: Easy! You just spin up a Kubernetes pod with Alpine image, map a couple of files inside, run a bash script of "date" with some parameters, redirect output to a mapped file, and then read the resulting file. That's all. Here's a YAML for you. Configuration, baby! (based on actual events)
  - jiggawatts 2y ago
    At first I assumed you were kidding, then I realised that sadly… you probably weren’t.
    - maratc 2y ago
      I wasn’t. This goes to show that when all you have is a YAML hammer, every problem has to look like a YAML-able nail. Still there would be people who would say I’m “blaming my tools” and “everything is covered in chapter 1 of yaml for dummies.”

by cmsj 2y ago

motorest 2y ago

> iterating a GitHub Actions workflow is a gigantic pain in the ass. Capturing all of the important logic in a script/makefile/whatever means I can iterate it locally way faster and then all I need github to do is provision an environment and call my scripts in the order I require. What's wrong with this? https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/adding-scripts-to-your-workflow https://docs.github.com/en/actions/writing-workflows/choosin...

kbolino 2y ago
When it gets realistic, with conditions, variable substitutions, etc., it ends up being 20 steps in a language that isn't shell but is calling shell over and over again, and can't be run outside of CI. Whereas, if you just wrote one shell script, it could've done all of those things in one language and been runnable locally too.
- motorest 2y ago
  > When it gets realistic, with conditions, variable substitutions, etc., What exactly do you find hard in writing your own scripts with a scripting language? Surely you are not a software developer who feels conditionals and variable substitutions are hard. > it ends up being 20 steps in a language that isn't shell but is calling shell over and over again, and can't be run outside of CI. Why are you writing your CICD scripts in a way that you cannot run them outside of a CICD pipeline? I mean, you're writing them yourself, aren't you? Why are you failing to meet your own requirements? If you have a requirement to run your own scripts outside of a pipeline, how come you're not writing them like that? It's CICD 101 that those scripts should be runnable outside of the pipeline. From your description, you're failing to even follow the most basic recommendations and best practices. Why? That doesn't sound like a YAML problem, does it?
  - kbolino 2y ago
    This is not about YAML in some general or abstract sense, it is about a YAML-based domain-specific language. If you think this is just about YAML, you are hyper-focused on the wrong detail. In order to use this domain-specific language properly, you first must learn it, and learning YAML is but a small part of that. Moreover, it is not immediately obvious that, once you know it, you actually want to avoid it. But you can't avoid it entirely, because it is the core language of the CI/CD platform. And you can't know how to avoid it effectively until you have spent some time just using it directly. Simplicity comes from tearing away what is unnecessary, but to discern necessary from unnecessary requires judgment gained by experience. There is no world in which this knowledge transfers immediately, frictionlessly, and losslessly. Furthermore, there is a lot that GitHub (replace with platform of choice) could have done to make this better. They largely have no incentive to do so, because platform lock-in isn't a bad thing to the platform owner, and it's a nontrivial amount of work on their part, just as it is a nontrivial amount of work on your part to learn and use their platform in a way that doesn't lock you into it.
tom_ 2y ago
Nothing significant on the face of it and I think that's pretty much exactly what's being suggested: don't have anything particularly interesting in the .yml file, just the bare minimum plus some small number of uncomplicated script invocations to install dependencies and actually do the build. (Iterating even on this stuff by waiting for the runner is still annoying though. You need to commit to the repo, push, and wait. Hence the suggestion of having scripts that you can also run locally, so you can test changes locally when you're iterating on them. This isn't any kind of guarantee, but it's far less annoying to do (say) 15 iterations locally followed by the inevitable extra 3 remotely than it is having to do all 18 remotely, waiting for the runner each time then debugging it by staring at the output logs. Even assuming you'd be able to get away with as few as 15 given that you don't have proper access to the machine.)
- bastardoperator 2y ago
  But GitHub recommends that, so if people don't follow best practices, and then complain when the docs are clear, who's at fault? The person writing against a system they don't understand because they haven't read the docs or the people who recommend what you're professing in the docs?
  - kbolino 2y ago
    > But GitHub recommends that Where?
    - bastardoperator 2y ago
      Here: https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/adding-scripts-to-your-workflow https://docs.github.com/en/actions/writing-workflows/choosin...
      - kbolino 2y ago
        Nothing in that document states anything remotely like the antecedent of "that", which was: > don't have anything particularly interesting in the .yml file, just the bare minimum plus some small number of uncomplicated script invocations to install dependencies and actually do the build It is a very basic "how to" with no recommendations. Moreover, they directly illustrate a bad practice: - name: Run the scripts run: | ./my-script.sh ./my-other-script.sh This is not running two scripts, this is running a shell command that invokes two scripts, and has no error handling if the first one fails. If that's the behavior you want, fine, but then put it in one shell script, not two. What am I supposed to do with this locally? If the first shell script fails, do I need to fix it, or do I just proceed on to the second one?
maratc 2y ago
Q: How do you determine what date it was 180 days ago? A: Easy! You just spin up a Kubernetes pod with Alpine image, map a couple of files inside, run a bash script of "date" with some parameters, redirect output to a mapped file, and then read the resulting file. That's all. Here's a YAML for you. Configuration, baby! (based on actual events)
- jiggawatts 2y ago
  At first I assumed you were kidding, then I realised that sadly… you probably weren’t.
  - maratc 2y ago
    I wasn’t. This goes to show that when all you have is a YAML hammer, every problem has to look like a YAML-able nail. Still there would be people who would say I’m “blaming my tools” and “everything is covered in chapter 1 of yaml for dummies.”