3 ms·

I think YAML anchors in GitHub Actions are very welcome, for example for DRYing the push/pull_request 'paths:' filters [1]. Now only if they supported paths fi

I think YAML anchors in GitHub Actions are very welcome, for example for DRYing the push/pull_request 'paths:' filters [1]. Now only if they supported paths filter for `workflow_call` [2] event in addition to push/pull_request and my life would be a lot easier. Nontrivial repos have an unfortunate habit of building some sort of broken version of change detection themselves. The limit of 20 unique workflow calls is quite low too but searching the docs for a source maybe they have removed it? It used to say > You can call a maximum of 20 unique reusable workflows from a single workflow file. but now it's max of 4 nested workflows without loops, which gives a lot of power for the more complex repos [3]. Ooh. Need to go test this. [1] https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#example-including-paths https://docs.github.com/en/actions/reference/workflows-and-a... [2] https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#onworkflow_call https://docs.github.com/en/actions/reference/workflows-and-a... [3] https://docs.github.com/en/actions/how-tos/reuse-automations/reuse-workflows#nesting-reusable-workflows https://docs.github.com/en/actions/how-tos/reuse-automations...

TheDong 1y ago
Wanna DRY out your github actions yaml? Generate it from Dhall, or cue, or python, or some real language that supports actual abstractions. If your problem is you want to DRY out yaml, and you use more yaml features to do it, you now have more problems, not fewer.
- usrme 1y ago
  Seconded. I've had huge success with generating workflows with CUE. Would definitely recommend it to anyone struggling with YAML.
- Spivak 1y ago
  Just introduce a templating language and the CI to generate the yaml as part of your pipeline as way to make it simpler? Above a certain level of complexity, sure. But having nothing in between is an annoying state of affairs. I use anchors in Gitlab pipelines and I hardly curse their names.
- nirvdrum 1y ago
  Having used CI systems and application frameworks that support YAML anchors for configuration, adding in a programming language would be a massive amount of complexity for very little gain. We're not talking about dozens of locations with hundreds of lines of shared code. Asking the team to add a new build dependency, learn a new language, and add a new build step would create considerably more problems, not fewer. Used sparingly and as needed, YAML anchors are quite easy to read. A good editor will even allow you to jump to the source definition just as it would any other variable. Being self-contained without any additional dependencies is a huge advantage, particularly for open source projects, IMHO. I'd wager very few people are going to learn Dhall in order to fix an issue with an open source project's CI.
  - baq 1y ago
    Your team doesn't know YAML, it knows github actions. There's zero transferable knowledge when switching from github actions to kubernetes deployments, as there is precisely the same zero correlation between kubernetes and ansible configs. 'It's all YAML' is a lie and I'm continuously surprised so many people are falling for it for so long. YAML is the code-as-data, but the interpreter determines what it all means.
    - nirvdrum 1y ago
      Oh, for goodness' sake. We know YAML syntax and that's the only part that's relevant here. Pointing out that different software uses different keys for their configuration or even takes different actions for keys that happen to share the same name isn't particularly insightful. We haven't been bamboozled.
      - baq 1y ago
        I don’t understand even more now. If you freely admit they’re different languages, the only reason to keep using the stupid deficient syntax is momentum, and while it isn’t a bad reason, it is costing you and everyone else in the long run.
        1y ago
- phanimahesh 1y ago
  This. So true. Yaml has always been an overly complicated format, with weird quirks ( like norway becoming false in a list of country codes ). I find it an absolute shame that languages like Dhall did not become more popular earlier. Now everything in devops is yaml, and I think many developers pick yaml configs not out of good reasons but defaulting to its ubiquity as sufficient.
  - true_blue 1y ago
    >norway yaml 1.2 was released in 2009, and it fixed this problem. this is an implementation issue. https://yaml.org/spec/1.2.2/#12-yaml-history https://yaml.org/spec/1.2.2/#12-yaml-history
    - TheDong 1y ago
      If your data format is so complicated that all commonly used implementations are not compliant with your spec, maybe it's a problem with the data-format. Every single implementation people actually use seems to be a messy mix of yaml 1.1 and 1.2.... Maybe if the yaml project wants to consider this fixed, they should have written some correct reference parsers themselves for any languages in need, and encouraged their use.
      - tkfu 1y ago
        I noted this in reply to the comment above, but: the YAML 1.2 spec doesn't actually mandate that parsers use the Core Schema. They left it as a recommendation. So I don't consider it to be "fixed" at all.
    - tkfu 1y ago
      I would not say it "fixed" the problem. It removed the _recommendation_ for parser implementations to use the regex `y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF` for parsing scalars as bools, it changed the _canonical_ presentation of bools from `y|n` to `true|false`, and it introduced the "schema" concept. It also introduced and recommended the use of the Core Schema, which uses `true|True|TRUE|false|False|FALSE` as the regex for inferring bools from scalars. But unsurprisingly, since using this schema is only a recommendation and not a requirement, many implementations elected to keep their backwards-compatible implementations that use the original regex. So the Norway problem persists.
- esafak 1y ago
  If we're going there let's just stay there and not translate it back to YAML, which is absolutely inappropriate for specifying CI pipelines.
- anttiharju 1y ago
  Or just use composite actions, it's not 2020 anymore. Templating GitHub Actions is very powerful (I've worked with such a setup) but it has its own headaches and if you don't _need_ custom tooling better to not have it. I can wish for improvements on the native setup without reaching out for the sledgehammer.
  - anttiharju 1y ago
    I think most of the pain with GitHub Actions goes away if you use actionlint, action-validator, prettier/yamlfmt in a single ci job to validate your setup. Can even add them as git hooks that automatically stage changes and give quick feedback when iterating.

TheDong 1y ago

Wanna DRY out your github actions yaml? Generate it from Dhall, or cue, or python, or some real language that supports actual abstractions. If your problem is you want to DRY out yaml, and you use more yaml features to do it, you now have more problems, not fewer.

usrme 1y ago
Seconded. I've had huge success with generating workflows with CUE. Would definitely recommend it to anyone struggling with YAML.
Spivak 1y ago
Just introduce a templating language and the CI to generate the yaml as part of your pipeline as way to make it simpler? Above a certain level of complexity, sure. But having nothing in between is an annoying state of affairs. I use anchors in Gitlab pipelines and I hardly curse their names.
nirvdrum 1y ago
Having used CI systems and application frameworks that support YAML anchors for configuration, adding in a programming language would be a massive amount of complexity for very little gain. We're not talking about dozens of locations with hundreds of lines of shared code. Asking the team to add a new build dependency, learn a new language, and add a new build step would create considerably more problems, not fewer. Used sparingly and as needed, YAML anchors are quite easy to read. A good editor will even allow you to jump to the source definition just as it would any other variable. Being self-contained without any additional dependencies is a huge advantage, particularly for open source projects, IMHO. I'd wager very few people are going to learn Dhall in order to fix an issue with an open source project's CI.
- baq 1y ago
  Your team doesn't know YAML, it knows github actions. There's zero transferable knowledge when switching from github actions to kubernetes deployments, as there is precisely the same zero correlation between kubernetes and ansible configs. 'It's all YAML' is a lie and I'm continuously surprised so many people are falling for it for so long. YAML is the code-as-data, but the interpreter determines what it all means.
  - nirvdrum 1y ago
    Oh, for goodness' sake. We know YAML syntax and that's the only part that's relevant here. Pointing out that different software uses different keys for their configuration or even takes different actions for keys that happen to share the same name isn't particularly insightful. We haven't been bamboozled.
    - baq 1y ago
      I don’t understand even more now. If you freely admit they’re different languages, the only reason to keep using the stupid deficient syntax is momentum, and while it isn’t a bad reason, it is costing you and everyone else in the long run.
      - 1y ago
phanimahesh 1y ago
This. So true. Yaml has always been an overly complicated format, with weird quirks ( like norway becoming false in a list of country codes ). I find it an absolute shame that languages like Dhall did not become more popular earlier. Now everything in devops is yaml, and I think many developers pick yaml configs not out of good reasons but defaulting to its ubiquity as sufficient.
- true_blue 1y ago
  >norway yaml 1.2 was released in 2009, and it fixed this problem. this is an implementation issue. https://yaml.org/spec/1.2.2/#12-yaml-history https://yaml.org/spec/1.2.2/#12-yaml-history
  - TheDong 1y ago
    If your data format is so complicated that all commonly used implementations are not compliant with your spec, maybe it's a problem with the data-format. Every single implementation people actually use seems to be a messy mix of yaml 1.1 and 1.2.... Maybe if the yaml project wants to consider this fixed, they should have written some correct reference parsers themselves for any languages in need, and encouraged their use.
    - tkfu 1y ago
      I noted this in reply to the comment above, but: the YAML 1.2 spec doesn't actually mandate that parsers use the Core Schema. They left it as a recommendation. So I don't consider it to be "fixed" at all.
  - tkfu 1y ago
    I would not say it "fixed" the problem. It removed the _recommendation_ for parser implementations to use the regex `y|Y|yes|Yes|YES|n|N|no|No|NO|true|True|TRUE|false|False|FALSE|on|On|ON|off|Off|OFF` for parsing scalars as bools, it changed the _canonical_ presentation of bools from `y|n` to `true|false`, and it introduced the "schema" concept. It also introduced and recommended the use of the Core Schema, which uses `true|True|TRUE|false|False|FALSE` as the regex for inferring bools from scalars. But unsurprisingly, since using this schema is only a recommendation and not a requirement, many implementations elected to keep their backwards-compatible implementations that use the original regex. So the Norway problem persists.
esafak 1y ago
If we're going there let's just stay there and not translate it back to YAML, which is absolutely inappropriate for specifying CI pipelines.
anttiharju 1y ago
Or just use composite actions, it's not 2020 anymore. Templating GitHub Actions is very powerful (I've worked with such a setup) but it has its own headaches and if you don't _need_ custom tooling better to not have it. I can wish for improvements on the native setup without reaching out for the sledgehammer.
- anttiharju 1y ago
  I think most of the pain with GitHub Actions goes away if you use actionlint, action-validator, prettier/yamlfmt in a single ci job to validate your setup. Can even add them as git hooks that automatically stage changes and give quick feedback when iterating.