5 ms·

Can you explain YAML? I've found declarative pipelines with it have been... fine?

fergie 2y ago
As a developer based in Norway, one fairly major drawback to YAML is the way that it processes the language code for Norwegian ("no").
- marcusramberg 2y ago
  That particular pain was fixed in yaml 1.2 :)
  - withinboredom 2y ago
    TIL I learned that yaml has versions and now I'm wondering what version of yaml parsers are running where. I think I might be on a new level of hell. It would be better to delete your comment so nobody else has to has to ever have this crisis.
- LeonM 2y ago
  The Norwegian ISO3166 code colliding with the English word 'no' is not a YAML problem per se, I've been bitten by that a few times in other situations as well. For example: Stripe uses constants for types of tax registration numbers (VAT/GST/TIN, etc.). So there is EU_VAT for European VAT numbers, US_TIN for US tax identification numbers, etc. But what value to use for tax-exempt organisations that don't have a tax number? Well... guess how I found out about NO_VAT... On the bright side, I did learn that way that although Norway is in the Schengen zone, apparently they are not part of the EU (hence the separation of EU_VAT and NO_VAT). I guess the 'no' name collision has taught many developers something about Norway :-)
dharmab 2y ago
1. The YAML spec is extremely complex with some things being ambiguous. You might not notice this if your restrict yourself to a small subset of the language. But you will notice it when different YAML libraries and programming languages interpret the same YAML file as different content. 2. Trying to encode logic and control flow in a YAML document is much more difficult than writing that flow in a "real" programming language. Debugging is especially much easier in "real" languages.
imp0cat 2y ago
Depends on the complexity of your pipeline.
deng 2y ago
YAML is fine for what it is: a markup language. I have no problem with it being used in simple configuration files, for instance. However, CI is not "configured", it is coded. It is simply the wrong tool. YAML was continuously extended to deal with that, so it developed into much more than just "markup", but it grew into this terrible chimera. Once you start using advanced features in GitLab's YAML like anchors and references to avoid writing the same stuff again and again, you'll notice that the whole tooling around YAML is simply not there. How does the resulting YAML look like? How do you run this stuff locally? How do you debug this? Just don't go there. You will not be able to avoid YAML completely, obviously, but use it the way it was originally intended to.
- LeonM 2y ago
  > CI is not "configured", it is coded. Finally! I was always struggling to explain to others why YAML is OK-ish as a language, but then never seems to work well for the things people tried doing with it. Especially stuff that needs to run commands, such as CI. > How does the resulting YAML look like? How do you run this stuff locally? How do you debug this? Just don't go there. Agreed. GitHub actions, or any remote CI runner for that matter, makes the problem even worse. The whole cycle of having to push CI code, wait 10 minutes while praying for it to work, still getting an error, trying to figure out the mistake, fixing one subtle syntax error, then pushing the code again in the hope that that works is just a terrible workflow. Massive waste of time. > You will not be able to avoid YAML completely, obviously, but use it the way it was originally intended to. Even for configurations YAML remains a pain, unfortunately. It could have been great for configs, but in my experience the whole strict whitespace (tabs-vs-spaces) part ruined it. It isn't a problem when you work from an IDE that protects you from accidentally using tabs (also, auto-formatting for the win!) but when you have to write YAML configuration (for example: Netplan) on a remote server using just an editor it quickly becomes a game of whack-a-mole.
  - motorest 2y ago
    > Especially stuff that needs to run commands, such as CI. I don't understand what problem you could possibly be experiencing. What exactly do you find hard about running commands in, say, GitLab CICD?
maratc 2y ago
You can't put a breakpoint in YAML. You can't evaluate variables in YAML. You can't print debugging info from YAML. You can't rerun YAML from some point. YAML is great for the happy-flow where everything works. It's absolutely terrible for any other flow.
- motorest 2y ago
  > You can't put a breakpoint in YAML. You can't evaluate variables in YAML. You can't print debugging info from YAML. You can't rerun YAML from some point It's a DSL. There is no execution, only configuration. The only thing that's executed are the custom scripts you create yourself, and any intro tutorial on the subject will eventually teach you that if you want to run anything beyond a single straight-forward command then you should move those instructions to a shell script to make them testable and reproducible. Things are so simple and straight forward that you need to go way out of your way to create your own problems. I wonder how many people in this discussion are blaming the tools when they even bothered to learn the very basics.
  - dharmab 2y ago
    > There is no execution, only configuration. The YAML is fed into an agent which reads it to decide what to execute. Any time you change the control flow of a system by changing data, you are doing a form of programming.
  - maratc 2y ago
    Your attitude of "how is everyone so stupid" does not help the discussion. > It's a DSL. There is no execution, only configuration. Jenkins pipelines are also DSL. I still can print out debugging information from them. "It's a DSL" is not an excuse for being a special case of shitty DSL. > any intro tutorial on the subject will eventually teach you Do these tutorials have a chapter on what to do when you join a company with 500 engineers and a ton of YAMLs that are not written in that way? > you should move those instructions to a shell script to make them testable Yeah, no. How am I supposed to test my script that is supposed to run on Github-supplied runner with a ton of injected secrets and Github-supplied JSON of 10,000 lines, when I don’t have the runner, the secrets, or the JSON?
crabbone 2y ago
YAML's problems: * Very easy to write the code you didn't mean to, especially in the context of CI where potentially a lot of languages are going to be mixed, a lot of quoting and escaping. YAML's string literals are a nightmare. * YAML has no way to express inheritance. Nor does it have a good way to express variables. Both are usually desperately needed in CI scripts, and are usually bolted on top with some extra-language syntax (all those dollars in GitHub actions, Helm charts, Ansible playbooks etc.) * Complexity skyrockets compared to the size of the file. I.e. in a language like C you can write a manageable program with millions of lines of code. In YAML you will give up after a few tens of thousands of lines (similar to SQL or any other language that doesn't have modules). * Whitespace errors are very hard to spot and fix. Often whitespace errors in YAML result in valid YAML which, however, doesn't do what you want...

As a developer based in Norway, one fairly major drawback to YAML is the way that it processes the language code for Norwegian ("no").

marcusramberg 2y ago
That particular pain was fixed in yaml 1.2 :)
- withinboredom 2y ago
  TIL I learned that yaml has versions and now I'm wondering what version of yaml parsers are running where. I think I might be on a new level of hell. It would be better to delete your comment so nobody else has to has to ever have this crisis.
LeonM 2y ago
The Norwegian ISO3166 code colliding with the English word 'no' is not a YAML problem per se, I've been bitten by that a few times in other situations as well. For example: Stripe uses constants for types of tax registration numbers (VAT/GST/TIN, etc.). So there is EU_VAT for European VAT numbers, US_TIN for US tax identification numbers, etc. But what value to use for tax-exempt organisations that don't have a tax number? Well... guess how I found out about NO_VAT... On the bright side, I did learn that way that although Norway is in the Schengen zone, apparently they are not part of the EU (hence the separation of EU_VAT and NO_VAT). I guess the 'no' name collision has taught many developers something about Norway :-)

dharmab 2y ago

1. The YAML spec is extremely complex with some things being ambiguous. You might not notice this if your restrict yourself to a small subset of the language. But you will notice it when different YAML libraries and programming languages interpret the same YAML file as different content. 2. Trying to encode logic and control flow in a YAML document is much more difficult than writing that flow in a "real" programming language. Debugging is especially much easier in "real" languages.

imp0cat 2y ago

Depends on the complexity of your pipeline.

deng 2y ago

YAML is fine for what it is: a markup language. I have no problem with it being used in simple configuration files, for instance. However, CI is not "configured", it is coded. It is simply the wrong tool. YAML was continuously extended to deal with that, so it developed into much more than just "markup", but it grew into this terrible chimera. Once you start using advanced features in GitLab's YAML like anchors and references to avoid writing the same stuff again and again, you'll notice that the whole tooling around YAML is simply not there. How does the resulting YAML look like? How do you run this stuff locally? How do you debug this? Just don't go there. You will not be able to avoid YAML completely, obviously, but use it the way it was originally intended to.

LeonM 2y ago
> CI is not "configured", it is coded. Finally! I was always struggling to explain to others why YAML is OK-ish as a language, but then never seems to work well for the things people tried doing with it. Especially stuff that needs to run commands, such as CI. > How does the resulting YAML look like? How do you run this stuff locally? How do you debug this? Just don't go there. Agreed. GitHub actions, or any remote CI runner for that matter, makes the problem even worse. The whole cycle of having to push CI code, wait 10 minutes while praying for it to work, still getting an error, trying to figure out the mistake, fixing one subtle syntax error, then pushing the code again in the hope that that works is just a terrible workflow. Massive waste of time. > You will not be able to avoid YAML completely, obviously, but use it the way it was originally intended to. Even for configurations YAML remains a pain, unfortunately. It could have been great for configs, but in my experience the whole strict whitespace (tabs-vs-spaces) part ruined it. It isn't a problem when you work from an IDE that protects you from accidentally using tabs (also, auto-formatting for the win!) but when you have to write YAML configuration (for example: Netplan) on a remote server using just an editor it quickly becomes a game of whack-a-mole.
- motorest 2y ago
  > Especially stuff that needs to run commands, such as CI. I don't understand what problem you could possibly be experiencing. What exactly do you find hard about running commands in, say, GitLab CICD?

maratc 2y ago

You can't put a breakpoint in YAML. You can't evaluate variables in YAML. You can't print debugging info from YAML. You can't rerun YAML from some point. YAML is great for the happy-flow where everything works. It's absolutely terrible for any other flow.

motorest 2y ago
> You can't put a breakpoint in YAML. You can't evaluate variables in YAML. You can't print debugging info from YAML. You can't rerun YAML from some point It's a DSL. There is no execution, only configuration. The only thing that's executed are the custom scripts you create yourself, and any intro tutorial on the subject will eventually teach you that if you want to run anything beyond a single straight-forward command then you should move those instructions to a shell script to make them testable and reproducible. Things are so simple and straight forward that you need to go way out of your way to create your own problems. I wonder how many people in this discussion are blaming the tools when they even bothered to learn the very basics.
- dharmab 2y ago
  > There is no execution, only configuration. The YAML is fed into an agent which reads it to decide what to execute. Any time you change the control flow of a system by changing data, you are doing a form of programming.
- maratc 2y ago
  Your attitude of "how is everyone so stupid" does not help the discussion. > It's a DSL. There is no execution, only configuration. Jenkins pipelines are also DSL. I still can print out debugging information from them. "It's a DSL" is not an excuse for being a special case of shitty DSL. > any intro tutorial on the subject will eventually teach you Do these tutorials have a chapter on what to do when you join a company with 500 engineers and a ton of YAMLs that are not written in that way? > you should move those instructions to a shell script to make them testable Yeah, no. How am I supposed to test my script that is supposed to run on Github-supplied runner with a ton of injected secrets and Github-supplied JSON of 10,000 lines, when I don’t have the runner, the secrets, or the JSON?

crabbone 2y ago

YAML's problems: * Very easy to write the code you didn't mean to, especially in the context of CI where potentially a lot of languages are going to be mixed, a lot of quoting and escaping. YAML's string literals are a nightmare. * YAML has no way to express inheritance. Nor does it have a good way to express variables. Both are usually desperately needed in CI scripts, and are usually bolted on top with some extra-language syntax (all those dollars in GitHub actions, Helm charts, Ansible playbooks etc.) * Complexity skyrockets compared to the size of the file. I.e. in a language like C you can write a manageable program with millions of lines of code. In YAML you will give up after a few tens of thousands of lines (similar to SQL or any other language that doesn't have modules). * Whitespace errors are very hard to spot and fix. Often whitespace errors in YAML result in valid YAML which, however, doesn't do what you want...