3 ms·

We basically need Restricted YAML. No 1000 ways to say true/false. No 1000 ways to have strings. No significant whitespace in config files (also helps out wit

by oblio 3y ago

We basically need Restricted YAML. No 1000 ways to say true/false. No 1000 ways to have strings. No significant whitespace in config files (also helps out with the 1000 types of YAML strings). Etc. A subset of YAML that drops all that garbage. And yeah, the CI/CD stuff should also use... a sandboxed programming language. I think Lua and Tcl can remove functions and language constructs in a sandboxed enviroment. Pick one of those, or improve it slightly (make Lua 0-index based if you so care), and then use that.

zufallsheld 3y ago
There already is strictyaml: https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml
- oblio 3y ago
  Cool, get back to me in 2040 when it's adopted by Ansible, Github Actions, Bitbucket, Kubernets & its menagerie :-p
  - hahn-kev 3y ago
    There's no reason you and your team couldn't choose to use it today no?
    - verdverm 3y ago
      Are there packages for language X? How would you enforce it?
      - mminer237 3y ago
        Looks like just Python at the moment. But you could enforce it with a pre-commit hook.
        verdverm 3y ago
        But then I also have to make sure python and the dependencies are available in all the places. Not an ideal situation for git hooks. What do you do when you bump the version in one branch and then switch branches?
cageface 3y ago
Significant whitespace is a mistake people keep making over and over again for some reason.
- hahn-kev 3y ago
  Probably because python is so popular
- verdverm 3y ago
  Use tabs to reduce whitespace chars :] I think a more precise statement is "scoping with invisible chars" Python has this issue too
- RealStickman_ 3y ago
  People should start using a good editor.
alexvitkov 3y ago
something like JSON maybe idk
- drdaeman 3y ago
  You missed the "/s" bit, I presume? But you certainly have a point. If JSON still fits somewhere - that's nice, it means the issue is just about YAML syntax (which is a matter of personal preference, really; but I thought we left all the entertainment of putting Pythonistas and Perl Hackers in a virtual ring in the past decade, somewhere next to /.'s grave). If JSON feels worse - it means YAML's syntax is not a problem, it's a symptom of YAML being misplaced to express something other than a simple data structure.
  - alexvitkov 3y ago
    No /s, either JSON is OK for you or it's not, and if not a slightly nicer looking but way more error prone JSON doesn't solve your problem. 1. If you're making a user-facing program you give them a nice settings menu and store the config as whatever. 2. if it's developer-targeted software, and your config starts approaching a scripting language you don't encode arbitrary computations in YAML (or XML, Microsoft), but rather let the user use a scripting language and preferably you don't invent a new one. We always cry about fragmentation and complexity in software and yet even in the rare cases when we have universally agreed upon formats/standards we will still go out of our way to make a new one for the amazing benefit of significant whitespace.
  - thayne 3y ago
    > If JSON feels worse - it means YAML's syntax is not a problem Only if you can say the same of json5. There are plenty of cases where you have a simple data structure, but having comments and multi-line strings is important.
drdaeman 3y ago
How's this supposed to make any difference about YAML (or this Restricted YAML) being used for things it's not supposed to be used for? It's not a problem that $CI YAML configs have multiple ways to encode strings and need indentation. Yes, "no" issue sucks a bit, but it's not that huge, just a minor papercut. The giant root issue is that YAML is simply not an appropriate format for this kind of stuff. People are writing more and more complex programs (modern CI pipeline is a program*) in it, when it's not a [suitable] programming language. And it's not suitable not because of whitespace. Syntax doesn't matter that much (unless you're a junior developer first time seeing APL - just kidding, of course), it's all about the semantics. YAML was never meant for writing programs, it doesn't have necessary constructs to support those (but people are inventing DSLs on top of it, and it looks and feels awful). Similar logic applies to some orchestration systems, some IaaC stuff and other areas where YAML is not a great fit because systems either "outgrew" simple configurations to something else (like programs, but not always) or YAML was a bad fit from the very beginning. CI is just the most obvious example. *) It made sense when CIs were simple, and those YAML files were configuration snippets rather than programs. Just like Makefile makes a lot of sense, when it's able to do all you need. But CI grew more and more complicated.

by oblio 3y ago

zufallsheld 3y ago

There already is strictyaml: https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml

oblio 3y ago
Cool, get back to me in 2040 when it's adopted by Ansible, Github Actions, Bitbucket, Kubernets & its menagerie :-p
- hahn-kev 3y ago
  There's no reason you and your team couldn't choose to use it today no?
  - verdverm 3y ago
    Are there packages for language X? How would you enforce it?
    - mminer237 3y ago
      Looks like just Python at the moment. But you could enforce it with a pre-commit hook.
      - verdverm 3y ago
        But then I also have to make sure python and the dependencies are available in all the places. Not an ideal situation for git hooks. What do you do when you bump the version in one branch and then switch branches?

cageface 3y ago

Significant whitespace is a mistake people keep making over and over again for some reason.

hahn-kev 3y ago
Probably because python is so popular
verdverm 3y ago
Use tabs to reduce whitespace chars :] I think a more precise statement is "scoping with invisible chars" Python has this issue too
RealStickman_ 3y ago
People should start using a good editor.

alexvitkov 3y ago

something like JSON maybe idk

drdaeman 3y ago
You missed the "/s" bit, I presume? But you certainly have a point. If JSON still fits somewhere - that's nice, it means the issue is just about YAML syntax (which is a matter of personal preference, really; but I thought we left all the entertainment of putting Pythonistas and Perl Hackers in a virtual ring in the past decade, somewhere next to /.'s grave). If JSON feels worse - it means YAML's syntax is not a problem, it's a symptom of YAML being misplaced to express something other than a simple data structure.
- alexvitkov 3y ago
  No /s, either JSON is OK for you or it's not, and if not a slightly nicer looking but way more error prone JSON doesn't solve your problem. 1. If you're making a user-facing program you give them a nice settings menu and store the config as whatever. 2. if it's developer-targeted software, and your config starts approaching a scripting language you don't encode arbitrary computations in YAML (or XML, Microsoft), but rather let the user use a scripting language and preferably you don't invent a new one. We always cry about fragmentation and complexity in software and yet even in the rare cases when we have universally agreed upon formats/standards we will still go out of our way to make a new one for the amazing benefit of significant whitespace.
- thayne 3y ago
  > If JSON feels worse - it means YAML's syntax is not a problem Only if you can say the same of json5. There are plenty of cases where you have a simple data structure, but having comments and multi-line strings is important.

drdaeman 3y ago

How's this supposed to make any difference about YAML (or this Restricted YAML) being used for things it's not supposed to be used for? It's not a problem that $CI YAML configs have multiple ways to encode strings and need indentation. Yes, "no" issue sucks a bit, but it's not that huge, just a minor papercut. The giant root issue is that YAML is simply not an appropriate format for this kind of stuff. People are writing more and more complex programs (modern CI pipeline is a program*) in it, when it's not a [suitable] programming language. And it's not suitable not because of whitespace. Syntax doesn't matter that much (unless you're a junior developer first time seeing APL - just kidding, of course), it's all about the semantics. YAML was never meant for writing programs, it doesn't have necessary constructs to support those (but people are inventing DSLs on top of it, and it looks and feels awful). Similar logic applies to some orchestration systems, some IaaC stuff and other areas where YAML is not a great fit because systems either "outgrew" simple configurations to something else (like programs, but not always) or YAML was a bad fit from the very beginning. CI is just the most obvious example. *) It made sense when CIs were simple, and those YAML files were configuration snippets rather than programs. Just like Makefile makes a lot of sense, when it's able to do all you need. But CI grew more and more complicated.