6 ms·

Abstraction, not syntax

102 points by unripe_syntax 1y ago

JohnMakin 1y ago
As someone who's spent most of their career in cloud IAC, and likes to think they are pretty read up on the latest going on in that world, if you didn't know better you'd think YAML is one of the greatest threats facing mankind. There are plenty of things I certainly hate about it, but every configuration syntax I've ever used I have similar gripes about. It's like once a month this kind of "The world is growing tired of yaml" claim is just thrown out there like everyone just agrees with it. Choose something that works for you. This author repeatedly mentions TOML but there are plenty of issues with that one I could point out too. Syntax is one very small part of what makes an ecosystem great or not so great. Most of my exposure to yaml is helm chart templates, which admittedly is not pure YAML, but it works fine enough for me, at least to where I don't feel like writing lengthy blog posts about how much I hate it. I even wrote a library that converts yaml templates to HCL for internal use because I got so sick of people having this exact same argument like it deeply mattered. And guess what? They hate the HCL too.
- compyman 1y ago
  I also think that a lot of the problems with yaml specifically are overblown, but this post is actually not about that! It is specifically saying the same problem exists in JSON/YAML/TOML, etc, which is that all these configuration languages don't have any real means of abstraction, and ultimately aren't expressive enough do to the job we require of them. as soon as you are templating config files with other configs, I agree, I have sorely felt this limitation with helm charts
  - kmoser 1y ago
    Serious question: do people who work with these config files frequently, or on large such files, use simple text editors, or are there "smart" editors that do things like prevent you from making typos or inserting the wrong data type, similar to an HTML form that does basic validation or a DB schema that rejects bad data? There is no single cure-all, of course, but surely we should be relying on computers to do much of the heavy lifting when it comes to validation and verification of these files, not just as linters after the fact but in realtime while we're editing, and with some sort of knowledge (even if derived programmatically) of what is right and what is wrong so we no longer have to worry about simple footguns.
    - kokada 1y ago
      I think one of the problems of those "configuration languages" is that you can extract semantic information without knowing the target, e.g., with has a specific meaning in GitHub Actions but it is otherwise an unremarkable word in the YAML specification. But when working with real programming languages it is completely different, you can take semantic information from the current code, and you can have things like types to give you safety.
      - JohnMakin 1y ago
        The problem is most configuration languages are declarative vs imperative like most “real” languages are. You could probably levy the same complaint against declarative languages in general - it’s just a different way of thinking
        kokada 1y ago
        Nix as used in NixOS is a declarative language and there is none of the issues I cited by being a "real" programming language (or as the article talks about, having "abstractions" like builtin.map). You can pretty easily setup a LSP to get code-completion (even between different projects, like NixOS vs Home-Manager). There is no proper type system in Nix but the module system does supplement it well.
- taeric 1y ago
  I'm trying to remember the phrase. Something like, "there is nothing as vicious as low stakes fights." Trying that on Google gets me https://en.wikipedia.org/wiki/Sayre%27s_law https://en.wikipedia.org/wiki/Sayre%27s_law. Is about right. :D
  - jdmichal 1y ago
    It's like bike shedding. It's a side effect of mixed expertise (and confidence) working together on things that are only partially understood by all. When something is clearly outside one's expertise, they are content to leave it to others. But then you'll get minor questions with low stakes like "what color to paint the shed". And how everyone feels like they can participate, so suddenly there's a huge discussion / debate / argument about a very, very minor thing.
- skywhopper 1y ago
  HCL is generally great, but has some issues with clarity of transformation to the underlying data structures. But anytime someone suggests TOML I have to double check to be sure they are serious because the TOML syntax for anything more complicated than single-layer maps is mind-bogglingly confusing to me. This is not a serious alternative to YAML.
- qezz 1y ago
  > And guess what? They hate the HCL too. Don't want to sound too harsh, but to me HCL is even worse than plain YAML. By expressiveness, HCL is somewhat similar to Ansible-flavoured YAML - in both you need to use magic keywords to create any kind of abstraction (e.g. a loop). HCL is worse than regular YAML because there's only one "true" parser for it, that is official Hashicorp's HCL parser. So if you are locked into Golang ecosystem, then sure it can work for you, otherwise you are out of luck. There are a couple of tools that convert HCL into JSON, I tried both, they somewhat work, but in the end of the day it's a big hack. At that point I just gave up on using HCL and started using something else that generates JSON. Hope you find some configuration layer that fits your users more than HCL or YAML.
  - JohnMakin 1y ago
    Not exactly true in terms of being the only parser - OpenTofu is a good project I’m highly supportive of
deleted 1y ago
[deleted]
Defletter 1y ago
Really wish people would just bite the bullet and do configuration as code instead of trying to make all these config petlangs.
- patrickmay 1y ago
  Exactly. Emacs Lisp is an existence proof that this can be done well.
  - taeric 1y ago
    You beat me to it! And for those that haven't taken a look at it, the "customize" menu and everything it supports is silly impressive. And it just writes out the results out, like a boss.* * Obviously, it has a lot of "don't edit below this line" complexity to it. But that doesn't change that it is right there.
- skydhash 1y ago
  My preference is towards simpler formats like: option value Easy to edit and manipulate. JSON and YAML is always a nightmare if it's user facing. As for ansible, I'd love to see some scheme/lisp variants.
  - mrheosuper 1y ago
    then how you distinguish between string "52" and number 52 ? Keep adding more edge cases and you have something resembles JSON
    - skydhash 1y ago
      Why do you need to differentiate between the two as an input? It’s config, not random data. If you have email test@example.com logo-size 100 background #adadaf modules auth modules db modules files The only reason to have special token here is if you multiple line values. Types are not a concern.
    - bandie91 1y ago
apalmer 1y ago
I don't think the title and the article really communicates it's case well. Did not understand the goal until 90% through the article when they showed the source code of RCL with the loops. This isn't syntax vs abstraction. This is how much programming language power do you want to enable in your configuration language. This is a big difference and I think we miss the interesting part of that discussion because we dip into this 'abstraction angle.
- dapperdrake 1y ago
  The "abstraction angle" seems to be a different encoding for "power of configuration language."
- jasaldivara 1y ago
  I think if people want a more powerful, programmable config language, maybe they should use something like Lua or Scheme instead of reinventing the wheel with those new niche languages.
dvrp 1y ago
Should be titled “the power spectrum of data” or something similar the article talks about the trade off between plain data structure versus abstract ones and that’s the main issue
nickelpro 1y ago
Yes, one more DSL on your Tower of Babel tech stack will save you. If you want configuration-as-code use Python. Please. Or Tcl if you must. Do not invent N+1 DSL for your engineers to waste time learning.
- qezz 1y ago
  Luckily, the main point of the article is that syntax doesn't matter, but abstractions do. In other words, you can use your favorite DSL, that being Python or TCL or something else.
kgen 1y ago
Code as configuration loses one primary benefit which is being able to read the actual config and know exactly what will apply. In the example in the article you would get the same benefit by disallowing users to edit the config directly and instead require the config to be generated via a cmdline app/service that encodes the same policy?
Splizard 1y ago
The problem with configuration formats, is not syntax, or abstraction, it's the lack of consistent language server integration, it's problem when I can't lookup the definition for a key, the expected type, or quickly jump to definitions that clearly show which keys are available to configure.
- atoav 1y ago
  To be frank, the clear problem with configuration format is that people have configurations so complex they probably should use something else. Example: We are programming a backend for a blog. If we were to not use templates, but instead try to get that functionality via the webservice configuration we would have to "invent" some format that gives us the flexibility of templates within let's say a YAML file. Needless to say that would be a horrible idea. Maybe I am being naive here, but I have yet to be convinced of the fact that it is really configuration formats that are the problem and not what people try to abuse them for. I have yet to work on a project where TOML wasn't enough for actual configuration. Usually when I need something more complex than what can be done with TOML it is a sign that this needs to be handled differently. Via templates or with a database or making a special DSL-like script file. E.g. if you're using python nothing (except security considerations) stops you from allowing users to use a python file for configuration. If your configuration needs are really that complex, why not use a real programming language for it?
  - skydhash 1y ago
    Two examples of complex user facing configurations I can think of are pretty trivial to implement: - Decision tree, where you only need comparison operators. The leaves are a specified list of actions. - actions list with macros (variables). You can be fancy and add some conventions for arrays. Anything more that that should just be a programming language. And if the relationship is adversarial (saas), you should really think hard about needing something that complex.
weavejester 1y ago
To me the solution seems like it's adding complexity that could cause more issues further down the line. The specific problems in the example could be solved by changing how the data is represented. Consider the following alternative representation, written in edn: {:aws.s3/buckets {:aws.region/eu-west {:alpha-hourly {:lifecycle/policy {:delete-after #interval/days 4}} :alpha-daily {:lifecycle/policy {:delete-after #interval/days 30}} :alpha-monthly {:lifecycle/policy {:delete-after #interval/days 365}} :bravo-hourly {:lifecycle/policy {:delete-after #interval/days 4}} :bravo-daily {:lifecycle/policy {:delete-after #interval/days 30}} :bravo-monthly {:lifecycle/policy {:delete-after #interval/days 365}}}}} This prevents issues where the region is mistyped for a single bucket, makes the interval more readable by using a custom tag, and as a bonus prevents duplicate bucket names via the use of a map. Obviously this doesn't prevent all errors, but it does prevent the specific errors that the RCL example solves, all without introducing a Turing-complete language.
- MathMonkeyMan 1y ago
  > The specific problems in the example could be solved by changing how the data is represented. Finding the "right" representation for a given set of data is an interesting problem, but most (all) of the time the representation is specified by someone/something else. In the past I've written a [preprocessor][1] that adds some power to the representation while avoiding general purpose computation. For example, (buckets (let ([(regional region (name policy) ...) ((['name name] ['region region] ['lifecycle_policy policy]) ...)]) (regional us-west (alpha-hourly (delete_after_seconds 345600)) (alpha-daily (delete_after_seconds 2592000)) (alpha-monthly (delete_after_seconds 31536000)) (bravo-hourly (delete_after_seconds 345600)) (bravo-daily (delete_after_seconds 259200)) (bravo-monthly (delete_after_seconds 31536000))))) Macros, basically. Arithmetic would help there, but that might be too much. [1]: https://github.com/dgoffredo/llama https://github.com/dgoffredo/llama
yawnxyz 1y ago
the answer seems to be both of both worlds - if you're going to do for loops why not just use python? the answer is both a faux programming language, and really bad ux / really hard to read / scan maybe what they need is a program that generates better readable text; and somehow you can flip between the determinism of code and ux of readable text?! (is that possible)
transfire 1y ago
FOR loops? YAML has a merge key <<:, which might be helpful. The merge key is a clever little trick, but it depends of the special hash key, so lists can’t be merged. Syntax does matter, which is why YAML matters — even if imperfect.
- rattyJ2 1y ago
  Merges and anchors are some of the least maintaineable and most error-prone config i've seen. Doesn't help every yaml parser has their own opinion on what a merge or an anchor should do, exactly.
rednafi 1y ago
The intention behind configuration languages is sound but there’s just too many of them. Not having a universally accepted one makes picking one harder. Also, migrating from one to another isn’t as straightforward. Plus, for database in ["alpha", "bravo"]: for period, days in period_retention_days: At this point, you’re better off writing a Python script that spits out some JSON. I‘m aware that even the blog mentions it. The benefit is - having a more expressive language at your fingertips and not having to fight your peers while trying to add yet another idiosyncratic dependency.
- sam_bristow 1y ago
  Stoke Space[1] uses a similar system, letting people write arbitrary code to generate a static configuration for their launch vehicle. It means you get all the power of something like Python during development but also a deterministic, bounded config for the critical flight systems. I think their config files are just TOML that is consumed by Rust. I'll try dig out a link to the talk one of their Flight Software Engineers did on the concept. [1] https://www.stokespace.com/ https://www.stokespace.com/
lwhsiao 1y ago
I'd be curious what the author thinks of KSON, which was also recently featured on HN [1]. [1]: https://news.ycombinator.com/item?id=45291858 https://news.ycombinator.com/item?id=45291858
- ruuda 1y ago
  I mention it in the first paragraph, and what I think of it in the second paragraph.
gorgoiler 1y ago
Somewhere along the way we got lost into thinking config files were remote procedure calls, and that YAML was the only RPC interface available to us. The caller generates YAML, the receiver parses it, but more often than not the two are on the same host and use the same language*: def say(message): o = dict(f=say, v=message)) call(yaml.dumps(o)) …meanwhile elsewhere… def eval(request): o = yaml.loads(request.body) match o[“f”]: case “say”: print(o[“v”]) Yes, referring to this as “madness” skips over the fact that you can now scale up your hello world printer across the internet, have auth, rate limit, etc etc. but for so many things the RPC just isn’t needed at all. As this article gets at, config files are one of those things, and they benefit hugely from the rampant abstraction violation of skipping an intermediary text format between the program doing the configuration and the program doing the work. *or have internal versions of their APIs available in matching languages.
radarsat1 1y ago
I got confused reading this because I wasn't sure how I, as a reader with no knowledge of the system under discussion, was supposed to know that all the buckets should be in the same region. Nor is it clear to me how the "for loop" version would handle the case where exceptionally one bucket is different. Which is a more interesting discussion imho, that's the whole point of having it as a configuration field, after all.
- Joker_vD 1y ago
  Like this: region = if name == "bravo-hourly" then "us-west" else "eu-west",
  - radarsat1 1y ago
    I see, not so clean imho but I guess it works.
- qezz 1y ago
  The example in the article shows a need for cartesian product of (bucket name) and (lifetime policy), with a fixed location. Nothing stops you from defining a separate category for the exceptions, and just append them to the resulting list. Or you may want to assign each bucket its own location, then you define it along side the bucket name.
teo_zero 1y ago
Before going all in with turing-complete configuration languages, why not simply augmenting existing declarative formats with parameter & arithmetic expansion à la bash? It's a syntax already familiar to many, that would mitigate the issues highlighted in TFA.
texuf 1y ago
Yeah no. The reason we don't write code in config files is that, if we did, in order to know what you are deploying you have to run the code in your head. Not just this code, the code that Steve wrote six months ago before people noticed he was grossly incompetent and fired. Also worth mentioning, I can run this code in my head. The author can run this code in their head, but not everyone can. And finally, if the author thinks copy pasting yaml files cause bugs, wait till the llms start copy pasting the for loops.
- saurik 1y ago
  Or you could run the code on the computer, instead of your head, and then view it on the computer as well... the same computer you were going to view the file in the first place? These files aren't being printed on paper... you have access to a computer: use it!
vivzkestrel 1y ago
The world is growing tired of YAML so someone invented MAML yesterday on HN https://news.ycombinator.com/item?id=45562056 https://news.ycombinator.com/item?id=45562056
theknarf 1y ago
Big fan of HCL as the configuration language to rule them all, being able to abstract stuff into reusable modules at the configurations language level is great. And there are implementation for HCL in multiple programming languages.
dapperdrake 1y ago
A "universal language" is a language with (a) sequences of instructions, (b) conditionals, ifs, branches, and (c) loops, repetition, iteration, y-combinator (the math one), recursion. They are Turing complete. Leaving off any of the three features (difficult for the y-combinator) yields a non-Turing complete language, for example Sieve Script for filtering email lacks (c). What the definition doesn’t cover is parameterization. Some people call this abstraction. Technically, lisp macros also fall under parameterization. With parameterization it really seems like any and all functions, mappings, and operators that fail to be injective and composable are counter-productive in practice. The math term is "generative effects". There seems to be this continuous contention between one side that wants its configuration files to basically be CSV and another side that, effectively, wants a full-blown programming language as their "configuration language". Both sides have a point. Just happen to land in the "essentially CSV" camp myself. Even a lisp macro can generate CSV. (Yes the phrase "universal language" is very difficult to search for if the specific academic term above is under consideration.)
Twey 1y ago
When Kubernetes exposed its APIs as declarative configuration objects, I get the impression that they didn't originally mean for people to write the configuration by hand. The YAML/JSON/… is a conveniently universal interchange format for interfacing with Kubernetes from bindings, and representing target state as documents is just a good way to encode idempotence in the API. I'd be interested to hear from someone involved with early Borg/K8s development what the original intention was.
dizlexic 1y ago
That's why I exclusively write my config files in PHP and output JSON \s
xg15 1y ago
Friends don't let friends write turing complete config languages.

102 points by unripe_syntax 1y ago

As someone who's spent most of their career in cloud IAC, and likes to think they are pretty read up on the latest going on in that world, if you didn't know better you'd think YAML is one of the greatest threats facing mankind. There are plenty of things I certainly hate about it, but every configuration syntax I've ever used I have similar gripes about. It's like once a month this kind of "The world is growing tired of yaml" claim is just thrown out there like everyone just agrees with it. Choose something that works for you. This author repeatedly mentions TOML but there are plenty of issues with that one I could point out too. Syntax is one very small part of what makes an ecosystem great or not so great. Most of my exposure to yaml is helm chart templates, which admittedly is not pure YAML, but it works fine enough for me, at least to where I don't feel like writing lengthy blog posts about how much I hate it. I even wrote a library that converts yaml templates to HCL for internal use because I got so sick of people having this exact same argument like it deeply mattered. And guess what? They hate the HCL too.

compyman 1y ago
I also think that a lot of the problems with yaml specifically are overblown, but this post is actually not about that! It is specifically saying the same problem exists in JSON/YAML/TOML, etc, which is that all these configuration languages don't have any real means of abstraction, and ultimately aren't expressive enough do to the job we require of them. as soon as you are templating config files with other configs, I agree, I have sorely felt this limitation with helm charts
- kmoser 1y ago
  Serious question: do people who work with these config files frequently, or on large such files, use simple text editors, or are there "smart" editors that do things like prevent you from making typos or inserting the wrong data type, similar to an HTML form that does basic validation or a DB schema that rejects bad data? There is no single cure-all, of course, but surely we should be relying on computers to do much of the heavy lifting when it comes to validation and verification of these files, not just as linters after the fact but in realtime while we're editing, and with some sort of knowledge (even if derived programmatically) of what is right and what is wrong so we no longer have to worry about simple footguns.
  - kokada 1y ago
    I think one of the problems of those "configuration languages" is that you can extract semantic information without knowing the target, e.g., with has a specific meaning in GitHub Actions but it is otherwise an unremarkable word in the YAML specification. But when working with real programming languages it is completely different, you can take semantic information from the current code, and you can have things like types to give you safety.
    - JohnMakin 1y ago
      The problem is most configuration languages are declarative vs imperative like most “real” languages are. You could probably levy the same complaint against declarative languages in general - it’s just a different way of thinking
      - kokada 1y ago
        Nix as used in NixOS is a declarative language and there is none of the issues I cited by being a "real" programming language (or as the article talks about, having "abstractions" like builtin.map). You can pretty easily setup a LSP to get code-completion (even between different projects, like NixOS vs Home-Manager). There is no proper type system in Nix but the module system does supplement it well.
taeric 1y ago
I'm trying to remember the phrase. Something like, "there is nothing as vicious as low stakes fights." Trying that on Google gets me https://en.wikipedia.org/wiki/Sayre%27s_law https://en.wikipedia.org/wiki/Sayre%27s_law. Is about right. :D
- jdmichal 1y ago
  It's like bike shedding. It's a side effect of mixed expertise (and confidence) working together on things that are only partially understood by all. When something is clearly outside one's expertise, they are content to leave it to others. But then you'll get minor questions with low stakes like "what color to paint the shed". And how everyone feels like they can participate, so suddenly there's a huge discussion / debate / argument about a very, very minor thing.
skywhopper 1y ago
HCL is generally great, but has some issues with clarity of transformation to the underlying data structures. But anytime someone suggests TOML I have to double check to be sure they are serious because the TOML syntax for anything more complicated than single-layer maps is mind-bogglingly confusing to me. This is not a serious alternative to YAML.
qezz 1y ago
> And guess what? They hate the HCL too. Don't want to sound too harsh, but to me HCL is even worse than plain YAML. By expressiveness, HCL is somewhat similar to Ansible-flavoured YAML - in both you need to use magic keywords to create any kind of abstraction (e.g. a loop). HCL is worse than regular YAML because there's only one "true" parser for it, that is official Hashicorp's HCL parser. So if you are locked into Golang ecosystem, then sure it can work for you, otherwise you are out of luck. There are a couple of tools that convert HCL into JSON, I tried both, they somewhat work, but in the end of the day it's a big hack. At that point I just gave up on using HCL and started using something else that generates JSON. Hope you find some configuration layer that fits your users more than HCL or YAML.
- JohnMakin 1y ago
  Not exactly true in terms of being the only parser - OpenTofu is a good project I’m highly supportive of

deleted 1y ago

[deleted]

Defletter 1y ago

Really wish people would just bite the bullet and do configuration as code instead of trying to make all these config petlangs.

patrickmay 1y ago
Exactly. Emacs Lisp is an existence proof that this can be done well.
- taeric 1y ago
  You beat me to it! And for those that haven't taken a look at it, the "customize" menu and everything it supports is silly impressive. And it just writes out the results out, like a boss.* * Obviously, it has a lot of "don't edit below this line" complexity to it. But that doesn't change that it is right there.
skydhash 1y ago
My preference is towards simpler formats like: option value Easy to edit and manipulate. JSON and YAML is always a nightmare if it's user facing. As for ansible, I'd love to see some scheme/lisp variants.
- mrheosuper 1y ago
  then how you distinguish between string "52" and number 52 ? Keep adding more edge cases and you have something resembles JSON
  - skydhash 1y ago
    Why do you need to differentiate between the two as an input? It’s config, not random data. If you have email test@example.com logo-size 100 background #adadaf modules auth modules db modules files The only reason to have special token here is if you multiple line values. Types are not a concern.
  - bandie91 1y ago

apalmer 1y ago

I don't think the title and the article really communicates it's case well. Did not understand the goal until 90% through the article when they showed the source code of RCL with the loops. This isn't syntax vs abstraction. This is how much programming language power do you want to enable in your configuration language. This is a big difference and I think we miss the interesting part of that discussion because we dip into this 'abstraction angle.

dapperdrake 1y ago
The "abstraction angle" seems to be a different encoding for "power of configuration language."
jasaldivara 1y ago
I think if people want a more powerful, programmable config language, maybe they should use something like Lua or Scheme instead of reinventing the wheel with those new niche languages.

dvrp 1y ago

Should be titled “the power spectrum of data” or something similar the article talks about the trade off between plain data structure versus abstract ones and that’s the main issue

nickelpro 1y ago

Yes, one more DSL on your Tower of Babel tech stack will save you. If you want configuration-as-code use Python. Please. Or Tcl if you must. Do not invent N+1 DSL for your engineers to waste time learning.

qezz 1y ago
Luckily, the main point of the article is that syntax doesn't matter, but abstractions do. In other words, you can use your favorite DSL, that being Python or TCL or something else.

kgen 1y ago

Code as configuration loses one primary benefit which is being able to read the actual config and know exactly what will apply. In the example in the article you would get the same benefit by disallowing users to edit the config directly and instead require the config to be generated via a cmdline app/service that encodes the same policy?

Splizard 1y ago

The problem with configuration formats, is not syntax, or abstraction, it's the lack of consistent language server integration, it's problem when I can't lookup the definition for a key, the expected type, or quickly jump to definitions that clearly show which keys are available to configure.

atoav 1y ago
To be frank, the clear problem with configuration format is that people have configurations so complex they probably should use something else. Example: We are programming a backend for a blog. If we were to not use templates, but instead try to get that functionality via the webservice configuration we would have to "invent" some format that gives us the flexibility of templates within let's say a YAML file. Needless to say that would be a horrible idea. Maybe I am being naive here, but I have yet to be convinced of the fact that it is really configuration formats that are the problem and not what people try to abuse them for. I have yet to work on a project where TOML wasn't enough for actual configuration. Usually when I need something more complex than what can be done with TOML it is a sign that this needs to be handled differently. Via templates or with a database or making a special DSL-like script file. E.g. if you're using python nothing (except security considerations) stops you from allowing users to use a python file for configuration. If your configuration needs are really that complex, why not use a real programming language for it?
- skydhash 1y ago
  Two examples of complex user facing configurations I can think of are pretty trivial to implement: - Decision tree, where you only need comparison operators. The leaves are a specified list of actions. - actions list with macros (variables). You can be fancy and add some conventions for arrays. Anything more that that should just be a programming language. And if the relationship is adversarial (saas), you should really think hard about needing something that complex.

weavejester 1y ago

To me the solution seems like it's adding complexity that could cause more issues further down the line. The specific problems in the example could be solved by changing how the data is represented. Consider the following alternative representation, written in edn: {:aws.s3/buckets {:aws.region/eu-west {:alpha-hourly {:lifecycle/policy {:delete-after #interval/days 4}} :alpha-daily {:lifecycle/policy {:delete-after #interval/days 30}} :alpha-monthly {:lifecycle/policy {:delete-after #interval/days 365}} :bravo-hourly {:lifecycle/policy {:delete-after #interval/days 4}} :bravo-daily {:lifecycle/policy {:delete-after #interval/days 30}} :bravo-monthly {:lifecycle/policy {:delete-after #interval/days 365}}}}} This prevents issues where the region is mistyped for a single bucket, makes the interval more readable by using a custom tag, and as a bonus prevents duplicate bucket names via the use of a map. Obviously this doesn't prevent all errors, but it does prevent the specific errors that the RCL example solves, all without introducing a Turing-complete language.

MathMonkeyMan 1y ago
> The specific problems in the example could be solved by changing how the data is represented. Finding the "right" representation for a given set of data is an interesting problem, but most (all) of the time the representation is specified by someone/something else. In the past I've written a [preprocessor][1] that adds some power to the representation while avoiding general purpose computation. For example, (buckets (let ([(regional region (name policy) ...) ((['name name] ['region region] ['lifecycle_policy policy]) ...)]) (regional us-west (alpha-hourly (delete_after_seconds 345600)) (alpha-daily (delete_after_seconds 2592000)) (alpha-monthly (delete_after_seconds 31536000)) (bravo-hourly (delete_after_seconds 345600)) (bravo-daily (delete_after_seconds 259200)) (bravo-monthly (delete_after_seconds 31536000))))) Macros, basically. Arithmetic would help there, but that might be too much. [1]: https://github.com/dgoffredo/llama https://github.com/dgoffredo/llama

yawnxyz 1y ago

the answer seems to be both of both worlds - if you're going to do for loops why not just use python? the answer is both a faux programming language, and really bad ux / really hard to read / scan maybe what they need is a program that generates better readable text; and somehow you can flip between the determinism of code and ux of readable text?! (is that possible)

transfire 1y ago

FOR loops? YAML has a merge key <<:, which might be helpful. The merge key is a clever little trick, but it depends of the special hash key, so lists can’t be merged. Syntax does matter, which is why YAML matters — even if imperfect.

rattyJ2 1y ago
Merges and anchors are some of the least maintaineable and most error-prone config i've seen. Doesn't help every yaml parser has their own opinion on what a merge or an anchor should do, exactly.

rednafi 1y ago

The intention behind configuration languages is sound but there’s just too many of them. Not having a universally accepted one makes picking one harder. Also, migrating from one to another isn’t as straightforward. Plus, for database in ["alpha", "bravo"]: for period, days in period_retention_days: At this point, you’re better off writing a Python script that spits out some JSON. I‘m aware that even the blog mentions it. The benefit is - having a more expressive language at your fingertips and not having to fight your peers while trying to add yet another idiosyncratic dependency.

sam_bristow 1y ago
Stoke Space[1] uses a similar system, letting people write arbitrary code to generate a static configuration for their launch vehicle. It means you get all the power of something like Python during development but also a deterministic, bounded config for the critical flight systems. I think their config files are just TOML that is consumed by Rust. I'll try dig out a link to the talk one of their Flight Software Engineers did on the concept. [1] https://www.stokespace.com/ https://www.stokespace.com/

lwhsiao 1y ago

I'd be curious what the author thinks of KSON, which was also recently featured on HN [1]. [1]: https://news.ycombinator.com/item?id=45291858 https://news.ycombinator.com/item?id=45291858

ruuda 1y ago
I mention it in the first paragraph, and what I think of it in the second paragraph.

gorgoiler 1y ago

Somewhere along the way we got lost into thinking config files were remote procedure calls, and that YAML was the only RPC interface available to us. The caller generates YAML, the receiver parses it, but more often than not the two are on the same host and use the same language*: def say(message): o = dict(f=say, v=message)) call(yaml.dumps(o)) …meanwhile elsewhere… def eval(request): o = yaml.loads(request.body) match o[“f”]: case “say”: print(o[“v”]) Yes, referring to this as “madness” skips over the fact that you can now scale up your hello world printer across the internet, have auth, rate limit, etc etc. but for so many things the RPC just isn’t needed at all. As this article gets at, config files are one of those things, and they benefit hugely from the rampant abstraction violation of skipping an intermediary text format between the program doing the configuration and the program doing the work. *or have internal versions of their APIs available in matching languages.

radarsat1 1y ago

I got confused reading this because I wasn't sure how I, as a reader with no knowledge of the system under discussion, was supposed to know that all the buckets should be in the same region. Nor is it clear to me how the "for loop" version would handle the case where exceptionally one bucket is different. Which is a more interesting discussion imho, that's the whole point of having it as a configuration field, after all.

Joker_vD 1y ago
Like this: region = if name == "bravo-hourly" then "us-west" else "eu-west",
- radarsat1 1y ago
  I see, not so clean imho but I guess it works.
qezz 1y ago
The example in the article shows a need for cartesian product of (bucket name) and (lifetime policy), with a fixed location. Nothing stops you from defining a separate category for the exceptions, and just append them to the resulting list. Or you may want to assign each bucket its own location, then you define it along side the bucket name.

teo_zero 1y ago

Before going all in with turing-complete configuration languages, why not simply augmenting existing declarative formats with parameter & arithmetic expansion à la bash? It's a syntax already familiar to many, that would mitigate the issues highlighted in TFA.

texuf 1y ago

Yeah no. The reason we don't write code in config files is that, if we did, in order to know what you are deploying you have to run the code in your head. Not just this code, the code that Steve wrote six months ago before people noticed he was grossly incompetent and fired. Also worth mentioning, I can run this code in my head. The author can run this code in their head, but not everyone can. And finally, if the author thinks copy pasting yaml files cause bugs, wait till the llms start copy pasting the for loops.

saurik 1y ago
Or you could run the code on the computer, instead of your head, and then view it on the computer as well... the same computer you were going to view the file in the first place? These files aren't being printed on paper... you have access to a computer: use it!

vivzkestrel 1y ago

The world is growing tired of YAML so someone invented MAML yesterday on HN https://news.ycombinator.com/item?id=45562056 https://news.ycombinator.com/item?id=45562056

theknarf 1y ago

Big fan of HCL as the configuration language to rule them all, being able to abstract stuff into reusable modules at the configurations language level is great. And there are implementation for HCL in multiple programming languages.

dapperdrake 1y ago

A "universal language" is a language with (a) sequences of instructions, (b) conditionals, ifs, branches, and (c) loops, repetition, iteration, y-combinator (the math one), recursion. They are Turing complete. Leaving off any of the three features (difficult for the y-combinator) yields a non-Turing complete language, for example Sieve Script for filtering email lacks (c). What the definition doesn’t cover is parameterization. Some people call this abstraction. Technically, lisp macros also fall under parameterization. With parameterization it really seems like any and all functions, mappings, and operators that fail to be injective and composable are counter-productive in practice. The math term is "generative effects". There seems to be this continuous contention between one side that wants its configuration files to basically be CSV and another side that, effectively, wants a full-blown programming language as their "configuration language". Both sides have a point. Just happen to land in the "essentially CSV" camp myself. Even a lisp macro can generate CSV. (Yes the phrase "universal language" is very difficult to search for if the specific academic term above is under consideration.)

Twey 1y ago

When Kubernetes exposed its APIs as declarative configuration objects, I get the impression that they didn't originally mean for people to write the configuration by hand. The YAML/JSON/… is a conveniently universal interchange format for interfacing with Kubernetes from bindings, and representing target state as documents is just a good way to encode idempotence in the API. I'd be interested to hear from someone involved with early Borg/K8s development what the original intention was.

dizlexic 1y ago

That's why I exclusively write my config files in PHP and output JSON \s

xg15 1y ago

Friends don't let friends write turing complete config languages.