5 ms·

Yeah, what I want is JSON with comments and nice multi-line string support. I don't like how much syntactic magic YAML does (I don't need or want the country co

Yeah, what I want is JSON with comments and nice multi-line string support. I don't like how much syntactic magic YAML does (I don't need or want the country code for Norway to be parsed as a boolean False value). I still don't know what the exclamation point does (e.g., !Ref). And clearly YAML is the wrong tool for infra-as-code since CloudFormation has to build a macro system, conditionality, referencing, and a couple different systems for defining and calling functions (templates being one and their implicit functions being another). We also see tools like Troposphere and CDK which are effectively different ways to generate CloudFormation YAML via programming languages (or more precisely programming languages that were designed for humans). And it's not just limitations inherent to CloudFormation--Helm has long had templates for generating YAML, but those also weren't sufficiently powerful/expressive/ergonomic so Helm3 is supporting Lua as well. And as I understand it, Terraform is constantly adding more and more powerful features into HCL. So what's the solution? It's pretty simple--we should keep the YAML around, but it should be the intermediate representation (IR), not the human interface. The human interface should be something like a functional language[^1] (or an imperative language that is written in a functional style) that evaluates to that YAML IR layer. The IR is then passed to something like Kubernetes or Terraform or CloudFormation which understand it, but it's not the human interface. As for the high-level language, something like [Starlark][0] would work well. It's purpose-built for being an evaluated configuration language. However, I would argue that a static type system (at least an optional static type system) is important--it's easy enough to imagine someone extending Starlark with type annotations and building a static type checker (which is much easier for Starlark since it's a subset of Python which is intended to be amenable to static analysis). This, I think, is the proper direction for infrastructure-as-code tooling. [^1]: Functional in that it is declarative instead of imperative--not necessarily that the syntax should be as hard to read as OCaml or Haskell. Also, while YAML is also declarative, it doesn't have a notion of evaluation or variables. [0]: https://docs.bazel.build/versions/master/skylark/language.html https://docs.bazel.build/versions/master/skylark/language.ht...

chris11 7y ago
TeamCity actually lets you set job configurations with kotlin in source control. The config is actually a kotlin dsl, so not only can you import settings, you can also have TeamCity generate projects, jobs, and settings from your code.
- ljm 7y ago
  Gradle also uses Kotlin these days.
visualphoenix 7y ago
Have you looked at jsonnet[0]? It has multiline strings and comments among other features... [0]: https://jsonnet.org/ref/spec.html https://jsonnet.org/ref/spec.html
dilyevsky 7y ago
We’ve been using starlark for kubernetes configs at my prev gig and I quite liked it. Open-sourced some of that stuff as https://github.com/cruise-automation/isopod https://github.com/cruise-automation/isopod (which is based on https://github.com/stripe/skycfg https://github.com/stripe/skycfg). I hear stripe are also using their thing with terraform although not sure to which extent.
sedachv 7y ago
I do not understand the need for all of these different new language implementations and data formats. GuixSD vs NixOS already showed that Scheme is a superior solution as a configuration language, scripting language, template language, and intermediate representation. A single language that has 30+ years of successful production use, tons of books and documentation. Why re-invent the wheel in four different, incompatible ways?
- weberc2 7y ago
  Is NixOS a scheme? Anyway, we moved away from Nix because of all its problems (the language being only a medium-sized one). Also, I detest CMake, but by your own standard (longevity, popularity), it is better than Nix or Guix. Frankly those tools haven’t shown themselves to be “superior” in any meaningful way. Yes, they have been around for a while, but having been around for a long time and not enjoying any significant adoption is not very compelling.
  - sedachv 7y ago
    > Is NixOS a scheme? That question does not make any sense. I am talking about Scheme the programming language: https://schemers.org/ https://schemers.org/ NixOS is a GNU/Linux distribution built on top of the Nix package manager. The Nix package manager has its own custom configuration language. GuixSD is a GNU/Linux distribution built on top of the Guix package manager. GuixSD uses Guile Scheme as the package configuration language, system configuration language, scripting language, and implementation language for many of the system services (such as init). GuixSD does more things better with an existing standard programming language than NixOS does with its own custom programming language. > Also, I detest CMake, but by your own standard (longevity, popularity), it is better than Nix or Guix. What does CMake have to do with anything? > Yes, they have been around for a while What are you talking about? The first stable version of NixOS, 13.10, was released in 2013. GuixSD 1.0 was only released this May. Your post is hard to make sense of.
    - weberc2 7y ago
      Yeah, I misunderstood your post—sorry about that. I thought you said “Guix and Nix show us that scheme is the answer”. In any case, I don’t know how you conclude that Guix won over Nix nor how you conclude that the winner is superior to all other configuration languages. It’s especially counter-intuitive that it should be an intermediate representation—the IR should not be executable, it should only be descriptive. The IaC technology shouldn’t need to interpret anything, and the language you pass into it therefore shouldn’t support evaluation/execution.
mbo 7y ago
I replied with this elsewhere in the thread, but maybe Dhall? https://dhall-lang.org/ https://dhall-lang.org/ Functional? Check. Type annotations? Check. Not Turing complete? Check. Compiles to multiple config formats? Check.
- weberc2 7y ago
  Yeah, I think Dhall is the right idea, but I think it’s going to have the same syntax/ergonomics issues that Haskell and OCaml suffer from. Our company invested heavily in Nix, but the developers really struggled with the expression language which seems quite similar to Dhall both syntactically and ergonomically. While Dhall might be great for Haskell/OCaml/F# shops, a configuration language isn’t the right place to push for new idioms or ways to think about programming.

chris11 7y ago

TeamCity actually lets you set job configurations with kotlin in source control. The config is actually a kotlin dsl, so not only can you import settings, you can also have TeamCity generate projects, jobs, and settings from your code.

ljm 7y ago
Gradle also uses Kotlin these days.

visualphoenix 7y ago

Have you looked at jsonnet[0]? It has multiline strings and comments among other features... [0]: https://jsonnet.org/ref/spec.html https://jsonnet.org/ref/spec.html

dilyevsky 7y ago

We’ve been using starlark for kubernetes configs at my prev gig and I quite liked it. Open-sourced some of that stuff as https://github.com/cruise-automation/isopod https://github.com/cruise-automation/isopod (which is based on https://github.com/stripe/skycfg https://github.com/stripe/skycfg). I hear stripe are also using their thing with terraform although not sure to which extent.

sedachv 7y ago

I do not understand the need for all of these different new language implementations and data formats. GuixSD vs NixOS already showed that Scheme is a superior solution as a configuration language, scripting language, template language, and intermediate representation. A single language that has 30+ years of successful production use, tons of books and documentation. Why re-invent the wheel in four different, incompatible ways?

weberc2 7y ago
Is NixOS a scheme? Anyway, we moved away from Nix because of all its problems (the language being only a medium-sized one). Also, I detest CMake, but by your own standard (longevity, popularity), it is better than Nix or Guix. Frankly those tools haven’t shown themselves to be “superior” in any meaningful way. Yes, they have been around for a while, but having been around for a long time and not enjoying any significant adoption is not very compelling.
- sedachv 7y ago
  > Is NixOS a scheme? That question does not make any sense. I am talking about Scheme the programming language: https://schemers.org/ https://schemers.org/ NixOS is a GNU/Linux distribution built on top of the Nix package manager. The Nix package manager has its own custom configuration language. GuixSD is a GNU/Linux distribution built on top of the Guix package manager. GuixSD uses Guile Scheme as the package configuration language, system configuration language, scripting language, and implementation language for many of the system services (such as init). GuixSD does more things better with an existing standard programming language than NixOS does with its own custom programming language. > Also, I detest CMake, but by your own standard (longevity, popularity), it is better than Nix or Guix. What does CMake have to do with anything? > Yes, they have been around for a while What are you talking about? The first stable version of NixOS, 13.10, was released in 2013. GuixSD 1.0 was only released this May. Your post is hard to make sense of.
  - weberc2 7y ago
    Yeah, I misunderstood your post—sorry about that. I thought you said “Guix and Nix show us that scheme is the answer”. In any case, I don’t know how you conclude that Guix won over Nix nor how you conclude that the winner is superior to all other configuration languages. It’s especially counter-intuitive that it should be an intermediate representation—the IR should not be executable, it should only be descriptive. The IaC technology shouldn’t need to interpret anything, and the language you pass into it therefore shouldn’t support evaluation/execution.

mbo 7y ago

I replied with this elsewhere in the thread, but maybe Dhall? https://dhall-lang.org/ https://dhall-lang.org/ Functional? Check. Type annotations? Check. Not Turing complete? Check. Compiles to multiple config formats? Check.

weberc2 7y ago
Yeah, I think Dhall is the right idea, but I think it’s going to have the same syntax/ergonomics issues that Haskell and OCaml suffer from. Our company invested heavily in Nix, but the developers really struggled with the expression language which seems quite similar to Dhall both syntactically and ergonomically. While Dhall might be great for Haskell/OCaml/F# shops, a configuration language isn’t the right place to push for new idioms or ways to think about programming.