4 ms·

Couple of contenders: - Jsonnet (https://jsonnet.org/ https://jsonnet.org/) - simpler syntax and less concepts to learn, just an extension of JSON. But no type

Couple of contenders: - Jsonnet (https://jsonnet.org/ https://jsonnet.org/) - simpler syntax and less concepts to learn, just an extension of JSON. But no type checking. An open source offspring of Google's internal config language (GCL/BCL) - Cue (https://github.com/cuelang/cue https://github.com/cuelang/cue) - a more ambitious attempt to fix GCL/BCL by replacing inheritance as the fundamental compositional primitive with constraint unification. Great thread comparing them against each other by the authors of both: https://github.com/cuelang/cue/issues/33 https://github.com/cuelang/cue/issues/33 Cue seems kind of similar to Dhall on first sight, but I haven't used either enough for an informed opinion yet.

Ericson2314 7y ago
Cue is a bit too cute trying to combine the subtyping and inhabitance relations into one.
- skybrian 7y ago
  What problems do you see?
dharmab 7y ago
We tried to introduce Jsonnet at our org. It failed miserably because ops kept mistaking the name for JSON which they hated. (International multilingual team). It was a real shame because ops then implemented some features of Jsonnet via scripts to to parse and merge YAML. What was 0 LOC in Jsonnet is now about 300 LOC plus custom CI checkers, all because of a marketing problem.
- Fnoord 7y ago
  If I go to the mentioned Jsonnet homepage it says "A simple extension of JSON". The graphic explains its relation to JSON. The example looks awfully similar to JSON. What I don't understand is the following: config files are read by text editors, and in the end, by human beings. Because of the latter they should have certain traits. We must agree on the importance of these traits before we can settle on a standard. For me, important features are that they must be readable, and easily editable. They must be readable with a certain text editor (vi) for backwards compatibility. So that means it shouldn't require syntax highlighting or schema. Well, these 2 simple requirements of mine rule out anything remotely resembling JSON. It just appears to me that JSON is for JavaScript developers, YAML for Python developers, and Dhall for ML (the whole family I suppose, not just Haskell) developers. Well then if we're going that route then perhaps all we need is some kind of glue between text config and binary config (which reminds me of Systemd...). Ie. that it accepts multiple config file formats.
  - jholman 7y ago
    I'm a little confused by one part of what you wrote. You say that your config files must be readable by vi, and that in turn adds a no-highlighting-required constraint and a no-schema-required constraint, and that in turn adds a no-JSON-nor-anything-like-it constraint. I deal with JSON all the time in vim, effortlessly. I'd be willing to deal with it in Notepad if necessary, and certainly in non-vim vi. Pipe it through a prettifier (lately I use `jq . file.json`) if necessary. I don't need syntax highlighting, and I don't need a formalized schema (although I certainly appreciate an informal one, interpreted by the 1.0 Human Meatbrain I carry around). Also, if by "vi" you meant "vim", this is EVEN MORE confusing, because vim syntax-highlights JSON.
    - Fnoord 7y ago
      With vi I mean vi, not vim. If I meant vim, I'd have written vim. I use vim if its available (with my own configuration which includes syntax highlighting), but it isn't always available. With my Human Meatbrain syntax highlighter I have far more issues with JSON than with say YAML or any other markup language. Consider, for example, how easy the syntax is of a Wireguard configuration file. It is basically akin to a shell script or ini configuration file. And these have a proven track record. Why is that way of configuration broken in the first place? You could do things such as variables in shell scripts as well. I also believe that the whole Systemd drama is basically because of moving away from such a proven track record. And it might very well be true that shell scripts are slow. That is why I argue for backwards compatibility and converting to/from formats. Which is something Dhall is able to (it can convert to/from YAML and JSON).
lihaoyi 7y ago
We make heavy use of Jsonnet at work (https://databricks.com/blog/2017/06/26/declarative-infrastructure-jsonnet-templating-language.html https://databricks.com/blog/2017/06/26/declarative-infrastru...). It's worked great. Having a hermetic, pure templating system whose only output is a set of JSON/YAML files means that you can refactor fearlessly: as long as the materialized JSON/YAML doesn't change, you are 100% sure your refactor is safe. Bazel's StarLark dialect of Python (https://github.com/bazelbuild/starlark https://github.com/bazelbuild/starlark) has similar benefits. The language is simple and remarkably well specified, enough that we implemented our own intellij plugin (https://plugins.jetbrains.com/plugin/10852-jsonnet https://plugins.jetbrains.com/plugin/10852-jsonnet) and even our own faster compiler (https://github.com/databricks/sjsonnet https://github.com/databricks/sjsonnet) without much effort at all. There are odd corners in the language, but not something that most people will end up bumping into in typical usage. The templates certainly get messy in large configurations, but no more messy than any other code, and the hermeticity/purity greatly helps in managing the messiness. It's certainly less messy/odd than the copy-paste configs or be-spoke JSON/YAML templating systems that inevitably appear in messy deployment environments! The last thing of note is the lack of static types: this definitely affects usability to some extent, and especially hinders IDE support from being as useful as it is in e.g. Java. But having a useful/ergonomic type system that fits this specific problem space is probably still an unsolved research question.
jbergstroem 7y ago
My favorite is still the nginx config-like libucl: https://github.com/vstakhov/libucl https://github.com/vstakhov/libucl
cppforlife 7y ago
i'll throw in one of my projects as a contender: ytt - YAML templating tool - https://get-ytt.io https://get-ytt.io (check out live playground!). it works with yaml structures (hence avoids text templating problems) and uses familiar python-like language, starlark, making quite easy to get started. it makes use of yaml comments to assign metadata/templating directives to yaml nodes, so it looks something like this: #@ load("@ytt:data", "data") #@ def labels(): app: echo org: test #@ end kind: Pod apiVersion: v1 metadata: name: echo-app labels: #@ labels() spec: containers: #@ for/end echo in data.values.echos: - name: #@ echo.name image: hashicorp/http-echo args: - #@ "-text=" + echo.text it doesn't include type checking, however, it does have a system to "overlay" structures on top of each other via overlay feature -- https://github.com/k14s/ytt/blob/master/docs/lang-ref-ytt-overlay.md https://github.com/k14s/ytt/blob/master/docs/lang-ref-ytt-ov.... merge/replace/remove operations expect to find one node by default so map key typos or wrong structural nesting problems are caught easily in common cases.

Ericson2314 7y ago

Cue is a bit too cute trying to combine the subtyping and inhabitance relations into one.

skybrian 7y ago
What problems do you see?

dharmab 7y ago

We tried to introduce Jsonnet at our org. It failed miserably because ops kept mistaking the name for JSON which they hated. (International multilingual team). It was a real shame because ops then implemented some features of Jsonnet via scripts to to parse and merge YAML. What was 0 LOC in Jsonnet is now about 300 LOC plus custom CI checkers, all because of a marketing problem.

Fnoord 7y ago
If I go to the mentioned Jsonnet homepage it says "A simple extension of JSON". The graphic explains its relation to JSON. The example looks awfully similar to JSON. What I don't understand is the following: config files are read by text editors, and in the end, by human beings. Because of the latter they should have certain traits. We must agree on the importance of these traits before we can settle on a standard. For me, important features are that they must be readable, and easily editable. They must be readable with a certain text editor (vi) for backwards compatibility. So that means it shouldn't require syntax highlighting or schema. Well, these 2 simple requirements of mine rule out anything remotely resembling JSON. It just appears to me that JSON is for JavaScript developers, YAML for Python developers, and Dhall for ML (the whole family I suppose, not just Haskell) developers. Well then if we're going that route then perhaps all we need is some kind of glue between text config and binary config (which reminds me of Systemd...). Ie. that it accepts multiple config file formats.
- jholman 7y ago
  I'm a little confused by one part of what you wrote. You say that your config files must be readable by vi, and that in turn adds a no-highlighting-required constraint and a no-schema-required constraint, and that in turn adds a no-JSON-nor-anything-like-it constraint. I deal with JSON all the time in vim, effortlessly. I'd be willing to deal with it in Notepad if necessary, and certainly in non-vim vi. Pipe it through a prettifier (lately I use `jq . file.json`) if necessary. I don't need syntax highlighting, and I don't need a formalized schema (although I certainly appreciate an informal one, interpreted by the 1.0 Human Meatbrain I carry around). Also, if by "vi" you meant "vim", this is EVEN MORE confusing, because vim syntax-highlights JSON.
  - Fnoord 7y ago
    With vi I mean vi, not vim. If I meant vim, I'd have written vim. I use vim if its available (with my own configuration which includes syntax highlighting), but it isn't always available. With my Human Meatbrain syntax highlighter I have far more issues with JSON than with say YAML or any other markup language. Consider, for example, how easy the syntax is of a Wireguard configuration file. It is basically akin to a shell script or ini configuration file. And these have a proven track record. Why is that way of configuration broken in the first place? You could do things such as variables in shell scripts as well. I also believe that the whole Systemd drama is basically because of moving away from such a proven track record. And it might very well be true that shell scripts are slow. That is why I argue for backwards compatibility and converting to/from formats. Which is something Dhall is able to (it can convert to/from YAML and JSON).

lihaoyi 7y ago

We make heavy use of Jsonnet at work (https://databricks.com/blog/2017/06/26/declarative-infrastructure-jsonnet-templating-language.html https://databricks.com/blog/2017/06/26/declarative-infrastru...). It's worked great. Having a hermetic, pure templating system whose only output is a set of JSON/YAML files means that you can refactor fearlessly: as long as the materialized JSON/YAML doesn't change, you are 100% sure your refactor is safe. Bazel's StarLark dialect of Python (https://github.com/bazelbuild/starlark https://github.com/bazelbuild/starlark) has similar benefits. The language is simple and remarkably well specified, enough that we implemented our own intellij plugin (https://plugins.jetbrains.com/plugin/10852-jsonnet https://plugins.jetbrains.com/plugin/10852-jsonnet) and even our own faster compiler (https://github.com/databricks/sjsonnet https://github.com/databricks/sjsonnet) without much effort at all. There are odd corners in the language, but not something that most people will end up bumping into in typical usage. The templates certainly get messy in large configurations, but no more messy than any other code, and the hermeticity/purity greatly helps in managing the messiness. It's certainly less messy/odd than the copy-paste configs or be-spoke JSON/YAML templating systems that inevitably appear in messy deployment environments! The last thing of note is the lack of static types: this definitely affects usability to some extent, and especially hinders IDE support from being as useful as it is in e.g. Java. But having a useful/ergonomic type system that fits this specific problem space is probably still an unsolved research question.

jbergstroem 7y ago

My favorite is still the nginx config-like libucl: https://github.com/vstakhov/libucl https://github.com/vstakhov/libucl

cppforlife 7y ago

i'll throw in one of my projects as a contender: ytt - YAML templating tool - https://get-ytt.io https://get-ytt.io (check out live playground!). it works with yaml structures (hence avoids text templating problems) and uses familiar python-like language, starlark, making quite easy to get started. it makes use of yaml comments to assign metadata/templating directives to yaml nodes, so it looks something like this: #@ load("@ytt:data", "data") #@ def labels(): app: echo org: test #@ end kind: Pod apiVersion: v1 metadata: name: echo-app labels: #@ labels() spec: containers: #@ for/end echo in data.values.echos: - name: #@ echo.name image: hashicorp/http-echo args: - #@ "-text=" + echo.text it doesn't include type checking, however, it does have a system to "overlay" structures on top of each other via overlay feature -- https://github.com/k14s/ytt/blob/master/docs/lang-ref-ytt-overlay.md https://github.com/k14s/ytt/blob/master/docs/lang-ref-ytt-ov.... merge/replace/remove operations expect to find one node by default so map key typos or wrong structural nesting problems are caught easily in common cases.