7 ms·

A radically different alternative with a lot going for it is Starlark: https://github.com/bazelbuild/starlark https://github.com/bazelbuild/starlark It’s a det

A radically different alternative with a lot going for it is Starlark: https://github.com/bazelbuild/starlark https://github.com/bazelbuild/starlark It’s a deterministic subset of Python. This means that if you have complex or repetitive configurations, you can use loops and functions to structure them. But it’s impossible to write an infinite loop or recursion.

remram 5y ago
Interesting! I started using jsonnet this year, but found that the language was needlessly quirky (e.g. the `::`, purely functional aspect, and no one wants to learn a new language to write configuration in the first place). More importantly, it is extremely slow (lazy evaluation without memoization...): rendering the Kubernetes YAML of my 5-container app taking over 10 seconds... I will look into this further.
vlovich123 5y ago
Have you had any experience building on top of it directly outside of blaze/bazel?
- dilyevsky 5y ago
  I have - https://github.com/cruise-automation/isopod https://github.com/cruise-automation/isopod and more generally with https://github.com/stripe/skycfg https://github.com/stripe/skycfg
civilized 5y ago
Not a bad idea but only implemented in Rust, Go, and Java so far. Meanwhile, all sorts of languages can interpret JSON and YAML. It's a cool idea to do configuration in a subset of Python but now you have to go implement that subset in every language.
seedless-sensat 5y ago
Plenty of tools lacking in the Starlark environment, e.g.: generating Starlark files, machine editting Starlark maps
xiaq 5y ago
> It’s a deterministic subset of Python. This means that if you have complex or repetitive configurations, you can use loops and functions to structure them. But it’s impossible to write an infinite loop or recursion. Starlark is indeed deterministic and guaranteed to terminate (the Go implementation has a flag that allows recursion, but it's off by default), but these are two orthogonal properties.
OskarS 5y ago
> Starlark is a dialect of Python. Like Python, it is a dynamically typed language with high-level data types, first-class functions with lexical scope, and garbage collection. If it has first-class functions, how can you avoid infinite recursion? Like, what stops me from running the omega combinator in it? This is why Meson (a similar language) does not allow those kinds of shenanigans, to keep the language non-Turing-complete.
- georgewfraser 5y ago
  No recursion and no lambda.
  - OskarS 5y ago
    So it doesn't have first class functions then?
slowmovintarget 5y ago
EDN [1] and Transit [2]... Elegant weapons for a more civilized system. [1] https://github.com/edn-format/edn https://github.com/edn-format/edn [2] https://github.com/cognitect/transit-format https://github.com/cognitect/transit-format
- kaliszad 5y ago
  Really came here to search why EDN wasn't mentioned. It is used in Clojure/ ClojureScript/ hylang ... projects a lot. It is a superset of JSON, is in my opinion a lot more readable than JSON but familiar enough too. It has native sets e.g. #{1 2 "three" '("four element list with a string inside")} and keywords. Tagged elements can be used for extending e.g. with a timestamp (such as the built-in #inst) or #uuid. And it also supports comments and discards for stuff, that should be omitted in evaluation. As a sysadmin, YAML seems nice until you have actually done anything more advanced with it. See Julien Pivotto's presentation about some of its pitfalls: https://www.slideshare.net/roidelapluie/yaml-magic?next_slideshow=1 https://www.slideshare.net/roidelapluie/yaml-magic?next_slid... Btw. Jsonnet doesn't seem too bad either: https://www.youtube.com/watch?v=LiQnSZ4SOnw https://www.youtube.com/watch?v=LiQnSZ4SOnw and here some examples: https://jsonnet.org/ https://jsonnet.org/ but in my book, EDN still wins.
im3w1l 5y ago
So one thing I wasnt sure of is: If you have a Starlark program how is the value of it decided? Is it simply the value of the last expression? And where does the print-output end up? Is it just for diagnostics and has no influence on the value?
leontrolski 5y ago
How about just nudge json a couple more notches towards js? https://github.com/leontrolski/dnjs https://github.com/leontrolski/dnjs
DonHopkins 5y ago
How about simply using pure full blown JavaScript or Python for config files, and not hiring people who you can't trust not to write infinite loops? Or if you really must, then simply interrupt processes that loop infinitely, and fix the bugs that caused it. You know, like you already do when you have an infinite loop. Infinite loops are not the end of the world, you know. Processes can be interrupted, and computers have reset buttons.
- 8192kjshad09- 5y ago
  Determing whether or not arbitrary code is looping is actually impossisble (halting problem).
- kortex 5y ago
  It's not just a trust thing. Knowing that some snipped has bounded evaluation is super important, for mental models, processin, security, etc. It's still resources to detect loops, it often involves introspection or privileged views; it's simply easier to prevent loops.
- repsilat 5y ago
  IMO using code that generates (possibly binary/opaque) config data is the sweet spot. It's one more layer of indirection, but it means you're language-agnostic, you have a "safe" interface, and your "config-generating" process can be as expressive as you like -- comments, loops, whatever. The underlying conundrum is: - systems need to be configured, - human-readability is obviously necessary at some level, - configuration is often very "compressible" (needs loops, needs variables to be maintainable), but - system-writers don't know the structure of your data, the axes on which you'd want to compress things, the best abstractions for you. Templating languages are an obvious direction, but they're uniformly bad. If they have limited expressiveness you'll run into the limits. Maybe there are templating languages with good unit testing frameworks, but I haven't seen them. "Look at the expanded diff" doesn't scale. And generating gobs of human-readable "data" (in a format that supports comments!) is very wasteful.

remram 5y ago

Interesting! I started using jsonnet this year, but found that the language was needlessly quirky (e.g. the `::`, purely functional aspect, and no one wants to learn a new language to write configuration in the first place). More importantly, it is extremely slow (lazy evaluation without memoization...): rendering the Kubernetes YAML of my 5-container app taking over 10 seconds... I will look into this further.

vlovich123 5y ago

Have you had any experience building on top of it directly outside of blaze/bazel?

dilyevsky 5y ago
I have - https://github.com/cruise-automation/isopod https://github.com/cruise-automation/isopod and more generally with https://github.com/stripe/skycfg https://github.com/stripe/skycfg

civilized 5y ago

Not a bad idea but only implemented in Rust, Go, and Java so far. Meanwhile, all sorts of languages can interpret JSON and YAML. It's a cool idea to do configuration in a subset of Python but now you have to go implement that subset in every language.

seedless-sensat 5y ago

Plenty of tools lacking in the Starlark environment, e.g.: generating Starlark files, machine editting Starlark maps

xiaq 5y ago

> It’s a deterministic subset of Python. This means that if you have complex or repetitive configurations, you can use loops and functions to structure them. But it’s impossible to write an infinite loop or recursion. Starlark is indeed deterministic and guaranteed to terminate (the Go implementation has a flag that allows recursion, but it's off by default), but these are two orthogonal properties.

OskarS 5y ago

> Starlark is a dialect of Python. Like Python, it is a dynamically typed language with high-level data types, first-class functions with lexical scope, and garbage collection. If it has first-class functions, how can you avoid infinite recursion? Like, what stops me from running the omega combinator in it? This is why Meson (a similar language) does not allow those kinds of shenanigans, to keep the language non-Turing-complete.

georgewfraser 5y ago
No recursion and no lambda.
- OskarS 5y ago
  So it doesn't have first class functions then?

slowmovintarget 5y ago

EDN [1] and Transit [2]... Elegant weapons for a more civilized system. [1] https://github.com/edn-format/edn https://github.com/edn-format/edn [2] https://github.com/cognitect/transit-format https://github.com/cognitect/transit-format

kaliszad 5y ago
Really came here to search why EDN wasn't mentioned. It is used in Clojure/ ClojureScript/ hylang ... projects a lot. It is a superset of JSON, is in my opinion a lot more readable than JSON but familiar enough too. It has native sets e.g. #{1 2 "three" '("four element list with a string inside")} and keywords. Tagged elements can be used for extending e.g. with a timestamp (such as the built-in #inst) or #uuid. And it also supports comments and discards for stuff, that should be omitted in evaluation. As a sysadmin, YAML seems nice until you have actually done anything more advanced with it. See Julien Pivotto's presentation about some of its pitfalls: https://www.slideshare.net/roidelapluie/yaml-magic?next_slideshow=1 https://www.slideshare.net/roidelapluie/yaml-magic?next_slid... Btw. Jsonnet doesn't seem too bad either: https://www.youtube.com/watch?v=LiQnSZ4SOnw https://www.youtube.com/watch?v=LiQnSZ4SOnw and here some examples: https://jsonnet.org/ https://jsonnet.org/ but in my book, EDN still wins.

im3w1l 5y ago

So one thing I wasnt sure of is: If you have a Starlark program how is the value of it decided? Is it simply the value of the last expression? And where does the print-output end up? Is it just for diagnostics and has no influence on the value?

leontrolski 5y ago

How about just nudge json a couple more notches towards js? https://github.com/leontrolski/dnjs https://github.com/leontrolski/dnjs

DonHopkins 5y ago

How about simply using pure full blown JavaScript or Python for config files, and not hiring people who you can't trust not to write infinite loops? Or if you really must, then simply interrupt processes that loop infinitely, and fix the bugs that caused it. You know, like you already do when you have an infinite loop. Infinite loops are not the end of the world, you know. Processes can be interrupted, and computers have reset buttons.

8192kjshad09- 5y ago
Determing whether or not arbitrary code is looping is actually impossisble (halting problem).
kortex 5y ago
It's not just a trust thing. Knowing that some snipped has bounded evaluation is super important, for mental models, processin, security, etc. It's still resources to detect loops, it often involves introspection or privileged views; it's simply easier to prevent loops.
repsilat 5y ago
IMO using code that generates (possibly binary/opaque) config data is the sweet spot. It's one more layer of indirection, but it means you're language-agnostic, you have a "safe" interface, and your "config-generating" process can be as expressive as you like -- comments, loops, whatever. The underlying conundrum is: - systems need to be configured, - human-readability is obviously necessary at some level, - configuration is often very "compressible" (needs loops, needs variables to be maintainable), but - system-writers don't know the structure of your data, the axes on which you'd want to compress things, the best abstractions for you. Templating languages are an obvious direction, but they're uniformly bad. If they have limited expressiveness you'll run into the limits. Maybe there are templating languages with good unit testing frameworks, but I haven't seen them. "Look at the expanded diff" doesn't scale. And generating gobs of human-readable "data" (in a format that supports comments!) is very wasteful.