8 ms·

Dhall – A Distributed, Safe Configuration Language

217 points by KirinDave 8y ago

KirinDave 8y ago
Dhall's really cool because you can use it even if your tooling doesn't support dhall. There is direct JSON/YAML support with dhall-json: https://github.com/dhall-lang/dhall-json https://github.com/dhall-lang/dhall-json If you wanted to generate something like an NGINX config (or, for example, TOML or Terraform) you could use dhall-text: https://github.com/dhall-lang/dhall-text https://github.com/dhall-lang/dhall-text Oh, did I say terraform? Someone's already on supporting that directly: https://github.com/blast-hardcheese/dhall-terraform https://github.com/blast-hardcheese/dhall-terraform Dhall is very unique among configuration because it's "distributed", meaning that Dhall can load any file as a dhall expression, and has builtin URI resolvers to make even remote endpoints part of local configuration. This is excellent for, say, CI integration. Dhall has "semantic hashing" so that if someone does change a dhall dependency in a surprising way, the dhall script will refuse to continue (and give a very clear error about what changed, where it was looking for the change, and why it's stopping). Dhall is a secret superpower for project configuration. Especially as of the 1.14+ releases, it's become an increasingly go-to tool for me. Even in just one-off JSON generation scripts, I find it to be a lifesaver.
- nv-vn 8y ago
  Dhall is really similar to Nix in that regard, though in practice I've never seen Nix used for real configs (sadly). Usually trying to do so in Nix ends up being really ugly because the Nix features don't translate directly to concepts in the config language.
  - chriswarbo 8y ago
    I dabbled in using Nix for this sort of thing, but it's rather painful to invoke the "Nix language" itself. In Nix 1.x we can evaluate a Nix expression (e.g. a templated string) like this: $ nix-instantiate --eval --read-write-mode -E 'Nix code goes here' That Nix code could be a file, or an import, or whatever. This is slightly improved with Nix 2.x: $ nix eval '(Nix code goes here)' Still, it's not particularly well suited to string processing from the commandline like this. We're closer to Nix's comfortable territory if we use it to build a config file, e.g. $ nix-build -E 'Nix code goes here' Or, more likely: $ nix-build myConfigFileDescription.nix In my experience this is more useful than trying to use Nix as a string processor. Still, if we're using Nix in a project or system, we might be better off using it to build the whole project (e.g. with a 'default.nix' file) or system (using a NixOS module or something). From what I've seen, Guix takes compatibility with non-Guix systems a little closer to heart, e.g. for generating standalone packages that don't require Guix to install/use.
- antpls 8y ago
  Sure, we can do all of that. But where is the cost analysis to justify adding more safety? I don't see how this couldn't be done whith a Python library. More people knows Python syntax, and more projects already have Python as a dependency. If that idea doesn't already exist in Python, is there even a need in the first place? Edit : downvoters are losers.
  - KirinDave 8y ago
    Python is a Turing complete language that can do anything. Dhall isn't. That lack of power means that a configuration language can still do a lot, but stick to its mission of configuration. It produces a final value (probably a map or list). Your real program reads that value and does the real work. Why complicate your real program with that? Offload it to dhall and you get a consistent configuration for free! Think of all those places (e.g., Chef and Circle) where people try to encode graphs and processes in YAML. Dhall could do much of the same work of producing a simple spec, but without the fear of infinite loops or nonsensical Bash parameters hitting a function. Dhall is such a simple language (really just function application and and some built-in datatypes and access principles) that it doesn't seem terribly costly as opposed to, say, embedding a complex graph into YAML (risking structural errors) or Python (risking extremely complex behavior and execute-only configuration). If what you really want is a declarative set of instructions to your real program, Dhall is probably one of the most awesome tools in the open source world for making it.
    - antpls 8y ago
      You sound like a marketer trying to sell a fridge on a phone call. > Why complicate your real program with that? Offload it to dhall and you get a consistent configuration for free! Why complicate my projects pipeline and force my team to learn yet another tool that may just be abandonned in 3 months? > Dhall could do much of the same work of producing a simple spec, but without the fear of infinite loops or nonsensical Bash parameters hitting a function. Last time I wrote an infinite loop in Python was... I can't remember. Maybe 5 years ago? Nonsensical is subjective. Maybe Dhall only makes sense to you. > Dhall is such a simple language There is no objective criteria to define a "simple" language. Perl was simple to some people. > If what you really want is a declarative set of instructions to your real program, Dhall is probably one of the most awesome tools in the open source world for making it. Or you could write a nice Python library, with clean API, and eventually a linter that makes sure the Python configuration file is a restricted subset of Python.
jdc0589 8y ago
is there a single example Dhall config file anywhere that has more than a couple feature examples? It's a lot easier to grok new stuff like this if there's just one big example to look at to get an idea of what it does rather than having to navigate tons of granular documentation (which is also great, though)
- KirinDave 8y ago
  Dhall has a cheatsheet that might help you get a handle on its full feature set: https://github.com/dhall-lang/dhall-lang/wiki/Cheatsheet https://github.com/dhall-lang/dhall-lang/wiki/Cheatsheet
  - stingraycharles 8y ago
    Hmm I agree with OP though, and this cheatsheet still isn’t it for me. I need to see a real world example to be able to fully grok its practical use. Do you happen to know of any?
    - vmchale 8y ago
      Here's an example from one of my projects: https://github.com/vmchale/polyglot/blob/master/atspkg.dhall https://github.com/vmchale/polyglot/blob/master/atspkg.dhall
- Gabriel439 8y ago
  Author here: I opened an issue to track this request and remind myself to do this: https://github.com/dhall-lang/dhall-lang/issues/189 https://github.com/dhall-lang/dhall-lang/issues/189
- dwohnitmok 8y ago
  Since the author's already responded, this might be redundant, but perhaps expanding the example at https://github.com/dhall-lang/dhall-lang#case-study https://github.com/dhall-lang/dhall-lang#case-study might be a helpful look at a running example of Dhall. I feel like I've seen multiple people completely skip over and not realize there was already an example in the README as opposed to just the detailed reference docs. Perhaps you're looking for something with even more features and applicability (e.g. an example of a Kubernetes config as Dhall's author points out), but hopefully that example gives at least a self-contained example of how to gradually start using Dhall.
ff_ 8y ago
Dhall was such a godsend to make our infra configs more safe & modular (we jointly configure Kubernetes & Terraform with it). If you need some example of use cases to see if it might help you, check out "Dhall in Production" [0] So far I'm a maintainer of dhall-kubernetes [1] and we'll soon open source some of our integration with Terraform. And because it's absolutely safe to distribute Dhall code I'm toying with the idea of making some kind of Dhall-Kafka pubsub, in which you can safely distribute Dhall code and data, with automatic version migrations (check this out for more info on why this is possible [2]) [0]: https://github.com/dhall-lang/dhall-lang/wiki/Dhall-in-production https://github.com/dhall-lang/dhall-lang/wiki/Dhall-in-produ... [1]: https://github.com/dhall-lang/dhall-kubernetes https://github.com/dhall-lang/dhall-kubernetes [2]: http://www.haskellforall.com/2017/11/semantic-integrity-checks-are-next.html?showComment=1511801973283#c6412191866661551590 http://www.haskellforall.com/2017/11/semantic-integrity-chec...
- deleted 8y ago
  [deleted]
- lomnakkus 8y ago
  This is great. I've been toying with the idea of using Dhall for various things and your post might just have given me the impetus to go actually do it!
- openasocket 8y ago
  RE: Dhall-Kafka pubsub Dhall seems to be safe in the sense that it will always terminate and will never crash or throw some kind of exception, but I don't think it's safe in the sense that it is safe to execute potentially malicious Dhall code, unless you restrict the allowed imports to a whitelist. At the very least that could be used to DDos some target by having the script try to import something from a victim domain. And you might be able to read data on local files and transmit that information back, I'm not sure. It depends on if imports are evaluated lazily, and the data of interest would have to be stored in a file on disk that can be imported. EDIT: Actually, it looks like you can import raw text, so it doesn't matter what format the on-disk data is that you are trying to extract. EDIT2: Actually, it doesn't even matter if imports are evaluated lazily or not, you can specify that a network import be made with given headers, so you could just set a header in the HTTP request to contain the sensitive data.
slyrus 8y ago
Remind me again... what's wrong with common-lisp:read?
- KirinDave 8y ago
  Dhall doesn't actually have any of those issues, though.
- js8 8y ago
  I interpret it as "What advantage Dhall has compare to CL:read?" So for example, in your configuration, you can define a common expression into a variable (or even parametrize it in a function). CL:read cannot do that without resorting to read macros (technically it doesn't have to be read macros, you can reinterpret it later, but then we are getting outside the realm of CL:read), which are Turing complete and can have bugs that can lead to security exploits. Dhall guarantees that the functions you define in your configuration cannot be exploited. This has interesting implications, for example, you can read configuration safely from semi-trusted source. With CL:read, you can do that only if you give up read macros, but then the semi-trusted source cannot define their own function. So where with CL:read you have to choose between little and total power, Dhall gives you a little bit of both, a medium power of sorts.
  - slyrus 8y ago
    Thanks for the explanation!
- rurban 8y ago
  Too much power. Think of recursion, access to the full CL stdlib, destructive list operations, ... Similar to YAML. Too much power, recursive via links, too insecure.
Cieplak 8y ago
Gabriel gave a great presentation on Dhall at Bayhac this year: https://www.youtube.com/watch?v=Ih9Ngu7FlCc https://www.youtube.com/watch?v=Ih9Ngu7FlCc
jarpineh 8y ago
This discussion here and Dhall's Github Readme seemed promising, so I went through the tutorial. I've recently had to write a lot of YAML like language with custom templating (think Ansible), so naturally I felt there was a way out with Dhall. I was hoping the author could perhaps point me towards explanations to what I noticed bothering me by this (admittedly short) experience that left me puzzled. I saw that lists can have only one type of elements without annotating their types. With type annotation list can have differently typed values, but only if each element is explicitly stated (note: this is my understanding that might not be correct) meaning there's no dynamic content in a list. In same vein a map of maps needs to have all its keys stated by type annotation. To me this seems too restrictive since the structure of data gets lost in the more verbose annotations. Not to mention the work of writing this annotation or the functionality to produce the same. In TypeScript I'd write something like this { [string] : [ Number | String ] } and I'd have my string keyed object with values of lists containing numbers and strings. Having a language like Dhall to help with creation of correct configuration code seems really useful instead of this messy combination of declarative and template language. I would like to understand things that can get better by using such an type system.
- Gabriel439 8y ago
  Dhall's lists are homogeneous lists, meaning that every element always has the same type of value. This is true whether or not you annotate list elements with a type or you annotate the list with a type. You only need to annotate the type of an empty list. Lists with at least one element don't require a type annotation because the type can be inferred from the type of that element. Dhall does not have buit-in support for homogeneous maps. Dhall does have statically typed heterogeneous records (i.e. something like `{ foo = Bool, bar = "ABC" }` which has type `{ foo : Bool, bar : Text }` for example). If you want to store different type of values in the same list you wrap them in a union. For example, if you want to store both `Text` values and `Natural` numbers in a list you would do: let union = constructors < Left : Natural | Right : Text > in [ union.Left 10, union.Right "ABC", union.Right "DEF", union.Left 4 ] The closest thing to a homogeneous map in Dhall is an association list of type `[ { mapKey : Text, mapValue : a } ]` but even that is still not an exact fit since it doesn't guarantee uniqueness of keys. However, Dhall's JSON/YAML integration does convert that automatically to a JSON/YAML homogeneous map (i.e. a JSON record where every field has the same type). In general, Dhall's JSON/YAML integration has several tricks and conventions that translate to weakly typed JSON idioms (such as homogeneous maps, omitting null values, and using tags).
totalperspectiv 8y ago
Is there a tutorial that isn't predicated on me already knowing haskell? I feel like Dhall is exactly the tool I've been hoping for. It feels like a non-hacky version of make + m4.
- totalperspectiv 8y ago
  Answering my own question, the README says that eventually the tutorial will be language agnostic.
- KirinDave 8y ago
  If you checkout dhall-text and dhall-json, you can see you can use it without any integration into your executable. I do this via shell scripts.
lifeisstillgood 8y ago
Look, I'm sorry but this is beyond 99% of the world's IT services right now. This is of course biased, unresearched anecdata but roughly speaking - 50 % of all IT installations cannot be rebuilt from scratch in an automated fashion if you have them the new hardware,plugged in. A further ten percent could be rebuilt with mostly automated scripts and a wiki page that's out of date by two months of the remaining 1/3 of the world's IT, 1/6 has dependencies it does not know about - the DNS server that "just works",the production routers that should not serve that subnet, the database that is "owned" by a different team whose configuration you have no control over. Good guy d.b. has its secret passwords on the wiki page, or stored on a seperate importable module, in source control, base64-encoded And the rest - the rest rely more on bash than ansible. Just get the world's enterprise services over the line and into "run this python script with these parma's and you get a complete rebuild across databases and routers" and then we can talk.
- markbnj 8y ago
  I don't know about the numbers but you're probably speaking something close to truth here. So I'd guess the reason for the downvotes is the tone of your implication that because of this the preceding doesn't matter and is... wasting our time or something? I think it does matter. I work on a fairly small team that has to manage a lot of stuff, and we probably fall somewhere in the middle of your rough ranking. The wiki page might even be older than 2 months :). We're always looking for smarter ways to automate things (and then hoping to get the time to work on them). So to people like us things like this do matter quite a lot.
- klibertp 8y ago
  That's completely beside the point. Are we supposed to shed a tear for each of that 50 %? I can do that, but it won't help me, at all. On the other hand, I can use Dhall to generate Kubernetes/Helm configs, which will be more DRY, less duplicated, and not a pain to work with. I know that this would help me quite a bit, not only because I hate my current "DevOps" team members with burning passion and I'd enjoy watching them try to figure out what is happening (well, I'd at least add the source files to a repo, that's already more than they do).
hiccuphippo 8y ago
I don't understand what the word "distributed" means here. Is it that it can load the configuration from multiple servers?
- klibertp 8y ago
  Apparently any, or almost any(?), element of a language can be replaced with a file path, which is then transparently read and used as if typed directly in that place. It works even for type annotations and is type-safe. This allows for easy factoring of the code into many files, and - by extension - to files on remote hosts as long as they're accessible via a (built-in) HTTP support.
shoo 8y ago
if there's anyone here who has used both dhall and jsonnet in anger, can you comment on your experiences?
ofrzeta 8y ago
What about those special characters like the lambda or the universal quantification? How to you type them?
- Gabriel439 8y ago
  There are ASCII equivalents. You can type `\` instead of `λ` and `forall` instead of `∀`. Also `dhall format` will automatically translate ASCII to Unicode for you. However, if you do want to type them then see: https://en.wikipedia.org/wiki/Unicode_input https://en.wikipedia.org/wiki/Unicode_input ... and the relevant code points are: * `λ`: `U+03BB` * `∀`: `U+2200`
nojvek 8y ago
I tried to read Dhall manual and figure out if I could make sense of it. I couldn’t. Looks a bit too complicated for me. Jsonnet strikes a great balance of json but with some nice template syntax so you can be DRY. Also what’s wrong with being Turing complete. I love for loops.

217 points by KirinDave 8y ago

Dhall's really cool because you can use it even if your tooling doesn't support dhall. There is direct JSON/YAML support with dhall-json: https://github.com/dhall-lang/dhall-json https://github.com/dhall-lang/dhall-json If you wanted to generate something like an NGINX config (or, for example, TOML or Terraform) you could use dhall-text: https://github.com/dhall-lang/dhall-text https://github.com/dhall-lang/dhall-text Oh, did I say terraform? Someone's already on supporting that directly: https://github.com/blast-hardcheese/dhall-terraform https://github.com/blast-hardcheese/dhall-terraform Dhall is very unique among configuration because it's "distributed", meaning that Dhall can load any file as a dhall expression, and has builtin URI resolvers to make even remote endpoints part of local configuration. This is excellent for, say, CI integration. Dhall has "semantic hashing" so that if someone does change a dhall dependency in a surprising way, the dhall script will refuse to continue (and give a very clear error about what changed, where it was looking for the change, and why it's stopping). Dhall is a secret superpower for project configuration. Especially as of the 1.14+ releases, it's become an increasingly go-to tool for me. Even in just one-off JSON generation scripts, I find it to be a lifesaver.

nv-vn 8y ago
Dhall is really similar to Nix in that regard, though in practice I've never seen Nix used for real configs (sadly). Usually trying to do so in Nix ends up being really ugly because the Nix features don't translate directly to concepts in the config language.
- chriswarbo 8y ago
  I dabbled in using Nix for this sort of thing, but it's rather painful to invoke the "Nix language" itself. In Nix 1.x we can evaluate a Nix expression (e.g. a templated string) like this: $ nix-instantiate --eval --read-write-mode -E 'Nix code goes here' That Nix code could be a file, or an import, or whatever. This is slightly improved with Nix 2.x: $ nix eval '(Nix code goes here)' Still, it's not particularly well suited to string processing from the commandline like this. We're closer to Nix's comfortable territory if we use it to build a config file, e.g. $ nix-build -E 'Nix code goes here' Or, more likely: $ nix-build myConfigFileDescription.nix In my experience this is more useful than trying to use Nix as a string processor. Still, if we're using Nix in a project or system, we might be better off using it to build the whole project (e.g. with a 'default.nix' file) or system (using a NixOS module or something). From what I've seen, Guix takes compatibility with non-Guix systems a little closer to heart, e.g. for generating standalone packages that don't require Guix to install/use.
antpls 8y ago
Sure, we can do all of that. But where is the cost analysis to justify adding more safety? I don't see how this couldn't be done whith a Python library. More people knows Python syntax, and more projects already have Python as a dependency. If that idea doesn't already exist in Python, is there even a need in the first place? Edit : downvoters are losers.
- KirinDave 8y ago
  Python is a Turing complete language that can do anything. Dhall isn't. That lack of power means that a configuration language can still do a lot, but stick to its mission of configuration. It produces a final value (probably a map or list). Your real program reads that value and does the real work. Why complicate your real program with that? Offload it to dhall and you get a consistent configuration for free! Think of all those places (e.g., Chef and Circle) where people try to encode graphs and processes in YAML. Dhall could do much of the same work of producing a simple spec, but without the fear of infinite loops or nonsensical Bash parameters hitting a function. Dhall is such a simple language (really just function application and and some built-in datatypes and access principles) that it doesn't seem terribly costly as opposed to, say, embedding a complex graph into YAML (risking structural errors) or Python (risking extremely complex behavior and execute-only configuration). If what you really want is a declarative set of instructions to your real program, Dhall is probably one of the most awesome tools in the open source world for making it.
  - antpls 8y ago
    You sound like a marketer trying to sell a fridge on a phone call. > Why complicate your real program with that? Offload it to dhall and you get a consistent configuration for free! Why complicate my projects pipeline and force my team to learn yet another tool that may just be abandonned in 3 months? > Dhall could do much of the same work of producing a simple spec, but without the fear of infinite loops or nonsensical Bash parameters hitting a function. Last time I wrote an infinite loop in Python was... I can't remember. Maybe 5 years ago? Nonsensical is subjective. Maybe Dhall only makes sense to you. > Dhall is such a simple language There is no objective criteria to define a "simple" language. Perl was simple to some people. > If what you really want is a declarative set of instructions to your real program, Dhall is probably one of the most awesome tools in the open source world for making it. Or you could write a nice Python library, with clean API, and eventually a linter that makes sure the Python configuration file is a restricted subset of Python.

jdc0589 8y ago

is there a single example Dhall config file anywhere that has more than a couple feature examples? It's a lot easier to grok new stuff like this if there's just one big example to look at to get an idea of what it does rather than having to navigate tons of granular documentation (which is also great, though)

KirinDave 8y ago
Dhall has a cheatsheet that might help you get a handle on its full feature set: https://github.com/dhall-lang/dhall-lang/wiki/Cheatsheet https://github.com/dhall-lang/dhall-lang/wiki/Cheatsheet
- stingraycharles 8y ago
  Hmm I agree with OP though, and this cheatsheet still isn’t it for me. I need to see a real world example to be able to fully grok its practical use. Do you happen to know of any?
  - vmchale 8y ago
    Here's an example from one of my projects: https://github.com/vmchale/polyglot/blob/master/atspkg.dhall https://github.com/vmchale/polyglot/blob/master/atspkg.dhall
Gabriel439 8y ago
Author here: I opened an issue to track this request and remind myself to do this: https://github.com/dhall-lang/dhall-lang/issues/189 https://github.com/dhall-lang/dhall-lang/issues/189
dwohnitmok 8y ago
Since the author's already responded, this might be redundant, but perhaps expanding the example at https://github.com/dhall-lang/dhall-lang#case-study https://github.com/dhall-lang/dhall-lang#case-study might be a helpful look at a running example of Dhall. I feel like I've seen multiple people completely skip over and not realize there was already an example in the README as opposed to just the detailed reference docs. Perhaps you're looking for something with even more features and applicability (e.g. an example of a Kubernetes config as Dhall's author points out), but hopefully that example gives at least a self-contained example of how to gradually start using Dhall.

ff_ 8y ago

Dhall was such a godsend to make our infra configs more safe & modular (we jointly configure Kubernetes & Terraform with it). If you need some example of use cases to see if it might help you, check out "Dhall in Production" [0] So far I'm a maintainer of dhall-kubernetes [1] and we'll soon open source some of our integration with Terraform. And because it's absolutely safe to distribute Dhall code I'm toying with the idea of making some kind of Dhall-Kafka pubsub, in which you can safely distribute Dhall code and data, with automatic version migrations (check this out for more info on why this is possible [2]) [0]: https://github.com/dhall-lang/dhall-lang/wiki/Dhall-in-production https://github.com/dhall-lang/dhall-lang/wiki/Dhall-in-produ... [1]: https://github.com/dhall-lang/dhall-kubernetes https://github.com/dhall-lang/dhall-kubernetes [2]: http://www.haskellforall.com/2017/11/semantic-integrity-checks-are-next.html?showComment=1511801973283#c6412191866661551590 http://www.haskellforall.com/2017/11/semantic-integrity-chec...

deleted 8y ago
[deleted]
lomnakkus 8y ago
This is great. I've been toying with the idea of using Dhall for various things and your post might just have given me the impetus to go actually do it!
openasocket 8y ago
RE: Dhall-Kafka pubsub Dhall seems to be safe in the sense that it will always terminate and will never crash or throw some kind of exception, but I don't think it's safe in the sense that it is safe to execute potentially malicious Dhall code, unless you restrict the allowed imports to a whitelist. At the very least that could be used to DDos some target by having the script try to import something from a victim domain. And you might be able to read data on local files and transmit that information back, I'm not sure. It depends on if imports are evaluated lazily, and the data of interest would have to be stored in a file on disk that can be imported. EDIT: Actually, it looks like you can import raw text, so it doesn't matter what format the on-disk data is that you are trying to extract. EDIT2: Actually, it doesn't even matter if imports are evaluated lazily or not, you can specify that a network import be made with given headers, so you could just set a header in the HTTP request to contain the sensitive data.

slyrus 8y ago

Remind me again... what's wrong with common-lisp:read?

KirinDave 8y ago
Dhall doesn't actually have any of those issues, though.
js8 8y ago
I interpret it as "What advantage Dhall has compare to CL:read?" So for example, in your configuration, you can define a common expression into a variable (or even parametrize it in a function). CL:read cannot do that without resorting to read macros (technically it doesn't have to be read macros, you can reinterpret it later, but then we are getting outside the realm of CL:read), which are Turing complete and can have bugs that can lead to security exploits. Dhall guarantees that the functions you define in your configuration cannot be exploited. This has interesting implications, for example, you can read configuration safely from semi-trusted source. With CL:read, you can do that only if you give up read macros, but then the semi-trusted source cannot define their own function. So where with CL:read you have to choose between little and total power, Dhall gives you a little bit of both, a medium power of sorts.
- slyrus 8y ago
  Thanks for the explanation!
rurban 8y ago
Too much power. Think of recursion, access to the full CL stdlib, destructive list operations, ... Similar to YAML. Too much power, recursive via links, too insecure.

Cieplak 8y ago

Gabriel gave a great presentation on Dhall at Bayhac this year: https://www.youtube.com/watch?v=Ih9Ngu7FlCc https://www.youtube.com/watch?v=Ih9Ngu7FlCc

jarpineh 8y ago

This discussion here and Dhall's Github Readme seemed promising, so I went through the tutorial. I've recently had to write a lot of YAML like language with custom templating (think Ansible), so naturally I felt there was a way out with Dhall. I was hoping the author could perhaps point me towards explanations to what I noticed bothering me by this (admittedly short) experience that left me puzzled. I saw that lists can have only one type of elements without annotating their types. With type annotation list can have differently typed values, but only if each element is explicitly stated (note: this is my understanding that might not be correct) meaning there's no dynamic content in a list. In same vein a map of maps needs to have all its keys stated by type annotation. To me this seems too restrictive since the structure of data gets lost in the more verbose annotations. Not to mention the work of writing this annotation or the functionality to produce the same. In TypeScript I'd write something like this { [string] : [ Number | String ] } and I'd have my string keyed object with values of lists containing numbers and strings. Having a language like Dhall to help with creation of correct configuration code seems really useful instead of this messy combination of declarative and template language. I would like to understand things that can get better by using such an type system.

Gabriel439 8y ago
Dhall's lists are homogeneous lists, meaning that every element always has the same type of value. This is true whether or not you annotate list elements with a type or you annotate the list with a type. You only need to annotate the type of an empty list. Lists with at least one element don't require a type annotation because the type can be inferred from the type of that element. Dhall does not have buit-in support for homogeneous maps. Dhall does have statically typed heterogeneous records (i.e. something like `{ foo = Bool, bar = "ABC" }` which has type `{ foo : Bool, bar : Text }` for example). If you want to store different type of values in the same list you wrap them in a union. For example, if you want to store both `Text` values and `Natural` numbers in a list you would do: let union = constructors < Left : Natural | Right : Text > in [ union.Left 10, union.Right "ABC", union.Right "DEF", union.Left 4 ] The closest thing to a homogeneous map in Dhall is an association list of type `[ { mapKey : Text, mapValue : a } ]` but even that is still not an exact fit since it doesn't guarantee uniqueness of keys. However, Dhall's JSON/YAML integration does convert that automatically to a JSON/YAML homogeneous map (i.e. a JSON record where every field has the same type). In general, Dhall's JSON/YAML integration has several tricks and conventions that translate to weakly typed JSON idioms (such as homogeneous maps, omitting null values, and using tags).

totalperspectiv 8y ago

Is there a tutorial that isn't predicated on me already knowing haskell? I feel like Dhall is exactly the tool I've been hoping for. It feels like a non-hacky version of make + m4.

totalperspectiv 8y ago
Answering my own question, the README says that eventually the tutorial will be language agnostic.
KirinDave 8y ago
If you checkout dhall-text and dhall-json, you can see you can use it without any integration into your executable. I do this via shell scripts.

lifeisstillgood 8y ago

Look, I'm sorry but this is beyond 99% of the world's IT services right now. This is of course biased, unresearched anecdata but roughly speaking - 50 % of all IT installations cannot be rebuilt from scratch in an automated fashion if you have them the new hardware,plugged in. A further ten percent could be rebuilt with mostly automated scripts and a wiki page that's out of date by two months of the remaining 1/3 of the world's IT, 1/6 has dependencies it does not know about - the DNS server that "just works",the production routers that should not serve that subnet, the database that is "owned" by a different team whose configuration you have no control over. Good guy d.b. has its secret passwords on the wiki page, or stored on a seperate importable module, in source control, base64-encoded And the rest - the rest rely more on bash than ansible. Just get the world's enterprise services over the line and into "run this python script with these parma's and you get a complete rebuild across databases and routers" and then we can talk.

markbnj 8y ago
I don't know about the numbers but you're probably speaking something close to truth here. So I'd guess the reason for the downvotes is the tone of your implication that because of this the preceding doesn't matter and is... wasting our time or something? I think it does matter. I work on a fairly small team that has to manage a lot of stuff, and we probably fall somewhere in the middle of your rough ranking. The wiki page might even be older than 2 months :). We're always looking for smarter ways to automate things (and then hoping to get the time to work on them). So to people like us things like this do matter quite a lot.
klibertp 8y ago
That's completely beside the point. Are we supposed to shed a tear for each of that 50 %? I can do that, but it won't help me, at all. On the other hand, I can use Dhall to generate Kubernetes/Helm configs, which will be more DRY, less duplicated, and not a pain to work with. I know that this would help me quite a bit, not only because I hate my current "DevOps" team members with burning passion and I'd enjoy watching them try to figure out what is happening (well, I'd at least add the source files to a repo, that's already more than they do).

hiccuphippo 8y ago

I don't understand what the word "distributed" means here. Is it that it can load the configuration from multiple servers?

klibertp 8y ago
Apparently any, or almost any(?), element of a language can be replaced with a file path, which is then transparently read and used as if typed directly in that place. It works even for type annotations and is type-safe. This allows for easy factoring of the code into many files, and - by extension - to files on remote hosts as long as they're accessible via a (built-in) HTTP support.

shoo 8y ago

if there's anyone here who has used both dhall and jsonnet in anger, can you comment on your experiences?

ofrzeta 8y ago

What about those special characters like the lambda or the universal quantification? How to you type them?

Gabriel439 8y ago
There are ASCII equivalents. You can type `\` instead of `λ` and `forall` instead of `∀`. Also `dhall format` will automatically translate ASCII to Unicode for you. However, if you do want to type them then see: https://en.wikipedia.org/wiki/Unicode_input https://en.wikipedia.org/wiki/Unicode_input ... and the relevant code points are: * `λ`: `U+03BB` * `∀`: `U+2200`

nojvek 8y ago

I tried to read Dhall manual and figure out if I could make sense of it. I couldn’t. Looks a bit too complicated for me. Jsonnet strikes a great balance of json but with some nice template syntax so you can be DRY. Also what’s wrong with being Turing complete. I love for loops.