6 ms·

This is a great analysis, but it's missing a fundamental point: why do we have a problem with these approximations of a programming language or just using a pro

by ff_ 8y ago

This is a great analysis, but it's missing a fundamental point: why do we have a problem with these approximations of a programming language or just using a programming language to template stuff? Because your build then becomes an actual program (i.e. Turing complete) and you have to refactor and maintain it! This is the common problem of using a "programming language as configuration" (e.g. gulp?) Dhall solves exactly this problem: https://dhall-lang.org https://dhall-lang.org It has the same premises of Pulumi, but without the Turing completeness (I don't know if/how Pulumi avoids that, but if it does it should be part of the pitch), so you cannot shoot yourself in the foot by building an abstraction castle in your build system/infrastructure config. We use it at work to generate all the Infra-as-Code configurations from a single Dhall config: Terraform, Kubernetes, SQL, etc. And there is already an integration with Kubernetes: https://github.com/dhall-lang/dhall-kubernetes https://github.com/dhall-lang/dhall-kubernetes

solatic 8y ago
> We use it at work to generate all the Infra-as-Code configurations from a single Dhall config This is the key bit and not something which is pitched well enough from the Dhall landing pages: using straight YAML forces you to repeat yourself in multiple areas for each Individual tool being used, and these repetitions have to stay consistent across multiple tools. What Dhall does is allow you to write a single config and use it to derive the correct configurations for each tool that you use. So you can write a single configuration file from which, eventually, every single part of your system is derived - Terraform infrastructure, Kubernetes objects, application config, everything. When you pull it off, it's simply magical. You can think of it like this: JavaScript is a horrible, no-good, very bad language, and yet all browser programming is done in JavaScript because every browser supports it - so too, are JSON and YAML horrible configuration languages. But JavaScript gave rise to abstractions like TypeScript which are much better languages which compile down to JavaScript for compatibility. TypeScript is to JavaScript what Dhall is to JSON and YAML - the fact is, pretty much everything is configured with JSON and YAML, and Dhall makes it much, much easier to live in that world, with no need for the systems being configured to support it. Considering the relative obscurity of Dhall, it's basically the best-kept secret in the DevOps world right now, and it's a shame more people don't know about it.
justincormack 8y ago
This is how Lua started, as a config language, but it gradually added more features that people found useful in config, and became Turing complete.
- icebraining 8y ago
  Lua was TC from the start, it came with the procedural concepts from Modula - if/while/repeat - and functions.
  - justincormack 8y ago
    I meant SOL, the predecessor, before Lua proper.
leg100 8y ago
I don't get the problem with using a turing complete language to generate configuration. There's nothing wrong with maintaining and refactoring a program, that's a natural process for any program. If you don't want an infinite loop, don't write one, as you wouldn't in any other program. You can choose as much or as little abstraction as you so wish. Give me a real language any day over dhall or jsonnet.
- ithkuil 8y ago
  FWIW jsonnet is a "real" language. It's a dynamically typed, lazily evaluated purely functional programming language).
  - leg100 8y ago
    Fair enough. I should have said "general purpose language" rather than "real", which makes for flame-bait.
    - ithkuil 8y ago
      I once built a mandelbrot fractal renderer which emitted a data-URL encoded PNG string to stdout in BCL (a spiritual predecessor of Jsonnet @ Google). Yeah, I know what you mean. It lacks generic input/output, you cannot read write arbitrary files and perform arbitrary network requests etc. I do like that restriction in the context of managing configuration systems, because it allows you to build hermetic evaluations. With kubecfg we added the ability to import from URLs, which I wish was available out of the box in jsonnet.
- Gabriel439 8y ago
  This explains the disadvantages of using a general-purpose programming language as a configuration language: https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarantees#effects https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarant...
metafunctor 8y ago
Dhall appears to be expressive enough that I can't see why you wouldn't have to refactor and maintain the Dhall code? Writing Dhall code look exactly like programming to me, and the programmer must possess the necessary programming skills to produce good Dhall code. A random guy with a text editor will make an equal mess in Dhall as they would with a “real” programming language. I don't see how the restrictions in Dhall really help much in this regard. Turing completeness feels like a red herring to me.
- js8 8y ago
  Not a user of Dhall, just a fan, but refactoring of Dhall configuration should be extremely easy. You make a change, and your configuration stays the same, which is easy to verify. (Thanks to https://en.wikipedia.org/wiki/Normalization_property_(abstract_rewriting) https://en.wikipedia.org/wiki/Normalization_property_(abstra... ) For TC languages, comparing if two programs (original and refactored) do the same thing is not solvable in general. If the language is not TC then it is more feasible.
  - gdfasfklshg4 8y ago
    This sounds like a render test?
  - metafunctor 8y ago
    You can compare the outputs of two programs. Sure, a TC program may not finish to produce output you can compare, but in my experience that's only a theoretical problem.
    - js8 8y ago
      Actually with Dhall, you should be able to compare the programs themselves, even without full "input" (there is even example on the Dhall page, see "You can reduce functions to normal form, even when they haven't been applied to all of their arguments"). So you can for example leave some parameters out of your config and still validate the correctness of refactoring. If you use general purpose programming language, then even comparing just output might be difficult - most languages allow to do I/O, so it's possible that the configuration is dependent on some side channel. I would say if you are only using general language "sensibly" for configuration then you are effectively restricting yourself in the same way that Dhall does.
TheCoelacanth 8y ago
> you have to refactor and maintain it You already have to do that, so why not do it in a reasonably powerful language?
- ff_ 8y ago
  Here's a nice explanation on why using "reasonably powerful languages" has many disadvantages: https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarantees#effects https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarant... Also you might be familiar with the Rule of Least Power: https://en.wikipedia.org/wiki/Rule_of_least_power https://en.wikipedia.org/wiki/Rule_of_least_power
willtim 8y ago
What's so bad about Turing Completeness? I haven't a decent look at Dhall, but I'm betting I could probably write an exponential Dhall program that won't terminate in the lifetime of the universe. The real reason for giving up Turing equivalence was probably to get dependent types. This gives very powerful static guarantees, including the presence/absence of fields under non-trivial record operations such as merge. In using dependent types, they have also had to give up significantly on type inference, which is really going to annoy the average JavaScript/Ruby programmer.

by ff_ 8y ago

solatic 8y ago

> We use it at work to generate all the Infra-as-Code configurations from a single Dhall config This is the key bit and not something which is pitched well enough from the Dhall landing pages: using straight YAML forces you to repeat yourself in multiple areas for each Individual tool being used, and these repetitions have to stay consistent across multiple tools. What Dhall does is allow you to write a single config and use it to derive the correct configurations for each tool that you use. So you can write a single configuration file from which, eventually, every single part of your system is derived - Terraform infrastructure, Kubernetes objects, application config, everything. When you pull it off, it's simply magical. You can think of it like this: JavaScript is a horrible, no-good, very bad language, and yet all browser programming is done in JavaScript because every browser supports it - so too, are JSON and YAML horrible configuration languages. But JavaScript gave rise to abstractions like TypeScript which are much better languages which compile down to JavaScript for compatibility. TypeScript is to JavaScript what Dhall is to JSON and YAML - the fact is, pretty much everything is configured with JSON and YAML, and Dhall makes it much, much easier to live in that world, with no need for the systems being configured to support it. Considering the relative obscurity of Dhall, it's basically the best-kept secret in the DevOps world right now, and it's a shame more people don't know about it.

justincormack 8y ago

This is how Lua started, as a config language, but it gradually added more features that people found useful in config, and became Turing complete.

icebraining 8y ago
Lua was TC from the start, it came with the procedural concepts from Modula - if/while/repeat - and functions.
- justincormack 8y ago
  I meant SOL, the predecessor, before Lua proper.

leg100 8y ago

I don't get the problem with using a turing complete language to generate configuration. There's nothing wrong with maintaining and refactoring a program, that's a natural process for any program. If you don't want an infinite loop, don't write one, as you wouldn't in any other program. You can choose as much or as little abstraction as you so wish. Give me a real language any day over dhall or jsonnet.

ithkuil 8y ago
FWIW jsonnet is a "real" language. It's a dynamically typed, lazily evaluated purely functional programming language).
- leg100 8y ago
  Fair enough. I should have said "general purpose language" rather than "real", which makes for flame-bait.
  - ithkuil 8y ago
    I once built a mandelbrot fractal renderer which emitted a data-URL encoded PNG string to stdout in BCL (a spiritual predecessor of Jsonnet @ Google). Yeah, I know what you mean. It lacks generic input/output, you cannot read write arbitrary files and perform arbitrary network requests etc. I do like that restriction in the context of managing configuration systems, because it allows you to build hermetic evaluations. With kubecfg we added the ability to import from URLs, which I wish was available out of the box in jsonnet.
Gabriel439 8y ago
This explains the disadvantages of using a general-purpose programming language as a configuration language: https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarantees#effects https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarant...

metafunctor 8y ago

Dhall appears to be expressive enough that I can't see why you wouldn't have to refactor and maintain the Dhall code? Writing Dhall code look exactly like programming to me, and the programmer must possess the necessary programming skills to produce good Dhall code. A random guy with a text editor will make an equal mess in Dhall as they would with a “real” programming language. I don't see how the restrictions in Dhall really help much in this regard. Turing completeness feels like a red herring to me.

js8 8y ago
Not a user of Dhall, just a fan, but refactoring of Dhall configuration should be extremely easy. You make a change, and your configuration stays the same, which is easy to verify. (Thanks to https://en.wikipedia.org/wiki/Normalization_property_(abstract_rewriting) https://en.wikipedia.org/wiki/Normalization_property_(abstra... ) For TC languages, comparing if two programs (original and refactored) do the same thing is not solvable in general. If the language is not TC then it is more feasible.
- gdfasfklshg4 8y ago
  This sounds like a render test?
- metafunctor 8y ago
  You can compare the outputs of two programs. Sure, a TC program may not finish to produce output you can compare, but in my experience that's only a theoretical problem.
  - js8 8y ago
    Actually with Dhall, you should be able to compare the programs themselves, even without full "input" (there is even example on the Dhall page, see "You can reduce functions to normal form, even when they haven't been applied to all of their arguments"). So you can for example leave some parameters out of your config and still validate the correctness of refactoring. If you use general purpose programming language, then even comparing just output might be difficult - most languages allow to do I/O, so it's possible that the configuration is dependent on some side channel. I would say if you are only using general language "sensibly" for configuration then you are effectively restricting yourself in the same way that Dhall does.

TheCoelacanth 8y ago

> you have to refactor and maintain it You already have to do that, so why not do it in a reasonably powerful language?

ff_ 8y ago
Here's a nice explanation on why using "reasonably powerful languages" has many disadvantages: https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarantees#effects https://github.com/dhall-lang/dhall-lang/wiki/Safety-guarant... Also you might be familiar with the Rule of Least Power: https://en.wikipedia.org/wiki/Rule_of_least_power https://en.wikipedia.org/wiki/Rule_of_least_power

willtim 8y ago

What's so bad about Turing Completeness? I haven't a decent look at Dhall, but I'm betting I could probably write an exponential Dhall program that won't terminate in the lifetime of the universe. The real reason for giving up Turing equivalence was probably to get dependent types. This gives very powerful static guarantees, including the presence/absence of fields under non-trivial record operations such as merge. In using dependent types, they have also had to give up significantly on type inference, which is really going to annoy the average JavaScript/Ruby programmer.