3 ms·

If your configurations are simple, you don’t need this. If they are not, you might. Because you have not run into the problems this has addressed in your caree

by n42 3y ago

If your configurations are simple, you don’t need this. If they are not, you might. Because you have not run into the problems this has addressed in your career, does not mean that you know better than Apple how to solve them. In fact, it means like you are uniquely unqualified to solve them. Acting like you are in a severely condescending dismissal of the engineers who worked on this makes for boring conversation.

flir 3y ago
Maybe give us an example? It's an interesting question - just how complex are our biggest configuration problems today?
- n42 3y ago
  A team of 6 responsible for overseeing the deployment of all software in a department of 1000 to any number of deployment environments with tight budgets on durability, delivery time, and the autonomy of the teams that are writing the configuration may reach for this, or something similar, over the Windows Registry
- cstrahan 3y ago
  Kubernetes config is a decent example. I had ChatGPT generate a representative silly example -- the content doesn't matter so much as the structure: https://gist.github.com/cstrahan/528b00cd5c3a22e3d8f057bb1a75ea61 https://gist.github.com/cstrahan/528b00cd5c3a22e3d8f057bb1a7... Now consider 100s (if not 1000s) of such files. I haven't given Pkl an in depth look yet, but I can say that the Industry Standard™ of "simple YAML" + string substitution (with delicate, error prone indentation -- since YAML is indentation sensitive) is easily beat by any of: - https://jsonnet.org/ https://jsonnet.org/ - https://nickel-lang.org/ https://nickel-lang.org/ - https://nixos.org/manual/nix/stable/language/index.html https://nixos.org/manual/nix/stable/language/index.html - https://dhall-lang.org/ https://dhall-lang.org/ - (insert many more here, probably including Pkl)
  - flir 3y ago
    You're absolutely right, k8s is a pig.
    - fzeindl 3y ago
      Especially since k8s allows using arbitrary labels for plugins, effectively creating „stringly typed“ programming. Nobody is stopped from compiling code, converting it to base64 and storing it into a label for later execution. Arbitrary parameters like this are the opposite of a unifying abstraction. That ingress-behavior wasn‘t defined but pluggable to suit existing load balancers also broke the abstraction.
      - flir 3y ago
        I do feel there's a simple abstraction under there somewhere - k8s config is just a big tree, after all. And adding pkl's loops on top of that would probably reduce a lot of duplication.
  - zelphirkalt 3y ago
    Maybe not always, but surely in most cases of too complex a config, it is a case of ad-hoc grown config, representing what one wants to actually configure badly, and/or underlying abstractions of the thing one wants to configure matching badly what one wants to do. In most cases it would be good to take a step back, or multiple ones at that, and really ask oneself: "What is it, that I actually want to configure here?" and think about why it cannot be a simpler config. What abstractions would actually make expressing that config easy. Often one will get to a very simple config format in the end. Of course, when one has to deal with very complex formats created by others, already widespread in use, on cannot easily change the format. Maybe that is the reason we get these meta config tools.
    - cwp 3y ago
      Sure. There are undoubtedly a lot of config formats that are overly complex. But sometimes the complexity is irreducible. Kubernetes is one such case. The model is very well thought out, and just about as simple as it could get without removing functionality. It has sensible defaults, built-in versioning, well-defined schema etc. But if you want to describe a complete installation of a distributed system with many heterogenous processes, spread across many hosts, communicating in specific ways, with specific permissions, persistence, isolation, automatic scaling, resilience, etc, there are a lot of details. I've worked with systems that have thousands of lines of configuration, and honestly that's not extraordinary. Many people on this site will rightly scoff and say, "psshh, that's nothing." Configuration languages are a really important area of research in the tech industry right now, and every time someone posts one on here, there are a huge number of dismissive comments. Fine. Not everyone has this problem, but it's a real problem, and solving it represents a real advance in the state of the art.
- kika 3y ago
  I just run a count, on the infrastructure configuration for a relatively small, but already public company. 13k lines of YAML alone. Some of it is generated (from "YAML with macros") by AWS CDK parody I invented before CDK became a thing (and being an old school Unix pervert I did it with awk and m4). There's also 1400 lines of written or generated JSON that still needs to be managed. And that's a successful, but single-product company. I also worked for a company which had a single product but ran it for their clients heavily (very heavily) customized, you can easily 10x the numbers above and you still have to do that with a single team and go back to your family every night and not riding an ambulance to the nearest mental facility. YAML is like the object code, it's not a configuration language. (and that's why I prefer JSON, it's less error prone and easier to generate). Writing YAML manually is like programming in machine code back in the 80s. Possible, but why?
Fischgericht 3y ago
Well, Apple is the gold standard in creating new proprietary programming languages for often just evil reasons - typically for vendor lock-in. They invented the Objective C and Swift, and made it pretty much impossible to use any standard language to target their platforms in a cross-platform way. I can access the Windows API with any language I want. I can access the Linux Kernel API with any language I want. I can access the BSD API with any language I want. I even can use any language I want on the C64 to call native operating system functions. So, yes, I am taking the liberty to believe I know better than Apple. TBH I don't regard this conclusion as rocket science, so no ego involved here.
- bioballer 3y ago
  There's no "walled garden" here. We want Pkl to be useful for many developers, for a variety of purposes. Hopefully that's clear enough with Java, Kotlin, and Go being amongst the various supported language bindings. We also already support Linux, and plan on supporting Windows. It's also open sourced under the Apache 2.0 license, which grants you the right to distribute and modify it. Cheers
  - Fischgericht 3y ago
    I appreciate you having good intentions. I don't mean to be disrespectful. This however does not change my position that we have more than enough scripting languages focused on string operations, and don't need another language. And especially not a bloated one that itself depends on huge frameworks. If the same task can be solved using a 76KB standard Unix tool, creating a custom language with a huge bloated runtime simply can not be justified. It's bringing a sledgehammer for a task where a swiss pocket knife would be appropriate. And my position also stands that it would make far more sense to finally agree on a config file standard format, with native parsers existing in every major programming language, so people stop re-inventing the wheel again and again and again, with the wheel becoming heavier and bulkier and less round every time.
    - latexr 3y ago
      > And my position also stands that it would make far more sense to finally agree on a config file standard format It would also make far more sense for humans to agree on most things and work towards a common goal that benefitted us all, but that’s just as much of a pipe dream. Proposing something that will never happen is not a practical solution. https://xkcd.com/927/ https://xkcd.com/927/

by n42 3y ago

flir 3y ago

Maybe give us an example? It's an interesting question - just how complex are our biggest configuration problems today?

n42 3y ago
A team of 6 responsible for overseeing the deployment of all software in a department of 1000 to any number of deployment environments with tight budgets on durability, delivery time, and the autonomy of the teams that are writing the configuration may reach for this, or something similar, over the Windows Registry
cstrahan 3y ago
Kubernetes config is a decent example. I had ChatGPT generate a representative silly example -- the content doesn't matter so much as the structure: https://gist.github.com/cstrahan/528b00cd5c3a22e3d8f057bb1a75ea61 https://gist.github.com/cstrahan/528b00cd5c3a22e3d8f057bb1a7... Now consider 100s (if not 1000s) of such files. I haven't given Pkl an in depth look yet, but I can say that the Industry Standard™ of "simple YAML" + string substitution (with delicate, error prone indentation -- since YAML is indentation sensitive) is easily beat by any of: - https://jsonnet.org/ https://jsonnet.org/ - https://nickel-lang.org/ https://nickel-lang.org/ - https://nixos.org/manual/nix/stable/language/index.html https://nixos.org/manual/nix/stable/language/index.html - https://dhall-lang.org/ https://dhall-lang.org/ - (insert many more here, probably including Pkl)
- flir 3y ago
  You're absolutely right, k8s is a pig.
  - fzeindl 3y ago
    Especially since k8s allows using arbitrary labels for plugins, effectively creating „stringly typed“ programming. Nobody is stopped from compiling code, converting it to base64 and storing it into a label for later execution. Arbitrary parameters like this are the opposite of a unifying abstraction. That ingress-behavior wasn‘t defined but pluggable to suit existing load balancers also broke the abstraction.
    - flir 3y ago
      I do feel there's a simple abstraction under there somewhere - k8s config is just a big tree, after all. And adding pkl's loops on top of that would probably reduce a lot of duplication.
- zelphirkalt 3y ago
  Maybe not always, but surely in most cases of too complex a config, it is a case of ad-hoc grown config, representing what one wants to actually configure badly, and/or underlying abstractions of the thing one wants to configure matching badly what one wants to do. In most cases it would be good to take a step back, or multiple ones at that, and really ask oneself: "What is it, that I actually want to configure here?" and think about why it cannot be a simpler config. What abstractions would actually make expressing that config easy. Often one will get to a very simple config format in the end. Of course, when one has to deal with very complex formats created by others, already widespread in use, on cannot easily change the format. Maybe that is the reason we get these meta config tools.
  - cwp 3y ago
    Sure. There are undoubtedly a lot of config formats that are overly complex. But sometimes the complexity is irreducible. Kubernetes is one such case. The model is very well thought out, and just about as simple as it could get without removing functionality. It has sensible defaults, built-in versioning, well-defined schema etc. But if you want to describe a complete installation of a distributed system with many heterogenous processes, spread across many hosts, communicating in specific ways, with specific permissions, persistence, isolation, automatic scaling, resilience, etc, there are a lot of details. I've worked with systems that have thousands of lines of configuration, and honestly that's not extraordinary. Many people on this site will rightly scoff and say, "psshh, that's nothing." Configuration languages are a really important area of research in the tech industry right now, and every time someone posts one on here, there are a huge number of dismissive comments. Fine. Not everyone has this problem, but it's a real problem, and solving it represents a real advance in the state of the art.
kika 3y ago
I just run a count, on the infrastructure configuration for a relatively small, but already public company. 13k lines of YAML alone. Some of it is generated (from "YAML with macros") by AWS CDK parody I invented before CDK became a thing (and being an old school Unix pervert I did it with awk and m4). There's also 1400 lines of written or generated JSON that still needs to be managed. And that's a successful, but single-product company. I also worked for a company which had a single product but ran it for their clients heavily (very heavily) customized, you can easily 10x the numbers above and you still have to do that with a single team and go back to your family every night and not riding an ambulance to the nearest mental facility. YAML is like the object code, it's not a configuration language. (and that's why I prefer JSON, it's less error prone and easier to generate). Writing YAML manually is like programming in machine code back in the 80s. Possible, but why?

Fischgericht 3y ago

Well, Apple is the gold standard in creating new proprietary programming languages for often just evil reasons - typically for vendor lock-in. They invented the Objective C and Swift, and made it pretty much impossible to use any standard language to target their platforms in a cross-platform way. I can access the Windows API with any language I want. I can access the Linux Kernel API with any language I want. I can access the BSD API with any language I want. I even can use any language I want on the C64 to call native operating system functions. So, yes, I am taking the liberty to believe I know better than Apple. TBH I don't regard this conclusion as rocket science, so no ego involved here.

bioballer 3y ago
There's no "walled garden" here. We want Pkl to be useful for many developers, for a variety of purposes. Hopefully that's clear enough with Java, Kotlin, and Go being amongst the various supported language bindings. We also already support Linux, and plan on supporting Windows. It's also open sourced under the Apache 2.0 license, which grants you the right to distribute and modify it. Cheers
- Fischgericht 3y ago
  I appreciate you having good intentions. I don't mean to be disrespectful. This however does not change my position that we have more than enough scripting languages focused on string operations, and don't need another language. And especially not a bloated one that itself depends on huge frameworks. If the same task can be solved using a 76KB standard Unix tool, creating a custom language with a huge bloated runtime simply can not be justified. It's bringing a sledgehammer for a task where a swiss pocket knife would be appropriate. And my position also stands that it would make far more sense to finally agree on a config file standard format, with native parsers existing in every major programming language, so people stop re-inventing the wheel again and again and again, with the wheel becoming heavier and bulkier and less round every time.
  - latexr 3y ago
    > And my position also stands that it would make far more sense to finally agree on a config file standard format It would also make far more sense for humans to agree on most things and work towards a common goal that benefitted us all, but that’s just as much of a pipe dream. Proposing something that will never happen is not a practical solution. https://xkcd.com/927/ https://xkcd.com/927/