5 ms·

> KCL's automatic merge mechanism for isolated configuration blocks promotes high scalability. This part scares me a bit. How does it merge? In cuelang, everyt

> KCL's automatic merge mechanism for isolated configuration blocks promotes high scalability. This part scares me a bit. How does it merge? In cuelang, everything has to agree otherwise it's an error. If KCL has a situation where one file can override another, I'm out. That gets way too hairy way too fast. Edit: data0 = {id: 1} | {id: 2} # Error：conflicting values between {'id': 2} and {'id': 1} data1 = {id: 1} | {id = 2} # Ok, the value of `data` is {"id": 2} There are the foot guns

drdaeman 2y ago
nixpkgs.lib has mkDefault/mkForce/mkOverride system (which I'm not a fond of because of the poor type system, but at least it works) that feels like a better approach than this weirdness.
- soraminazuki 2y ago
  The Nixpkgs module system that you're describing does have types though. The whole purpose of that system is to attach types and documentation to data.
  - drdaeman 2y ago
    Unless I'm misunderstanding how it all works, it's just attrsets underneath. Like JSON objects with some semantics given to them - and those (and their YAML counterparts) are not exactly great. In particular, I believe (I could be wrong here) this results in sometimes hard-to-understand error traces if something gets messed up.
    - soraminazuki 2y ago
      The module system type checks each attribute in a attrset. It merges all modules, ensures there are no conflicts, type checks and returns the resulting attrset. Error messages indicate the offending option definition along with its location in the case of conflicts or type errors, which should be enough for enough for debugging.
      - drdaeman 2y ago
        I vaguely remember shooting myself in my foot by doing some weird stuff with deep attrset transformations, where I managed to screw-up those "special" attrsets by making a mistake in a function that tried to build module options dynamically from another "transplant" module's options. Surely, that was weird stuff, and sadly I don't have a history of what exactly I did back then (it was couple years ago), but I remember having to introspect with Nix REPL to understand where I messed up. I would've loved it more if Nix had ability to create custom types, ideally GADTs. Tagged attrsets work for sure, but a better type system would've probably made things easier and nicer.
- yjftsjthsd-h 2y ago
  ansible has what I think is a sort of similar system, where a role can define default values for its variables but they can be overridden in a fairly deep hierarchy of successive priorities; it helps that the lowest tier is literally labeled "defaults" so you automatically expect it to be overwritten, though in the upper layers I still have to look up to see which thing takes precedence.
  - soraminazuki 2y ago
    The merging behavior of Nixpkgs modules are much simpler. It doesn't have orders or complex priority rules that depend on where an option is defined. Options have types, which also specifies the merging behavior. Merging of collection types just work.
aliasxneo 2y ago
This seems to be a feature to some people. They must exclusively work on tiny, compartmentalized projects. Otherwise, I don't understand the allure. We use CUE a lot, and when I teach people about this particular constraint (only one concrete value is allowed), they get frustrated, as if the alternative is better.
patrick451 2y ago
What do you mean `{id = 2}` ? The ROS parameter system does automatic overwriting of values. You end up with different subsystems loading different param files in the parameter server, with each one overwriting values from the last one. It is a complete nightmare.
lmpdev 2y ago
Hahaha oh my lord State will always err… find a way
peefy 2y ago
Multiple configuration merging strategies do not imply inconsistency or uncertainty, nor do they mean that one file can overwrite or patch another. KCL also provides a certain degree of immutability, such as ensuring that exported variables are immutable and that configurations in a file cannot be modified. You can only import it and use it to generate new configurations. In addition, in the cloud native field such as Kubernetes, configuration merging operations can be seen everywhere, such as the built-in JsonMergePatch strategy, Kustomize, and multiple environment configurations for developers. You can find similar corresponding features in KCL. In fact, KCL also has the idempotent merge operation of cuelang, and languages such as jsonnet, pkl, Nickel also have certain configuration merge capabilities. Only cuelang strictly follows the idempotency of merging, which to some extent limits its use and understanding.
- from-nibly 2y ago
  yeah, just because people are doing it doesn't mean it's good. You need to have the strictness of cuelang. But I recognize that KCL will probably win overall cause it's shiny and easy to get started with. But this will cause a LOT of problems for everyone as it already does, with things like json merge patch etc.

drdaeman 2y ago

nixpkgs.lib has mkDefault/mkForce/mkOverride system (which I'm not a fond of because of the poor type system, but at least it works) that feels like a better approach than this weirdness.

soraminazuki 2y ago
The Nixpkgs module system that you're describing does have types though. The whole purpose of that system is to attach types and documentation to data.
- drdaeman 2y ago
  Unless I'm misunderstanding how it all works, it's just attrsets underneath. Like JSON objects with some semantics given to them - and those (and their YAML counterparts) are not exactly great. In particular, I believe (I could be wrong here) this results in sometimes hard-to-understand error traces if something gets messed up.
  - soraminazuki 2y ago
    The module system type checks each attribute in a attrset. It merges all modules, ensures there are no conflicts, type checks and returns the resulting attrset. Error messages indicate the offending option definition along with its location in the case of conflicts or type errors, which should be enough for enough for debugging.
    - drdaeman 2y ago
      I vaguely remember shooting myself in my foot by doing some weird stuff with deep attrset transformations, where I managed to screw-up those "special" attrsets by making a mistake in a function that tried to build module options dynamically from another "transplant" module's options. Surely, that was weird stuff, and sadly I don't have a history of what exactly I did back then (it was couple years ago), but I remember having to introspect with Nix REPL to understand where I messed up. I would've loved it more if Nix had ability to create custom types, ideally GADTs. Tagged attrsets work for sure, but a better type system would've probably made things easier and nicer.
yjftsjthsd-h 2y ago
ansible has what I think is a sort of similar system, where a role can define default values for its variables but they can be overridden in a fairly deep hierarchy of successive priorities; it helps that the lowest tier is literally labeled "defaults" so you automatically expect it to be overwritten, though in the upper layers I still have to look up to see which thing takes precedence.
- soraminazuki 2y ago
  The merging behavior of Nixpkgs modules are much simpler. It doesn't have orders or complex priority rules that depend on where an option is defined. Options have types, which also specifies the merging behavior. Merging of collection types just work.

aliasxneo 2y ago

This seems to be a feature to some people. They must exclusively work on tiny, compartmentalized projects. Otherwise, I don't understand the allure. We use CUE a lot, and when I teach people about this particular constraint (only one concrete value is allowed), they get frustrated, as if the alternative is better.

patrick451 2y ago

What do you mean `{id = 2}` ? The ROS parameter system does automatic overwriting of values. You end up with different subsystems loading different param files in the parameter server, with each one overwriting values from the last one. It is a complete nightmare.

lmpdev 2y ago

Hahaha oh my lord State will always err… find a way

peefy 2y ago

Multiple configuration merging strategies do not imply inconsistency or uncertainty, nor do they mean that one file can overwrite or patch another. KCL also provides a certain degree of immutability, such as ensuring that exported variables are immutable and that configurations in a file cannot be modified. You can only import it and use it to generate new configurations. In addition, in the cloud native field such as Kubernetes, configuration merging operations can be seen everywhere, such as the built-in JsonMergePatch strategy, Kustomize, and multiple environment configurations for developers. You can find similar corresponding features in KCL. In fact, KCL also has the idempotent merge operation of cuelang, and languages such as jsonnet, pkl, Nickel also have certain configuration merge capabilities. Only cuelang strictly follows the idempotency of merging, which to some extent limits its use and understanding.

from-nibly 2y ago
yeah, just because people are doing it doesn't mean it's good. You need to have the strictness of cuelang. But I recognize that KCL will probably win overall cause it's shiny and easy to get started with. But this will cause a LOT of problems for everyone as it already does, with things like json merge patch etc.