24 ms·

> I don’t really think this is a problem at all? I mean, at the particular design point the language occupies there are basically three possibilities: How so?

> I don’t really think this is a problem at all? I mean, at the particular design point the language occupies there are basically three possibilities: How so? Where's the Java/Python-like languages in this consideration, why wouldn't those work? Or something more like Ansible? I'm pretty sure that you can express the same concepts for the most part in something imperative as well.

colordrops 5y ago
I don't see how you could maintain purity and reproduceability with something imperative. I used ansible for a while and it was a mess that broke half the time and was difficult to maintain.
- tracnar 5y ago
  Doesn't Bazel do exactly that with their Starlark language based on Python? https://bazel.build/rules/language#mutability https://bazel.build/rules/language#mutability
  - pharmakom 5y ago
    Key aspects of Bazel are not imperative. For example the “load” functions commands must not be computed.
- tjoff 5y ago
  How does that have anything to do with it being imperative and not the fundamental design of the system? How nix is architectured is exactly why it achieves purity and reproducibility, not the configuration language. It isn't even a goal of ansible in the same sense, per design. The building blocks of nix is simple, of course it can be implemented in something imperative.
  - xena 5y ago
    Think about it like this: with Ansible you start from some unknown state of the system and then iteratively do actions to hopefully get what you want. With Nix you know what the base state of the system is, and the configuration language lets you describe your desired end state. Then the computer figures out what to do in the middle. The fundamental design of Ansible is that you do imperative steps to reach an end state. The fundamental design of NixOS is that you describe the end state and the compiler derives the intermediate steps. Yes, normally things work out with Ansible, but the devil is in the details and can involve a lot of fighting configurations that are non-cromulent.
    - tjoff 5y ago
      Yes - that was exactly my point. It has nothing to do with the configuration language. The end result of nix(os) could just as well be done from a bash script (not advisable, no). And the problems with ansible would not be any better off if they had used something like the nix-language instead.
      - xena 5y ago
        Fun fact: Nix is really a compiler from a weird Haskell like DSL to bash!
    - sciurus 5y ago
      > The fundamental design of Ansible is that you do imperative steps to reach an end state. No, the fundamental design of configuration management systems (Ansible, Puppet, Chef, Salt, CFEngine) is that they are declarative. You say "this software should be installed, this user should exist, these files should have these contents and permissions, etc" and they figure out the right steps to take on your OS to move from the current state to your desired end state. You are right though that they can produce unexpected results since they do not describe the entire system from the ground up.
      - colordrops 5y ago
        That's fair, though ansible has ordered steps, so it's not entirely declarative. You are right that imperative definition is not a problem for reproduceability, but it does make things harder to reason about since you have to mentally keep track of state between steps.
- deltaonefour 5y ago
  You can. So long as the impurity is scoped to be local within a package. And the system itself only reads the output of your language to configure something. Example: def packageInstaller(pkgs: Pkg) -> Derivation: x = { req: [pkgs.a, pkgs.b] build: ... } x[req].append(pkgs.c) return buildDerivation(x) packageInstaller outputs a single package description, but you can also create something similar that easily outputs an entire configuration for the OS. You just need to keep mutation scoped within the language. The configuration that is outputted is immutable and whatever thing that's processing that configuration is not within user control. Keep in mind that mutability and immutability share a certain isomorphism. Within the nix expression language you often do something similar to mutation just as any other functional language. For example: listVersion3 = listVersion1 ++ AdditionalStuff It's just that in functional languages you're forced to keep versions of each mutation.
cwp 5y ago
Yes and no. Yes, because Turing completeness. But no, because if you tried to do something like the Nix package manager with a language like Python or Java or even Scheme (yeah, I'm looking at you Guix), you'd end up with a worse system. What's great about Nix the language is that its design matches beautifully with its purpose. Nixpkgs is a single giant program, the output of which is a set of 70,000 package definitions. Since Nix uses lazy evaluation, you can specify one of the attributes of that set, a single package, and the Nix interpreter will evaluate just that one package expression. Of course, if that package depends on other packages, those will get evaluated as well, because they're needed as inputs to the package you care about. So package dependencies emerge naturally as computational dependencies between function invocations. This is really powerful, and over the years, Nixpkgs has evolved more and more sophisticated ways of defining packages - package overrides, the overlay system, and most recently, flakes. So yes, using an imperative language would technically be possible, but I doubt that you could use it to build the largest and freshest package repository with a fraction of the contributors that other packaging systems have.
- emteycz 5y ago
  Well said. Using imperative language would be possible but then it'd look like a bunch of function invocations, not like a class declaration. And people would be asking "this is crazy, why don't you create a DSL for this? My IDE can't ever understand this".
- tjoff 5y ago
  I really struggle with selling argument of lazy evaluation in the nix context, it is not a big deal. Sure, nix has been built around it, but it could just as well have been built around something else. And most certainly can't see how that would influence the number of packages. And honestly, the package situation is the biggest downfall of nix(os), there are tons of packages, yes. But they are often buggy (simply because nix is so different), not maintained (understandable, people move on), or plainly missing (because the few maintainers don't cover even popular niches). Of course that is going to be the case for a new system, but I'm flabbergasted every time it is being touted as an advantage of nix(os). And I feel that it is quite counter-productive tricking people into the ecosystem and giving the impression that it is on par or even better than any mainstream OS. It is making great progress, but it has a long way to go.
yakak 5y ago
> Or something more like Ansible? This seems to be the argument that comes up over and over with go, JavaScript, Ansible, etc. Sure one can take an imperative language and try to follow FP principles to emulate good things. Then one is comfortable with how it works while not really understanding FP and when it trashes the system by being implemented as a sequence of trashes of the state it is just a learning experience of user error.
ris 5y ago
> Or something more like Ansible? If you make it more like Ansible, you will end up with ... Ansible. If you want to use Ansible you are perfectly welcome to, but as a Nix person the entire Ansible way looks like a completely wrongheaded approach to system management.
- sshine 5y ago
  If manual Linux server maintenance is like working with clay, Ansible is like a clay gun. It works great when you have to initiate work on a server you have limited control over; e.g. one that is provisioned from a cloud provider. And where I don’t need more than a few commands, and never need to replay them. Ansible is “bottom-up” (you start in the middle of a server’s lifetime without knowing everything); many more cloudy solutions like Kubernetes assume that a bunch of infrastructure is already set up; container registries, build servers, and the control plane. My first Ansible use-case was to install K3s on a dedicated machine where I could only choose the OS image from a few options like Debian and CentOS.
mananaysiempre 5y ago
You could make a package manager or distribution along the principles of Nix with any reasonable language, of course. Insisting on first-order data (i.e. not functions) exclusively would probably have the standard problems of trying to cram into a declarative configuration something that really wants to be a program (“YAML ops”, programs-as-SQL-tables disasters like Windows Installer, XML hells like Ant and XSLT, etc.); but once you go all the way to a general programming language, sure. I would probably prefer Lua to Python because while Python can sort of do declarative EDSLs they invariably suck, and the expressive power and general style of the two is largely similar; Java has all those problems times a hundred so I expect you would end up staging the whole thing into a config processor in Java and a config in something else; but while the ergonomics may vary it’s certainly not impossible. Guix, Pacman with Aconfmgr, or the venerable GNU Stow are all this to some degree. That is not what I was saying, though. The semantics of the Nix language (JSON-like data model, lazy, higher-order, dynamic types) are a vital part of the implementation approach taken by the Nix package manager, and once those semantics are fixed the syntax (so I argued) has to be broadly similar to what it is now. (You could force a Java- or Python-like syntax upon an ML- or Scheme-like language if you really wanted, but the result would just be miserable to program in; the F# people tried but mostly gave up.) Is that approach the right one? I suspect so, but it is still to early in the life of the whole idea to be really sure. The idea that a dynamically-typed lazy (thus pure) higher-order language is a good middle ground between a completely declarative configuration and a completely imperative setup script (neither of which can usually stand alone) is a fresh and interesting one, and I’d very much like to see it explored. I’ll be the first to admit that the documentation on Nix internals is lacking, but once the requisite source spelunking is done it’s delightful how the whole thing comes together and how much flexibility being written in itself affords it. (Did you know that NixOS was originally a fun experimental addon to the whole thing? The packaging language accidentally ended up being powerful enough to make a usable and novel system configuration language.) But the complaint seemed to be that the syntax (which is relatively superficial and does not require particularly deep argumentation) is Haskell-like, not that the implementation approach (which is crucial to the whole design and requires arguments on the scale of a research paper) is wrong. My point was that the latter pretty much implies the former.
- 5y ago
soraminazuki 5y ago
The Nix language's main concern is handling large scale configuration. General purpose programming and scripting is completely out of scope. The Nixpkgs repository has over 80,000 packages [1] contained in a single key-value store. The design of the Nix language not only makes this sustainable, but even compelling. For one thing, Nix package descriptions are composable and easy to manipulate programmatically. It's common practice to create new packages on the fly out of existing package definitions, and it's what the Nix language is particularly good at. Things like this add a lot of complex inter-dependencies within the Nixpkgs code base, and it still manages to be comprehensible. The same approach isn't practical with general purpose imperative programming languages. A quick glance at Nixpkgs repository reveals that it contains over 2.4 million lines of Nix code. Imagine that much Java/Python code manipulating a single global mutable data structure. It can only result in a nightmare. Additionally, the notion that the Nix is Haskell and is therefore incomprehensible is an often repeated meme without any backing details and needs to stop. If any language is similar to Nix, it's Jsonnet. But I haven't seen a single person criticizing Jsonnet [2] for being Haskell. If you can understand JSON and have no problem writing functions for any common language, then there really should be no problem learning the Nix language. [1]: https://search.nixos.org/packages https://search.nixos.org/packages [2]: https://jsonnet.org https://jsonnet.org

colordrops 5y ago

I don't see how you could maintain purity and reproduceability with something imperative. I used ansible for a while and it was a mess that broke half the time and was difficult to maintain.

tracnar 5y ago
Doesn't Bazel do exactly that with their Starlark language based on Python? https://bazel.build/rules/language#mutability https://bazel.build/rules/language#mutability
- pharmakom 5y ago
  Key aspects of Bazel are not imperative. For example the “load” functions commands must not be computed.
tjoff 5y ago
How does that have anything to do with it being imperative and not the fundamental design of the system? How nix is architectured is exactly why it achieves purity and reproducibility, not the configuration language. It isn't even a goal of ansible in the same sense, per design. The building blocks of nix is simple, of course it can be implemented in something imperative.
- xena 5y ago
  Think about it like this: with Ansible you start from some unknown state of the system and then iteratively do actions to hopefully get what you want. With Nix you know what the base state of the system is, and the configuration language lets you describe your desired end state. Then the computer figures out what to do in the middle. The fundamental design of Ansible is that you do imperative steps to reach an end state. The fundamental design of NixOS is that you describe the end state and the compiler derives the intermediate steps. Yes, normally things work out with Ansible, but the devil is in the details and can involve a lot of fighting configurations that are non-cromulent.
  - tjoff 5y ago
    Yes - that was exactly my point. It has nothing to do with the configuration language. The end result of nix(os) could just as well be done from a bash script (not advisable, no). And the problems with ansible would not be any better off if they had used something like the nix-language instead.
    - xena 5y ago
      Fun fact: Nix is really a compiler from a weird Haskell like DSL to bash!
  - sciurus 5y ago
    > The fundamental design of Ansible is that you do imperative steps to reach an end state. No, the fundamental design of configuration management systems (Ansible, Puppet, Chef, Salt, CFEngine) is that they are declarative. You say "this software should be installed, this user should exist, these files should have these contents and permissions, etc" and they figure out the right steps to take on your OS to move from the current state to your desired end state. You are right though that they can produce unexpected results since they do not describe the entire system from the ground up.
    - colordrops 5y ago
      That's fair, though ansible has ordered steps, so it's not entirely declarative. You are right that imperative definition is not a problem for reproduceability, but it does make things harder to reason about since you have to mentally keep track of state between steps.
deltaonefour 5y ago
You can. So long as the impurity is scoped to be local within a package. And the system itself only reads the output of your language to configure something. Example: def packageInstaller(pkgs: Pkg) -> Derivation: x = { req: [pkgs.a, pkgs.b] build: ... } x[req].append(pkgs.c) return buildDerivation(x) packageInstaller outputs a single package description, but you can also create something similar that easily outputs an entire configuration for the OS. You just need to keep mutation scoped within the language. The configuration that is outputted is immutable and whatever thing that's processing that configuration is not within user control. Keep in mind that mutability and immutability share a certain isomorphism. Within the nix expression language you often do something similar to mutation just as any other functional language. For example: listVersion3 = listVersion1 ++ AdditionalStuff It's just that in functional languages you're forced to keep versions of each mutation.

cwp 5y ago

Yes and no. Yes, because Turing completeness. But no, because if you tried to do something like the Nix package manager with a language like Python or Java or even Scheme (yeah, I'm looking at you Guix), you'd end up with a worse system. What's great about Nix the language is that its design matches beautifully with its purpose. Nixpkgs is a single giant program, the output of which is a set of 70,000 package definitions. Since Nix uses lazy evaluation, you can specify one of the attributes of that set, a single package, and the Nix interpreter will evaluate just that one package expression. Of course, if that package depends on other packages, those will get evaluated as well, because they're needed as inputs to the package you care about. So package dependencies emerge naturally as computational dependencies between function invocations. This is really powerful, and over the years, Nixpkgs has evolved more and more sophisticated ways of defining packages - package overrides, the overlay system, and most recently, flakes. So yes, using an imperative language would technically be possible, but I doubt that you could use it to build the largest and freshest package repository with a fraction of the contributors that other packaging systems have.

emteycz 5y ago
Well said. Using imperative language would be possible but then it'd look like a bunch of function invocations, not like a class declaration. And people would be asking "this is crazy, why don't you create a DSL for this? My IDE can't ever understand this".
tjoff 5y ago
I really struggle with selling argument of lazy evaluation in the nix context, it is not a big deal. Sure, nix has been built around it, but it could just as well have been built around something else. And most certainly can't see how that would influence the number of packages. And honestly, the package situation is the biggest downfall of nix(os), there are tons of packages, yes. But they are often buggy (simply because nix is so different), not maintained (understandable, people move on), or plainly missing (because the few maintainers don't cover even popular niches). Of course that is going to be the case for a new system, but I'm flabbergasted every time it is being touted as an advantage of nix(os). And I feel that it is quite counter-productive tricking people into the ecosystem and giving the impression that it is on par or even better than any mainstream OS. It is making great progress, but it has a long way to go.

yakak 5y ago

> Or something more like Ansible? This seems to be the argument that comes up over and over with go, JavaScript, Ansible, etc. Sure one can take an imperative language and try to follow FP principles to emulate good things. Then one is comfortable with how it works while not really understanding FP and when it trashes the system by being implemented as a sequence of trashes of the state it is just a learning experience of user error.

ris 5y ago

> Or something more like Ansible? If you make it more like Ansible, you will end up with ... Ansible. If you want to use Ansible you are perfectly welcome to, but as a Nix person the entire Ansible way looks like a completely wrongheaded approach to system management.

sshine 5y ago
If manual Linux server maintenance is like working with clay, Ansible is like a clay gun. It works great when you have to initiate work on a server you have limited control over; e.g. one that is provisioned from a cloud provider. And where I don’t need more than a few commands, and never need to replay them. Ansible is “bottom-up” (you start in the middle of a server’s lifetime without knowing everything); many more cloudy solutions like Kubernetes assume that a bunch of infrastructure is already set up; container registries, build servers, and the control plane. My first Ansible use-case was to install K3s on a dedicated machine where I could only choose the OS image from a few options like Debian and CentOS.

mananaysiempre 5y ago

You could make a package manager or distribution along the principles of Nix with any reasonable language, of course. Insisting on first-order data (i.e. not functions) exclusively would probably have the standard problems of trying to cram into a declarative configuration something that really wants to be a program (“YAML ops”, programs-as-SQL-tables disasters like Windows Installer, XML hells like Ant and XSLT, etc.); but once you go all the way to a general programming language, sure. I would probably prefer Lua to Python because while Python can sort of do declarative EDSLs they invariably suck, and the expressive power and general style of the two is largely similar; Java has all those problems times a hundred so I expect you would end up staging the whole thing into a config processor in Java and a config in something else; but while the ergonomics may vary it’s certainly not impossible. Guix, Pacman with Aconfmgr, or the venerable GNU Stow are all this to some degree. That is not what I was saying, though. The semantics of the Nix language (JSON-like data model, lazy, higher-order, dynamic types) are a vital part of the implementation approach taken by the Nix package manager, and once those semantics are fixed the syntax (so I argued) has to be broadly similar to what it is now. (You could force a Java- or Python-like syntax upon an ML- or Scheme-like language if you really wanted, but the result would just be miserable to program in; the F# people tried but mostly gave up.) Is that approach the right one? I suspect so, but it is still to early in the life of the whole idea to be really sure. The idea that a dynamically-typed lazy (thus pure) higher-order language is a good middle ground between a completely declarative configuration and a completely imperative setup script (neither of which can usually stand alone) is a fresh and interesting one, and I’d very much like to see it explored. I’ll be the first to admit that the documentation on Nix internals is lacking, but once the requisite source spelunking is done it’s delightful how the whole thing comes together and how much flexibility being written in itself affords it. (Did you know that NixOS was originally a fun experimental addon to the whole thing? The packaging language accidentally ended up being powerful enough to make a usable and novel system configuration language.) But the complaint seemed to be that the syntax (which is relatively superficial and does not require particularly deep argumentation) is Haskell-like, not that the implementation approach (which is crucial to the whole design and requires arguments on the scale of a research paper) is wrong. My point was that the latter pretty much implies the former.

5y ago

soraminazuki 5y ago

The Nix language's main concern is handling large scale configuration. General purpose programming and scripting is completely out of scope. The Nixpkgs repository has over 80,000 packages [1] contained in a single key-value store. The design of the Nix language not only makes this sustainable, but even compelling. For one thing, Nix package descriptions are composable and easy to manipulate programmatically. It's common practice to create new packages on the fly out of existing package definitions, and it's what the Nix language is particularly good at. Things like this add a lot of complex inter-dependencies within the Nixpkgs code base, and it still manages to be comprehensible. The same approach isn't practical with general purpose imperative programming languages. A quick glance at Nixpkgs repository reveals that it contains over 2.4 million lines of Nix code. Imagine that much Java/Python code manipulating a single global mutable data structure. It can only result in a nightmare. Additionally, the notion that the Nix is Haskell and is therefore incomprehensible is an often repeated meme without any backing details and needs to stop. If any language is similar to Nix, it's Jsonnet. But I haven't seen a single person criticizing Jsonnet [2] for being Haskell. If you can understand JSON and have no problem writing functions for any common language, then there really should be no problem learning the Nix language. [1]: https://search.nixos.org/packages https://search.nixos.org/packages [2]: https://jsonnet.org https://jsonnet.org