10 ms·
A Nix terminology primer by a newcomer
- Kednicma 6y agoI wish I understood how to bring new things to Nix. Like, if I have some language or framework with a bunch of plugins and modules, and I have the freedom to bend the framework a bit in order to get it working with Nix, then what is the best path for doing that? All of the documentation I've read seems like it's been written by folks who deeply understand not just each different language, but also some sort of central Nix philosophy.
- ingenieroariel 6y agoAre you stuck with the interface for the user? Based on what I saw with python and postgres the latest and greatest seems to be: yourthing_custom = yourthing.withPackages(ps: [ps.plugin1 ps.plugin2 ] ) Or are you stuck with how to implement it?
- juliosueiras 6y agoI think one way to look at nix is that it want to act(in good way) the central source of truth for deterministic package, and one way it goes about it is by doing the step for both system level packages, and language level packages(for example, if you approach a nodejs app, you will lock both the nodejs version, the underlying openssl or other headers lib used by node-gyp, and the node_module packages itself in nix) and that is one of the main flow of nix, take a existing project/app/software, and convert it to usable nix packages, and that work even in edge cases like: - app that is binary , so you can't build it from source, but is using dynamic libraries, then you can use autoPatchelfHook to tell nix to patch the binary to swap out the libraries to nix's version - app that uses a lot of hard coded file system path that you would prefer not to patch them all out, then you can use buildFHSUserEnv to package/run the application/package in full FHS-compatible scenario but the main point is that, nix is extreme in its approach, and the best approach in my opinion, is to do swapping step by step, ex: you want to convert a existing project to nix, lets say nodejs - you add the main deps in the default.nix (nodejs, yarn, high level utility needed to build) - use sandBox false to allow building the app with yarn/npm in internet accessible(by default nix uses sandBox which disallow internet inside build step, which causes issue like npm install not working, etc) - confirm working, then start using tool likes yarn2nix, other similar, and convert the packages itself to a usable set of nix packages though I do agree that nix have the knowledge bias issue, and alot of my learning of nix involve me looking at the nixpkg sources itself and sometime even the nix's code
- throwaway894345 6y agoThe trouble is that it's not just that nix is opinionated, it's that much of the complexity is in patterns and conventions that are implemented in nix. Learning everything about everything about the Nix toolchain is only one third of the battle; the rest is learning all of the conventions and patterns for writing Nix expressions, and these conventions and patterns are subject to variance between target languages. This is made terribly difficult because the language is dynamically typed and it's almost impossible to know about the "shape" of any given parameter because there is no way to know where it is defined in the enormous nixpkgs repo except to grep around and try to find the place where the caller is invoked, what is passed into it, and trace that back to an original import statement (and from there to a file on disk and the corresponding definition). That long tedious process is the hot path for developing in Nix.
- yjftsjthsd-h 6y agoVery much agreed; nix is its own world, which makes it really hard to get into. I wish someone had written "nix but in python" (or bash, or whatever) that still used a nix-store equivalent, kept the overall design and the immutable and reproducible packaging, still connected everything by hash, did all the hydra-style build infrastructure.. but didn't use nix-the-language, and made an explicit effort to use more conventional language wherever reasonably possible. Ideally, it'd even let you write expressions in arbitrary languages; it should be perfectly possible to say, "here is a directory containing 'inputs' and 'output' subdirectories based on your declared dependencies; place any build steps you want in build.sh, so long as they deterministically populate 'output' from the package directories in 'inputs'" (and then run the build a few times, without network access, to make sure). I don't see anything in nix that actually needs nix-the-language, or even a functional language at all, and I think nix-the-package-manager would be far more accessible if they'd not tied nix-the-language to everything.
- chpatrick 6y agoI think functional languages are a good fit for the problem though. Given some input, compute a static build plan. It's very rare that you need to do imperative-style computation for that. What I think Nix really needs is some kind of static typing, because right now if you make a type error the error might appear in a completely random location, making it difficult to debug.
- whateveracct 6y agomkDerivation is a nice hammer to try
- mschwaig 6y agoIf you want to know what the result could look like I would recommend the following blog post by Christine Dodrill where she shows off her setup for working with Go code in Nix: https://christine.website/blog/i-was-wrong-about-nix-2020-02-10 https://christine.website/blog/i-was-wrong-about-nix-2020-02... It's more focused on the result then on how you get there, but that result really impressed me.
- soraminazuki 6y agoAside from the official docs and wikis, I find the nixpkgs repository[1] extremely useful as a reference and is definitely one of the things that make me love Nix. It helped me a lot to get started. Any official package from Nix belongs to this single repository, making it easy to see what other packages are doing. If I want to see how python packaging is done, I simply search "python" in the nixpkgs repository. It's also easier to grasp what is going on in nixpkgs than other packaging systems. The Nix expression language offers better readability than say, RPM spec files. Meanwhile the nixpkgs repository includes support for numerous build systems and frameworks. This combined allows you to create packages with minimal effort most of the time. [1]: https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs
- carlmr 6y agoI've been trying to get a nix file running today, and the terminology definitely gets arcane very fast. I also have the feeling that the tutorials start with the details, instead of starting with an overview so you can understand what is what. This is guide us already very helpful and I think one should read it before the official tutorial. Thank you!
- tomberek 6y agoGot a project or want some guidance? I can help out.
- thibran 6y agoQuestion 1 - What is the correct way to build and install self created derivations? Right now I have a custom fish function: function nix-local-build nix-build --no-out-link \ -E '(import <nixpkgs> {}).callPackage ./'$argv[1]' {}' end To build and install something I run: nix-local-build some.nix nix-env -i [store-path-from-build] The .nix file is in the same format as e.g. pkgs/tools/misc/bat/default.nix. Question 2 - How to include such a derivation into configuration.nix?
- nomurrcy 6y agoOverlays can be very helpful for this type of thing. I have overlays in ~/.config/nixpkgs/overlays.nix (though you do this various ways) My overlay file has the form: let my-packages-overlay = self: super: { foo = super.callPackage ./path/to/package.nix {} } in [ my-packages-overlay ] Then you can just nix-env -iA your-package. See: https://nixos.wiki/wiki/Overlays https://nixos.wiki/wiki/Overlays
- mason55 6y ago> Then you can just nix-env -iA your-package. This is what always gets me stuck. It seems like lots of tutorials assume you're installing packages by hand still. For me, the whole reason I'm using nix is so that I can use the declarative package management. I get having local dev dependencies on a per-project basis and running a nix shell to make those dependencies available in your development environment. But it's just confusing to me how many tutorials suggest running nix-env -i to imperatively install a package. Why would I do that instead of taking advantage of the joys of declarative package management?
- julianeon 6y agoI guess the ideal, for me, would be to have a service where I could upload a Nix file, and then have it generate a server from that. Darklang may be working with the general idea, but I'd like to see it for Nix too.
- ris 6y agoYou're describing something a little like NixOps, which is something like a cross between nix and terraform.
- chpatrick 6y agoYep, that's nixops, although it's a command-line application.
- whateveracct 6y agoNixOps + the kind of money mainstream projects get would be really nice. Sadly NixOps is a lil rough around the edges. I like using it for my projects already though. I manually provision on DO instead of letting the tool do that.
- takeda 6y agoAgree, they approached this by creating a common interface to provision VMs. The good part is that you can deploy the image to different cloud providers with minimum changes. The bad think is that it is utilizing the common components among all providers, that means you essentially can just deploy VM and that's it. No autoscaling etc.
- takeda 6y agoPeople mentioned NixOps (I personally think it is weaker part of Nix, but it is still good for specific use cases) There's also another option, nixpkgs actually has a functionality to generate ec2 image, basd on your configuration.nix[1] You can also generate other images[2] [1] https://github.com/NixOS/nixpkgs/blob/master/nixos/maintainers/scripts/ec2/amazon-image.nix https://github.com/NixOS/nixpkgs/blob/master/nixos/maintaine... [2] https://github.com/nix-community/nixos-generators https://github.com/nix-community/nixos-generators edit: I believe you can use [2] even for AWS images, the raw and qcow2 should work on EC2 as well, the [1] is including some additional settings that make NixOS work better there, although there's nothing stopping you from adding these to your own configuration.nix Also, this is not really a fault of Nix, but in EC2 if you import image it has to go through their conversion mechansim, which takes about 10 minutes :( A kind of workaround around it is to have another instance with attached extra EBS drive, use dd to write raw image on that disk. Then disconnect the EBS, create a snapshot of it and convert the snapshot to an AMI.
- velcrovan 6y agoI like the idea of NixOS. It's somewhat hurt by the fact that it's not a first-class OS at most VPS providers. It also doesn't seem to be quite as capable as Ansible yet for configuration. If there isn't a provided variable for the setting you want, there's not a generic way to add a line to a configuration file, for example. (I might be wrong about this!)
- chpatrick 6y agoYou're right in general, but most nixos modules have an `extraConfig` field where you can put whatever you want.
- thequux 6y agoBoth of these are true, but I find that the second, at least, isn't as much of a problem as you'd expect. Most modules that configure services give you the ability to specify complete config files, which you can build whatever higher level structure you like on top of. For example, the built-in firewall is fairly basic (at least for running on a router; it does nearly everything you'd need for most desktop and server applications). I run NixOS on my router, so I needed something more. So, I wrote https://github.com/thequux/nix-zone-firewall https://github.com/thequux/nix-zone-firewall (note: the readme is slightly out of date, but it does give you the gist of it). First, I built a way to declaratively put rules in different chains (core.nix), and then I built a zone-based firewall on top of that. I then have another layer of configuration options in my router's config (not that repo) to be able to spread configuration across multiple files. Even though this was the first significant bit of Nix that I wrote, I still was able to put the entire thing together in a single evening after work.
- 1000101 6y agoHave you seen https://github.com/elitak/nixos-infect https://github.com/elitak/nixos-infect ? It allows you to easily re-create/infect existing distros into NixOS. Super helpful, works as a charm (tested on DO).
- seddona 6y agoI'm a longtime Ubuntu user and recently gave NixOS a try as most of our team use it. I have to admit I was highly skeptical as it all looks very complicated. The learning curve was a bit steep due to the documentation and odd terminology. But after you get going it solves so many problems. The fact you can declare you're entire desired set up in a single configuration.nix then "nixos-rebuild switch" and you're done saves so much time.
- samvher 6y agoI tried to fully switch my work environment to NixOS 2 or 3 years ago. I really like the concept and I think it's a very good approach to reproducible build environments, in theory it's much cleaner than the massive data replication you get with e.g. Docker. I only lasted for a month or two though, for a couple of reasons. The main one was that I had a lot of trouble with Python packages - Nix suggests a certain approach for dealing with them which didn't work very well for me and I didn't manage to set up an alternative that I liked. (For Haskell I loved working with Nix though!) A lot of software I use also wasn't available in the repositories and I tried to build some packages but found the Nix language a bit obtuse. The basic syntax and logic are not too hard but I still found many of the packages I used as examples very confusing. In the end I'm back to Ubuntu using Docker whenever I need reproducible environments.
- metta2uall 6y agoThanks. I also found this Cheatsheet from the NixOS Wiki with comparisons to apt-get to be useful: https://nixos.wiki/index.php?title=Cheatsheet&useskin=vector https://nixos.wiki/index.php?title=Cheatsheet&useskin=vector I would also love to see a tutorial for beginners on Linux that covers common tasks like viewing changelogs, creating a simple package, and pinning multiple dependencies in an environment.
- 1000101 6y agoI'm actually aiming to do that in a series of post in the upcoming month(s)... I've started with a noob-friendly (written by a noob!) introduction and plan to continue with installer, basic package creation, module creation, discussing how to do rollback, what are generations etc. etc. ... I did some presentations on NixOS and these topics so I'd like to distill that into a short and easy to grasp blog posts @ https://nixos.online https://nixos.online - feedback is more than welcome! Maybe I could also include my own cheatsheet, which is refurbished version of the "official" one, but just for the basic commands in direct comparison to Debian-like distros.
- aaronchall 6y agoA straight-up cheatsheet would be useful. I didn't find the direct comparison to apt to be useful before installing nixos, to be honest, but as primarily a `nix-shell --pure` user who just got nixos installed on my server, I suppose I might revisit it...
- PaulDavisThe1st 6y agoJust a minor footnote. My interest in Nix is not high and what there is motivated mostly by the existence of NixOS. Why do I have any interest in NixOS? We build and distribute Ardour (a cross-platform open source DAW) in a single package that runs on every Linux distribution that has libc, libstdc++, and some version of Xlib. Every Linux distribution that is, except NixOS. They decided to patch the runtime linker so that the approach usable on every other Linux distro breaks there. This says something about NixOS, and presumably Nix too, though I'm not sure what.
- aidenn0 6y agoI didn't think NixOS patched the runtime linker (though I could definitely be wrong). They do eliminate any ability to automatically find libraries, and the dynamic loader does not exist at the path where it exists on most other distros, which means binaries linked elsewhere will not run unmodified on nixos.
- danieldk 6y agoTo expand a bit on this for people not familiar with Nix: the dynamic linker is treated as any other dependency and therefore stored in the Nix store, using the derivation's hash as part of the store path. The binary's .interp section indicates which dynamic linker must be used. On a regular Linux distribution, you will find that this is a global path: $ readelf -p .interp $(which cp) String dump of section '.interp': [ 0] /lib64/ld-linux-x86-64.so.2 No global library paths such as /lib or /lib64 exists on NixOS. This is one of the reasons why binaries compiled on other Linux distributions, such as Ubuntu, do not work out of the box. And you have to use patchelf to change the dynamic loader's path. Instead, the dynamic loader of the glibc version that a program was built against is used: $ readelf -p .interp $(which cp) String dump of section '.interp': [ 0] /nix/store/jx19wa4xlh9n4324xdl9rjnykd19mmq3-glibc-2.30/lib/ld-linux-x86-64.so.2
- solarkraft 6y agoWhy aren't there symlinks for things standard linux directories, a bit like current-system in the user environment? Why go through the effort of patching the actual binary? Doesn't that cause enormous problems with signed stuff? Edit: I assume you can't do it very well system-wide because individual packages might need conflicting versions. Okay. Has something like runtime injection been considered? What are the trade-offs here? (I think it's fairly obvious that I'm not very familiar with the whole matter yet).
- iElectric2 6y agoI've started writing tutorials at https://nix.dev/ https://nix.dev/ to help people set up their development environment and get up and running quickly. Disclaimer: the owner of https://cachix.org https://cachix.org
- iElectric2 6y agoFeel free to request a tutorial athttps://github.com/domenkozar/nix.dev/issues https://github.com/domenkozar/nix.dev/issues