10 ms·

Show HN: Brioche – A new Nix-like package manager

168 points by kylewlacy 2y ago

This is a project I've wanted to write for a long time now. I really love the ideas from Nix and I still have a ton of respect for the project, but Nix-the-language never felt intuitive to me and I wanted something with more approachable tooling (although this was circa 2016, so I'm sure Nix has improved a lot since then too-- that was before Flakes were around!) Anyway, I started on the current iteration of Brioche about 6 months ago, and I finally cut an initial release. I'd still consider this a "technical preview" version (performance especially is pretty painful, so that'll be a focus of mine in the coming weeks). But it's finally at a point where it does work end-to-end and folks can take it for a test drive!

tombert 2y ago
Flakes definitely help with the giant megarepo annoyances of NixOS, though they're still a little irritating. If you are writing in a languages that doesn't hasn't had its packages directly integrated into the build system (like Python's has), it can be really irritating to do anything with them, since the `nix build` command disables network access and you therefore cannot use regular package manager. I'm doing a project in Julia, and I'm using Nix Flakes to do it, but it's been immensely annoying to actually get that working [1]. As a result, I've had avoid using the `nix build` command entirely (though the flakes are still useful for the `nix develop` command). All that said, do you plan on having Brioche work with reproducible builds, and if so do you have a plan to make what I mentioned a bit less irritating? [1] I know Julia2Nix exists, and I have never managed to actually get that working on any platform.
- kylewlacy 2y ago
  This was one of my bigger pain points with Nix as well: there was a lot of "reinventing the world" just to avoid network access. With Brioche, I stuck with disabling network access by default, but there's an escape hatch to specifically opt-in to networking (by calling `.unsafe({ networking: true })` on a process recipe). My thoughts are that Cargo, NPM, Poetry, etc. have all done a great job building amazing tooling to download and verify resources from the network against a lockfile, and I wanted to be able to leverage as much of that as possible. So, for example, `npm clean-install` will give you more-or-less the same guarantees that Nix does, so my thought was as long as it's encapsulated properly, I'd rather lean on existing tooling (that's also why I used the term "unsafe", you need to make sure you only access the network with tools that do sufficient verification!) I've generally stayed away from using the term "reproducible build" when talking about Brioche, because I don't feel like it fits the reproducible-builds.org definition (though I don't think Nix does either). But, if a build is cached locally or in the registry, then you're guaranteed to get the same result, since it'll just re-use the cache The sandboxing also gives pretty strong guarantees around hermetic builds[1]. So I think you could do reproducible builds _within_ Brioche (and I'd like to add tools to make this even easier), but I'd say Brioche itself doesn't give you reproducible builds out of the box [1]: https://bazel.build/basics/hermeticity https://bazel.build/basics/hermeticity
  - fire_lake 2y ago
    > have all done a great job building amazing tooling to download and verify resources from the network against a lockfile This is a bad assumption
    - IshKebab 2y ago
      Why? Are you referring to the fact that it's opt-in; you have to use `cargo build --locked` or `npm ci`?
      - bavarianbob 2y ago
        I interpreted the parent to mean that it might not be fair to assume existing package managers have done a _great_ job at downloading and verifying, especially verifying, resources from the network. There are businesses that exist attempting to solve this problem like socket.io. Safely installing the correct dependencies for a project is still not a guarantee from any of the major package managers.
    - YuukiRey 2y ago
      If the underlying package eco system is a mess then it’ll also be a mess in Nix. To this day Nix doesn’t really have a good answer to Python or JavaScript packaging. If the underlying package manager is good (Rust, Go) then there’s just no need to reinvent the wheel.
  - tombert 2y ago
    That's actually really great to hear! I might need to play with this tonight then. It looks like the project files aren't radically dissimilar to Flakes, so I think you're really on the right track for making something that could be really useful for a lot of people. Great work!
- forgotpwd16 2y ago
  >`nix build` command disables network access Only if building with sandbox enabled. Can disable it if network access is required. Seems someone opened an issue asking[1] for granular permissions (explicit network restriction) but has been marked as stale. In same issue someone else has made comment providing an hybrid approach. Since you mentioned Julia, it's possible to build Julia environments (with arbitrary packages) using the `.withPackages` function. E.g. `julia.withPackages ["Plots"]`. [1]: https://github.com/NixOS/nix/issues/4584 https://github.com/NixOS/nix/issues/4584
  - tombert 2y ago
    I didn’t realize that. I will give that a look.
    - Filligree 2y ago
      Instead of disabling it completely, you can set it to 'relaxed'. That still allows network access, but maintains the other isolation guarantees. Obviously this introduces a potential impurity. I use it for installing NeoForge into a minecraft server derivation; the downloaded files could change behind my back, but so long as the version doesn't change it should still be compatible. It hasn't yet caused trouble. NixOS has a lot of escape hatches once you look around, and you shouldn't be afraid to use them. The downside of impure derivations aren't quite as bad as not-using-derivations, anyway. Another option is to declare it as a fixed-output derivation, in which case network access is enabled by default. This doesn't work for most installers, though; at a minimum you'll need to delete logfiles that might contain timestamps.
- Rucadi 2y ago
  If you mark a derivation as impure with __impure=true; and activate "ca-derivations flakes nix-command" as experimental features, you can access the internet. I agree sometimes is a pain tho. But most of the time, If you need internet access, if you can specify the "hash" of the result file, anything that you do to achieve that file can have internet access.
Aerbil313 2y ago
Beat me to it! It's my dream to write a Nix replacement in a more approachable language. I can't state enough how the choice of a bespoke DSL that doesn't immediately make sense to most developers keeps Nix/Nixos community from growing. And Nix is 18 years old now, it's long overdue for a rewrite[1]. Don't implement channels to start with. Focus on full reproducibility. (...getting into sci-fi territory...) Build in QEMU and KVM virtualization to be able to build and run most pieces of software ever published. No, Docker is a horribly leaky abstraction. Run a torrent network by default as cache.nixos.org equivalent, because you can't keep up with the demand with centralized solutions, even nixpkgs gave up on it despite their free credits on AWS. 1: I believe all software without exception needs a full rewrite at least every 10 years.
- tombert 2y ago
  I think Flakes kind of help with the centralized problems. At least individual flakes can be run or installed separately, they don't have to be merged into the core nixpkgs repo, and they (at least in theory) should be giving the same build reproducibility as you'd get from installing directly from nixpkgs. I agree with the torrent idea though.
phlip9 2y ago
Congrats on the release! I love the focus on devex w/ typescript and autocomplete. That's probably one of my biggest pain points with Nix -- writing any non-trivial package always requires a ripgrep adventure through nixpkgs. Finding the right poorly documented and poorly discoverable derivation attributes is always such a chore. What are your plans for cross-compilation or heavy package customization? One of nixpkgs coolest party tricks imo is that you can just change the stdenv and get a musl static binary or cross-compiled binary.
- kylewlacy 2y ago
  > What are your plans for cross-compilation or heavy package customization? One of nixpkgs coolest party tricks imo is that you can just change the stdenv and get a musl static binary or cross-compiled binary. So in general, I don't think I'm going to have anything quite as powerful as Nix's overrides. But I'm hoping most of the use-cases for it will be covered by some simpler options: - Since build definitions are functions, package authors can just take arguments for the things they want downstream users to be able to customize (e.g. `configure` flags, optional features and plugins, etc.) - I haven't built it yet, but I think adding support for dependency overrides would be fairly easy, a la Cargo. Basically, you'd just fork or clone the package you want to tweak, make your tweaks, then set an "overrides" option to use it instead. I know that's not a super satisfying answer, but that should help cover a lot of use cases - For toolchains specifically, I have an idea in mind for this as well (also not implemented at all). At a high level, the idea is that packages could use "dynamic bindings", which you can then override for downstream recipes (this would require some new runtime features in Brioche itself). The toolchain itself would effectively be a dynamic binding, letting you pick a different recipe (so you could swap glibc for musl, or gcc for clang, etc). Cross-compilation would also be built on this same feature
charlotte-fyi 2y ago
Very exciting. The ideas behind Nix are so good and everything else so bad. It seems like a lot of people are trying to solve this by building abstractions on top of Nix, but I'm really skeptical that is the solution. As crazy as a full rewrite is, I hope someone succeeds!
- tombert 2y ago
  Agreed. NixOS is a marvel of engineering to me, and kind of hard to go back from once you get used to it. Automatic snapshotting on every configuration change, the entire system state being configurable through text files and therefore never being ambiguous, being able to temporarily install stuff without it polluting your path for forever by using nix-shells, clearly being able to see and define stuff like boot parameters and kernel modules are just insanely wonderful things, all while still using (I think) a vanilla kernel and really no runtime overhead, allowing you to make an insanely lean system without ever being unsure if you're missing something or having to worry about extra crap that you installed by accident lingering forever. In my mind about as close to an "objectively better" way to handle an OS (at least for people who are technical) as one can ask for. I have no desire to go back to any other distro for my server. But the Nix language itself is really quite annoying. I mean, I've more or less gotten used to its annoyances, and I do think that some of the DSLs it has are excellent (I really like the Nginx and systemd configuration stuff, for example), and a lot of the configs are just `services.myservice.enable = true` which is fine, but a lot of the time I'm kind of confused about what syntax is allowed and how loops work and the like. It's not horrible or anything, just a bit annoying because I'll occasionally have to do a nixos-rebuild like three or four times because I messed up some subtle syntax, and it's especially annoying if I have to go dig at the root Nix package to find out what I did wrong [1]. I think decentralizing stuff in the form of flakes might be able to help with this, if for no other reason the area in which you'd be forced to look for configuration stuff could be reduced, but I do think NixOS would benefit from some rearchitecture. [1] Which happened yesterday with an ethernet card configuration: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/nixos/modules/system/boot/networkd.nix https://github.com/NixOS/nixpkgs/blob/nixos-24.05/nixos/modu...
- whateveracct 2y ago
  ever use nix repl? i found my complaints about grokkability disappeared once i realized i could easily introspect my build (and all my dependencies' builds) at any level. that + learning the like 5 or so idioms that pervade nixpkgs and you can use Nix quite successfully imo.
andrewla 2y ago
I'm very excited by all the attempts to replace Nix, but I don't think I'll be exploring this much deeper. In my opinion the issue with Nix is that the data model is not crisply defined -- it's there, but hidden under a lot of goop that is the Nix language itself and the various assumptions and baggage that goes with it. What I want is a primarily declarative syntax supporting a rich set of data structures, ideally a non-Turing set of primitives, with a much more intuitive way of gluing things together. So basically bash (or even sh) with a well-defined way of transmitting environment variables and setting up the environment. The idea of importing a language that has broader support (typescript) as an alternative to the Nix language seems appealing at first, but typescript is such a high-dependency system that it's hard to get excited about it.
- infogulch 2y ago
  Agreed on all counts, especially the central issue with nix and the properties that I'd want out of a replacement. I think CUE ( https://cuelang.org/ https://cuelang.org/ ) is a perfect language for this.
  - nine_k 2y ago
    Cue is fun, but already the unification example (https://cuelang.org/docs/tour/basics/duplicate-fields/ https://cuelang.org/docs/tour/basics/duplicate-fields/) shouts "footguns" to me :( TS is certainly excessively powerful though.
    - infogulch 2y ago
      I don't think this would be a problem in practice. By default structs are "open" in cue, i.e. as long as matching fields unify, disparate fields just merge. There are also "closed" structs that allow creating a definition that fails to unify with fields not explicitly listed. https://cuelang.org/docs/tour/types/closed/ https://cuelang.org/docs/tour/types/closed/
    - from-nibly 2y ago
      What foot guns are you seeing specifically?
zamalek 2y ago
I'm also exploring a post-Nix package manager, wip: https://github.com/porkg/porkg https://github.com/porkg/porkg My mantra has been to avoid "getting bored" and inventing DSLs for the longest time. I initially sought to use Nickel-lang, but it was missing some features that would make it an ideal candidate for this. I started writing my own (you may find this in the history) before realizing "WTF are you doing writing another shitty DSL?" I have subsequently decided that shell scripts (or anything you can shebang) are good enough, i.e. pkgbuild inspiration. I also plan to avoid making a derivation the source of reproducibility. Instead, a lockfile will offer that. This should alleviate the issue whereby updating the like of glibc cascades into an entire rebuild. Any *OS and home-manager would need to bring in a configuration language. I think Cue really makes the most sense, but that's still a long way off. Nice to see you used JS instead of yet another DSL :)
- Aerbil313 2y ago
  I suggest the post author and you look into nu as the package build language. It's uniquely suited for this task, being a cross-platform shell[1] with real programming language features, data types, and using built-in uutils instead of relying on platform-dependent coreutils. It also limits mutability[2], favoring a more functional than imperative approach, which would at least theoretically help with reproducibility. A guy from Determinate Systems recently made an experiment on integrating nushell to nix and the results were positive. Take a look: https://determinate.systems/posts/nuenv/ https://determinate.systems/posts/nuenv/ 1: https://www.nushell.sh/ https://www.nushell.sh/ 2: https://www.nushell.sh/book/thinking_in_nu.html#variables-are-immutable https://www.nushell.sh/book/thinking_in_nu.html#variables-ar...
  - kylewlacy 2y ago
    Nushell has definitely been on my radar for a while! but I missed DetSys was playing around with it, I'll have to give that a look One neat thing about Brioche is that Bash isn't really baked into it the same way as the Nix ecosystem. In a lot of the docs and examples, I used the `std.runBash` Brioche function for running scripts, but that's a fairly simple 11-line function[^1]. I wanted to make it just as easy to add `runZsh`, `runFish`, `runNushell`, etc. functions that work just as well as Bash (and they could either be put in the std library or in their own packages). So hopefully, there will be a Brioche `runNushell` function in the near future :) [^1]: https://github.com/brioche-dev/brioche-packages/blob/9fd51090a56cf83f4aa9c5002b78f2b2054ad3ad/packages/std/extra/run_bash.bri#L37-L47 https://github.com/brioche-dev/brioche-packages/blob/9fd5109...
__MatrixMan__ 2y ago
Some of the oddities of the nix language are pretty useful for its domain. Recursive attribute sets, for instance, save a lot of headaches if you're trying to have only a single source of truth. Do you feel like these translate to typescript nicely? As somebody who knows nix but doesn't know typescript, I found myself looking for a rosetta stone page where I could look at two chunks of code that do the same thing, but in separate languages. This would let me use the familiar language to scrutinize the other. I wouldn't normally ask for such a thing, but if you're putting "Nix-like" in the title then maybe it might be worth adding a comparison page to the docs.
- charlotte-fyi 2y ago
  The syntax isn’t as clean but you can add property accessors to objects in JavaScript that accomplishes the same thing.
- kylewlacy 2y ago
  I've been asked about overrides and attributes a lot! That was one of the sacrifices I had to make to go with a more traditional language, so it's definitely a negative point compared to Nix. That said, I'm hoping to have some conventions and features that will cover _most_ of the use-cases that overrides give you, but that's definitely still future work at this point and I probably won't be able to cover everything overrides do And yep, I think having a "Brioche for Nix users" guide makes a lot of sense, although it's not the first time that question so I'll probably stand up a first-pass version of it sooner rather than later (my Nix skills are also pretty rusty-- I'll need to brush up a bit first before I write it!)
- hamandcheese 2y ago
  I think the lack of true laziness will be a big performance problem for a large build graph. On the other hand the monolithic nature of the nixpkgs package set is one of the authors gripes with nix, so performance at that scale may be a non-goal.
  - kylewlacy 2y ago
    I'd definitely like to have good performance even for large build graphs! I'm hoping the laziness exists "where it counts". To walk through an example, if you build your backend, and your backend calls the function `postgres()`, and that calls `openssl()`, and THAT calls `gcc()`, etc., etc., each function is basically building an object to represent its chunk of the build graph (each function returns a "recipe"). Nothing gets built until that object gets returned from the top-level function and the runtime does something with it In other words, the eager part is basically constructing the build graph. Maybe I'm wrong but I don't that this would necessarily be slower than the lazy version. In practice the most complex build graph I've made is basically the full chain of Linux From Scratch builds (that's the basis for my toolchain currently), and I think that takes about 400-500ms to evaluate. It's about 160 build steps, so it's not _simple_ but I know build graphs can also get a lot more complex, so I'll just have to keep an eye on performance as I start to get into more and more complex builds Maybe I'm missing something but intuitively I'd expect this approach to be fairly efficient-- as long as build scripts only call these functions when they're used as part of the build graph
leoh 2y ago
I am not writing typescript -- heaven forbid -- so I can manage cargo, so I can write Rust. Dropping the snark, though -- most times I have seen folks attempt to fold other build systems into some polyglot common one, it seems to create a lot of problems -- for at the end of the day, build tools (whether npm, cargo, gradle, docker, etc.) each have their issues. When these issues arise, they require understanding the original build tool. Bazel, for example, is a "polyglot" system which many try to use to manage npm. In my experience, when folks use bazel but have never used npm directly, they often get lost, because they now have to deal with two abstractions (bazel, npm) simultaneously neither have a way for bazel to do the thing they need, nor an understanding of how to get bazel to do that thing (nor npm itself to do the thing), also because they have never worked with npm directly. Same goes for cargo, incidentally. Switching over to that build tool -- like any major build tool, it has an insane amount of well-honed documentation and thousands of answered questions online (not to mention LLMs trained somewhat well on it). Users when they need to do something with brioche or have an issue with it will not necessarily have the benefit of the massive collaborative support available for cargo. Incidentally, I don't think that nix is free of this problem, though it does attempt to manage these issues -- sometimes reasonably successfully -- by virtue of keeping interfaces to other build tools in simple, direct ways. Even then, however, the best way I have seen nix used is as a system for declaring all binary dependencies or other dependencies that are either not manageable or especially competently managed by other build tools -- which nix allows for in a virtual environment by shipping a shell.nix or flake. In other words -- bringing in the right version of java, gradle, node, etc. But then just directing users to the specific, native build tools that often have a massive number of contributors. The closest thing to a tool that does this (guides developers to other build tools) is flox, which uses TOML, but uses nix for its backend via C API. I haven't had a chance to use it much yet, but it looks very promising https://flox.dev https://flox.dev. === Okay, all this rambling aside -- building a tool like this for yourself or a small team, especially when you have the time to do it, it works well, and you're having fun... well, there's an insane amount to be said for that; and it will doubtlessly lead you to learning a ton about other build tools and likely offer you deep insights that are only possible when attempting to build something like brioche. So, on that count, major kudos to you.
Zambyte 2y ago
I'm really happy to see more work on package managers in this space. I used Nix for a little bit, but I shared the frustration expressed by other commenters on the language itself. I have found my zen on GNU Guix, but more options are better :)
- notthemessiah 2y ago
  Tvix aims to be a modulary implementation of Nix, which could create a bridge between Nix and Guix, according to a claim by its author: "It should be possible to write an evaluator that plugs in the Guile language (for compatibility with GNU Guix), to use arbitrary builders, and to replace the store implementation." https://tvix.dev/ https://tvix.dev/
zarathustreal 2y ago
I’ve generally found the most interesting things are the ones that don’t feel intuitive at first. Familiarity bias can really hold you back. The language semantics of Nix are basically Haskell’s which.. is probably the One True Way to write correct code if we’re honest with ourselves.
- hexo 2y ago
  nix does not have haskells semantics
  - zarathustreal 2y ago
    Yes it does? Purely functional, lazily evaluated.. declarative emphasis. Yea all it’s missing is strong typing, which is why I said it’s similar, not the same.
- davidalayachew 2y ago
  I strongly disagree. Haskell is great in a pure world, but it has an undue amount of friction when in the IO world. It does work in the IO world just fine, but languages like Java handle the actual concerns of the IO world better than Haskell. And for that reason, I say that no, Haskell is not the one true way to write correct code. It definitely is the One True Way to write happy path code though.
raffraffraff 2y ago
Reading the comments makes me think I'm turning into my dad. He was a simple man in terms of education and career. He was a fire man on a stream train in 1950, a farmer, barman, handyman. Out in the sticks, he was everyone's go-to guy when a vet couldn't be found, because he kept a "livestock first aid kit" including penicillin & syringes. He talked to vets and read books and knew about the most common ailments and how to treat them. He serviced his own car and tractor, including stripping the tractor engine down in his yard without proper tools, and getting cylinders bored and pistons sleeved, and putting it back together. He ran electric cable in farm building and sheds (after chatting to an electrician) and he did a ton of building work around the house. But at a certain age, maybe around 50, all new technology was out. Electricity was fine but electronics were out. Mechanical problems were fine because you could look straight at a thing and figure out how it clicked together, but once it had a microcontroller and a panel that abstracted away the actual working of the machine it just became some weird arbitrary "thing" that blocked his view of the machine. I'm sure if he was in his 20s when this stuff appeared he'd be all over it, but he was born in 1926, so I think the mighty micro was too late for him. He flatly refused to use the VCR, alarm systems, any type of computer. Show him a rubis cube, he's interested. Show him a microwave oven with more than 5 buttons and he'll make a sandwich instead. This is me with nix. I'm ok with all the stuff that's emerged in my field (infra/sysadmin) since the 90s. But nix makes me instantly glaze over. Several times I tried, but it's like my brain unplugs itself. I'm sure it does some wonderful stuff at an extremely high price that I'm not willing to pay. Nix has a huge mindshare, which I don't think you can win back if Nix keeps fixing the core issues mentored in these comments. Best of luck with this, though.
- otabdeveloper4 2y ago
  Nix solves the same problem we used to solve back in the day (before Docker taught us to run everything as root and disable firewalls, #yolo) ad-hoc with random poorly maintained scripts. If you never had to solve this problem then it probably isn't for you. If you did, then being able to finally replace the old contraptions with something properly engineered is a breath of fresh air.
  - Dalewyn 2y ago
tkz1312 2y ago
Looks awesome! Congrats on the release. I would be curious to hear your thoughts on how this compares to garn [1], a typescript front end to the existing nixpkgs ecosystem. 1: https://garn.io/ https://garn.io/
- kylewlacy 2y ago
  I think garn is really cool! If I were still in the Nix ecosystem, garn would probably be my go-to Building something on Nix obviously has a ton of benefits: well-proven tooling, lots of mindshare already, TONS of packages. Everyone knows that Nix-the-language is fairly unintuitive for beginners, and tools like garn, Devenv, Devbox, etc. all seem to be trying to let folks use Nix without exposing them to Nix-the-language. I think this is a good goal to have and helps make Nix more accessible But I really thought it was worth exploring the Nix-free design space too. For example, I think derivations in Nix are... kinda complicated and weird! Having a fixed set of phases specifically designed for autotools builds always seemed a little weird to me. By ditching Nix, I was able to see what the world looks like when you ditch the idea of phases completely, for example Another interesting one was ditching the absolute path for the store path. Getting everything working by resolving relative paths was surprisingly tricky, but the end result is that you don't need to be root to install Brioche, and you can also take a Brioche-built binary (along with the resource dir it outputs) and ship it to another Linux machine and it'll just work. That's only something you can do if you see what happens if you explore outside the Nix space So I'm hoping Brioche can _at least_ be an interesting case study in how you can take a lot of Nix's ideas and get a lot of the same value without having to take Nix's approach directly (or who knows-- maybe some of those ideas end up back in Nix someday!)
doctor_phil 2y ago
I actually haven't tried Nix (or other nix-alternatives) yet. I've seen a lot of complaints about the language, but a lot of praise and love of the idea. I though Guix was the "main" competitor to nix, using a more mainstream language (Scheme). Is that still the case? How would Brioche compare against Guix if you are OK with both Scheme and Typescript as languages?
- rgoulter 2y ago
  > I've seen a lot of complaints about the language, but a lot of praise and love of the idea I think when people say they like the idea of nix, they're referring to how it builds packages from programmatic package formulae into an isolated store (so e.g. you can easily have different programs running against different versions of dependencies), and the use cases this allows for. -- With an expressive language for describing packaging, you get stuff like declarative developer environments, or even declared Linux configuration with quick/easy rollback. I think complaints about the language are partly from Nix being unusual enough that it takes time to get an intuition for the basics of how to write anything in it, but largely from the Nixpkgs (the large package repository the Nix package manager uses) being a large codebase, which grew organically with many contributors, with many similar-but-different solutions to problems, and sometimes the codebase is a bit too clever.
- cmrdporcupine 2y ago
  Guix is the main competitor but it's also focused on free software so mainly excludes things that don't fit that licensing philosophy. Yes there are I believe repositories that maintain non-free things but ... Nix already doesn't have declarations for a lot of things. Guix even less so.
Gud 2y ago
I’ve always wondered why no one ports FreeBSDs pkg/ports package management system. From all the operating systems I’ve used it’s the clear leader.
- deleted 2y ago
  [deleted]
- justinsaccount 2y ago
  https://pkgsrc.org/ https://pkgsrc.org/ from netbsd runs on many systems
amelius 2y ago
Can't we have something Python based, since it is already the lingua franca and seems to be the language that people are least frustrated about?
hisamafahri 2y ago
Wow, congrats for the release!!! But, can we revisit the name? “Brioche” sounds cool but (at least for me) it’s so awkward to type on the terminal and pronounce

168 points by kylewlacy 2y ago

tombert 2y ago

Flakes definitely help with the giant megarepo annoyances of NixOS, though they're still a little irritating. If you are writing in a languages that doesn't hasn't had its packages directly integrated into the build system (like Python's has), it can be really irritating to do anything with them, since the `nix build` command disables network access and you therefore cannot use regular package manager. I'm doing a project in Julia, and I'm using Nix Flakes to do it, but it's been immensely annoying to actually get that working [1]. As a result, I've had avoid using the `nix build` command entirely (though the flakes are still useful for the `nix develop` command). All that said, do you plan on having Brioche work with reproducible builds, and if so do you have a plan to make what I mentioned a bit less irritating? [1] I know Julia2Nix exists, and I have never managed to actually get that working on any platform.

kylewlacy 2y ago
This was one of my bigger pain points with Nix as well: there was a lot of "reinventing the world" just to avoid network access. With Brioche, I stuck with disabling network access by default, but there's an escape hatch to specifically opt-in to networking (by calling `.unsafe({ networking: true })` on a process recipe). My thoughts are that Cargo, NPM, Poetry, etc. have all done a great job building amazing tooling to download and verify resources from the network against a lockfile, and I wanted to be able to leverage as much of that as possible. So, for example, `npm clean-install` will give you more-or-less the same guarantees that Nix does, so my thought was as long as it's encapsulated properly, I'd rather lean on existing tooling (that's also why I used the term "unsafe", you need to make sure you only access the network with tools that do sufficient verification!) I've generally stayed away from using the term "reproducible build" when talking about Brioche, because I don't feel like it fits the reproducible-builds.org definition (though I don't think Nix does either). But, if a build is cached locally or in the registry, then you're guaranteed to get the same result, since it'll just re-use the cache The sandboxing also gives pretty strong guarantees around hermetic builds[1]. So I think you could do reproducible builds _within_ Brioche (and I'd like to add tools to make this even easier), but I'd say Brioche itself doesn't give you reproducible builds out of the box [1]: https://bazel.build/basics/hermeticity https://bazel.build/basics/hermeticity
- fire_lake 2y ago
  > have all done a great job building amazing tooling to download and verify resources from the network against a lockfile This is a bad assumption
  - IshKebab 2y ago
    Why? Are you referring to the fact that it's opt-in; you have to use `cargo build --locked` or `npm ci`?
    - bavarianbob 2y ago
      I interpreted the parent to mean that it might not be fair to assume existing package managers have done a _great_ job at downloading and verifying, especially verifying, resources from the network. There are businesses that exist attempting to solve this problem like socket.io. Safely installing the correct dependencies for a project is still not a guarantee from any of the major package managers.
  - YuukiRey 2y ago
    If the underlying package eco system is a mess then it’ll also be a mess in Nix. To this day Nix doesn’t really have a good answer to Python or JavaScript packaging. If the underlying package manager is good (Rust, Go) then there’s just no need to reinvent the wheel.
- tombert 2y ago
  That's actually really great to hear! I might need to play with this tonight then. It looks like the project files aren't radically dissimilar to Flakes, so I think you're really on the right track for making something that could be really useful for a lot of people. Great work!
forgotpwd16 2y ago
>`nix build` command disables network access Only if building with sandbox enabled. Can disable it if network access is required. Seems someone opened an issue asking[1] for granular permissions (explicit network restriction) but has been marked as stale. In same issue someone else has made comment providing an hybrid approach. Since you mentioned Julia, it's possible to build Julia environments (with arbitrary packages) using the `.withPackages` function. E.g. `julia.withPackages ["Plots"]`. [1]: https://github.com/NixOS/nix/issues/4584 https://github.com/NixOS/nix/issues/4584
- tombert 2y ago
  I didn’t realize that. I will give that a look.
  - Filligree 2y ago
    Instead of disabling it completely, you can set it to 'relaxed'. That still allows network access, but maintains the other isolation guarantees. Obviously this introduces a potential impurity. I use it for installing NeoForge into a minecraft server derivation; the downloaded files could change behind my back, but so long as the version doesn't change it should still be compatible. It hasn't yet caused trouble. NixOS has a lot of escape hatches once you look around, and you shouldn't be afraid to use them. The downside of impure derivations aren't quite as bad as not-using-derivations, anyway. Another option is to declare it as a fixed-output derivation, in which case network access is enabled by default. This doesn't work for most installers, though; at a minimum you'll need to delete logfiles that might contain timestamps.
Rucadi 2y ago
If you mark a derivation as impure with __impure=true; and activate "ca-derivations flakes nix-command" as experimental features, you can access the internet. I agree sometimes is a pain tho. But most of the time, If you need internet access, if you can specify the "hash" of the result file, anything that you do to achieve that file can have internet access.

Aerbil313 2y ago

Beat me to it! It's my dream to write a Nix replacement in a more approachable language. I can't state enough how the choice of a bespoke DSL that doesn't immediately make sense to most developers keeps Nix/Nixos community from growing. And Nix is 18 years old now, it's long overdue for a rewrite[1]. Don't implement channels to start with. Focus on full reproducibility. (...getting into sci-fi territory...) Build in QEMU and KVM virtualization to be able to build and run most pieces of software ever published. No, Docker is a horribly leaky abstraction. Run a torrent network by default as cache.nixos.org equivalent, because you can't keep up with the demand with centralized solutions, even nixpkgs gave up on it despite their free credits on AWS. 1: I believe all software without exception needs a full rewrite at least every 10 years.

tombert 2y ago
I think Flakes kind of help with the centralized problems. At least individual flakes can be run or installed separately, they don't have to be merged into the core nixpkgs repo, and they (at least in theory) should be giving the same build reproducibility as you'd get from installing directly from nixpkgs. I agree with the torrent idea though.

phlip9 2y ago

Congrats on the release! I love the focus on devex w/ typescript and autocomplete. That's probably one of my biggest pain points with Nix -- writing any non-trivial package always requires a ripgrep adventure through nixpkgs. Finding the right poorly documented and poorly discoverable derivation attributes is always such a chore. What are your plans for cross-compilation or heavy package customization? One of nixpkgs coolest party tricks imo is that you can just change the stdenv and get a musl static binary or cross-compiled binary.

kylewlacy 2y ago
> What are your plans for cross-compilation or heavy package customization? One of nixpkgs coolest party tricks imo is that you can just change the stdenv and get a musl static binary or cross-compiled binary. So in general, I don't think I'm going to have anything quite as powerful as Nix's overrides. But I'm hoping most of the use-cases for it will be covered by some simpler options: - Since build definitions are functions, package authors can just take arguments for the things they want downstream users to be able to customize (e.g. `configure` flags, optional features and plugins, etc.) - I haven't built it yet, but I think adding support for dependency overrides would be fairly easy, a la Cargo. Basically, you'd just fork or clone the package you want to tweak, make your tweaks, then set an "overrides" option to use it instead. I know that's not a super satisfying answer, but that should help cover a lot of use cases - For toolchains specifically, I have an idea in mind for this as well (also not implemented at all). At a high level, the idea is that packages could use "dynamic bindings", which you can then override for downstream recipes (this would require some new runtime features in Brioche itself). The toolchain itself would effectively be a dynamic binding, letting you pick a different recipe (so you could swap glibc for musl, or gcc for clang, etc). Cross-compilation would also be built on this same feature

charlotte-fyi 2y ago

Very exciting. The ideas behind Nix are so good and everything else so bad. It seems like a lot of people are trying to solve this by building abstractions on top of Nix, but I'm really skeptical that is the solution. As crazy as a full rewrite is, I hope someone succeeds!

tombert 2y ago
Agreed. NixOS is a marvel of engineering to me, and kind of hard to go back from once you get used to it. Automatic snapshotting on every configuration change, the entire system state being configurable through text files and therefore never being ambiguous, being able to temporarily install stuff without it polluting your path for forever by using nix-shells, clearly being able to see and define stuff like boot parameters and kernel modules are just insanely wonderful things, all while still using (I think) a vanilla kernel and really no runtime overhead, allowing you to make an insanely lean system without ever being unsure if you're missing something or having to worry about extra crap that you installed by accident lingering forever. In my mind about as close to an "objectively better" way to handle an OS (at least for people who are technical) as one can ask for. I have no desire to go back to any other distro for my server. But the Nix language itself is really quite annoying. I mean, I've more or less gotten used to its annoyances, and I do think that some of the DSLs it has are excellent (I really like the Nginx and systemd configuration stuff, for example), and a lot of the configs are just `services.myservice.enable = true` which is fine, but a lot of the time I'm kind of confused about what syntax is allowed and how loops work and the like. It's not horrible or anything, just a bit annoying because I'll occasionally have to do a nixos-rebuild like three or four times because I messed up some subtle syntax, and it's especially annoying if I have to go dig at the root Nix package to find out what I did wrong [1]. I think decentralizing stuff in the form of flakes might be able to help with this, if for no other reason the area in which you'd be forced to look for configuration stuff could be reduced, but I do think NixOS would benefit from some rearchitecture. [1] Which happened yesterday with an ethernet card configuration: https://github.com/NixOS/nixpkgs/blob/nixos-24.05/nixos/modules/system/boot/networkd.nix https://github.com/NixOS/nixpkgs/blob/nixos-24.05/nixos/modu...
whateveracct 2y ago
ever use nix repl? i found my complaints about grokkability disappeared once i realized i could easily introspect my build (and all my dependencies' builds) at any level. that + learning the like 5 or so idioms that pervade nixpkgs and you can use Nix quite successfully imo.

andrewla 2y ago

I'm very excited by all the attempts to replace Nix, but I don't think I'll be exploring this much deeper. In my opinion the issue with Nix is that the data model is not crisply defined -- it's there, but hidden under a lot of goop that is the Nix language itself and the various assumptions and baggage that goes with it. What I want is a primarily declarative syntax supporting a rich set of data structures, ideally a non-Turing set of primitives, with a much more intuitive way of gluing things together. So basically bash (or even sh) with a well-defined way of transmitting environment variables and setting up the environment. The idea of importing a language that has broader support (typescript) as an alternative to the Nix language seems appealing at first, but typescript is such a high-dependency system that it's hard to get excited about it.

infogulch 2y ago
Agreed on all counts, especially the central issue with nix and the properties that I'd want out of a replacement. I think CUE ( https://cuelang.org/ https://cuelang.org/ ) is a perfect language for this.
- nine_k 2y ago
  Cue is fun, but already the unification example (https://cuelang.org/docs/tour/basics/duplicate-fields/ https://cuelang.org/docs/tour/basics/duplicate-fields/) shouts "footguns" to me :( TS is certainly excessively powerful though.
  - infogulch 2y ago
    I don't think this would be a problem in practice. By default structs are "open" in cue, i.e. as long as matching fields unify, disparate fields just merge. There are also "closed" structs that allow creating a definition that fails to unify with fields not explicitly listed. https://cuelang.org/docs/tour/types/closed/ https://cuelang.org/docs/tour/types/closed/
  - from-nibly 2y ago
    What foot guns are you seeing specifically?

zamalek 2y ago

I'm also exploring a post-Nix package manager, wip: https://github.com/porkg/porkg https://github.com/porkg/porkg My mantra has been to avoid "getting bored" and inventing DSLs for the longest time. I initially sought to use Nickel-lang, but it was missing some features that would make it an ideal candidate for this. I started writing my own (you may find this in the history) before realizing "WTF are you doing writing another shitty DSL?" I have subsequently decided that shell scripts (or anything you can shebang) are good enough, i.e. pkgbuild inspiration. I also plan to avoid making a derivation the source of reproducibility. Instead, a lockfile will offer that. This should alleviate the issue whereby updating the like of glibc cascades into an entire rebuild. Any *OS and home-manager would need to bring in a configuration language. I think Cue really makes the most sense, but that's still a long way off. Nice to see you used JS instead of yet another DSL :)

Aerbil313 2y ago
I suggest the post author and you look into nu as the package build language. It's uniquely suited for this task, being a cross-platform shell[1] with real programming language features, data types, and using built-in uutils instead of relying on platform-dependent coreutils. It also limits mutability[2], favoring a more functional than imperative approach, which would at least theoretically help with reproducibility. A guy from Determinate Systems recently made an experiment on integrating nushell to nix and the results were positive. Take a look: https://determinate.systems/posts/nuenv/ https://determinate.systems/posts/nuenv/ 1: https://www.nushell.sh/ https://www.nushell.sh/ 2: https://www.nushell.sh/book/thinking_in_nu.html#variables-are-immutable https://www.nushell.sh/book/thinking_in_nu.html#variables-ar...
- kylewlacy 2y ago
  Nushell has definitely been on my radar for a while! but I missed DetSys was playing around with it, I'll have to give that a look One neat thing about Brioche is that Bash isn't really baked into it the same way as the Nix ecosystem. In a lot of the docs and examples, I used the `std.runBash` Brioche function for running scripts, but that's a fairly simple 11-line function[^1]. I wanted to make it just as easy to add `runZsh`, `runFish`, `runNushell`, etc. functions that work just as well as Bash (and they could either be put in the std library or in their own packages). So hopefully, there will be a Brioche `runNushell` function in the near future :) [^1]: https://github.com/brioche-dev/brioche-packages/blob/9fd51090a56cf83f4aa9c5002b78f2b2054ad3ad/packages/std/extra/run_bash.bri#L37-L47 https://github.com/brioche-dev/brioche-packages/blob/9fd5109...

__MatrixMan__ 2y ago

Some of the oddities of the nix language are pretty useful for its domain. Recursive attribute sets, for instance, save a lot of headaches if you're trying to have only a single source of truth. Do you feel like these translate to typescript nicely? As somebody who knows nix but doesn't know typescript, I found myself looking for a rosetta stone page where I could look at two chunks of code that do the same thing, but in separate languages. This would let me use the familiar language to scrutinize the other. I wouldn't normally ask for such a thing, but if you're putting "Nix-like" in the title then maybe it might be worth adding a comparison page to the docs.

charlotte-fyi 2y ago
The syntax isn’t as clean but you can add property accessors to objects in JavaScript that accomplishes the same thing.
kylewlacy 2y ago
I've been asked about overrides and attributes a lot! That was one of the sacrifices I had to make to go with a more traditional language, so it's definitely a negative point compared to Nix. That said, I'm hoping to have some conventions and features that will cover _most_ of the use-cases that overrides give you, but that's definitely still future work at this point and I probably won't be able to cover everything overrides do And yep, I think having a "Brioche for Nix users" guide makes a lot of sense, although it's not the first time that question so I'll probably stand up a first-pass version of it sooner rather than later (my Nix skills are also pretty rusty-- I'll need to brush up a bit first before I write it!)
hamandcheese 2y ago
I think the lack of true laziness will be a big performance problem for a large build graph. On the other hand the monolithic nature of the nixpkgs package set is one of the authors gripes with nix, so performance at that scale may be a non-goal.
- kylewlacy 2y ago
  I'd definitely like to have good performance even for large build graphs! I'm hoping the laziness exists "where it counts". To walk through an example, if you build your backend, and your backend calls the function `postgres()`, and that calls `openssl()`, and THAT calls `gcc()`, etc., etc., each function is basically building an object to represent its chunk of the build graph (each function returns a "recipe"). Nothing gets built until that object gets returned from the top-level function and the runtime does something with it In other words, the eager part is basically constructing the build graph. Maybe I'm wrong but I don't that this would necessarily be slower than the lazy version. In practice the most complex build graph I've made is basically the full chain of Linux From Scratch builds (that's the basis for my toolchain currently), and I think that takes about 400-500ms to evaluate. It's about 160 build steps, so it's not _simple_ but I know build graphs can also get a lot more complex, so I'll just have to keep an eye on performance as I start to get into more and more complex builds Maybe I'm missing something but intuitively I'd expect this approach to be fairly efficient-- as long as build scripts only call these functions when they're used as part of the build graph

leoh 2y ago

I am not writing typescript -- heaven forbid -- so I can manage cargo, so I can write Rust. Dropping the snark, though -- most times I have seen folks attempt to fold other build systems into some polyglot common one, it seems to create a lot of problems -- for at the end of the day, build tools (whether npm, cargo, gradle, docker, etc.) each have their issues. When these issues arise, they require understanding the original build tool. Bazel, for example, is a "polyglot" system which many try to use to manage npm. In my experience, when folks use bazel but have never used npm directly, they often get lost, because they now have to deal with two abstractions (bazel, npm) simultaneously neither have a way for bazel to do the thing they need, nor an understanding of how to get bazel to do that thing (nor npm itself to do the thing), also because they have never worked with npm directly. Same goes for cargo, incidentally. Switching over to that build tool -- like any major build tool, it has an insane amount of well-honed documentation and thousands of answered questions online (not to mention LLMs trained somewhat well on it). Users when they need to do something with brioche or have an issue with it will not necessarily have the benefit of the massive collaborative support available for cargo. Incidentally, I don't think that nix is free of this problem, though it does attempt to manage these issues -- sometimes reasonably successfully -- by virtue of keeping interfaces to other build tools in simple, direct ways. Even then, however, the best way I have seen nix used is as a system for declaring all binary dependencies or other dependencies that are either not manageable or especially competently managed by other build tools -- which nix allows for in a virtual environment by shipping a shell.nix or flake. In other words -- bringing in the right version of java, gradle, node, etc. But then just directing users to the specific, native build tools that often have a massive number of contributors. The closest thing to a tool that does this (guides developers to other build tools) is flox, which uses TOML, but uses nix for its backend via C API. I haven't had a chance to use it much yet, but it looks very promising https://flox.dev https://flox.dev. === Okay, all this rambling aside -- building a tool like this for yourself or a small team, especially when you have the time to do it, it works well, and you're having fun... well, there's an insane amount to be said for that; and it will doubtlessly lead you to learning a ton about other build tools and likely offer you deep insights that are only possible when attempting to build something like brioche. So, on that count, major kudos to you.

Zambyte 2y ago

I'm really happy to see more work on package managers in this space. I used Nix for a little bit, but I shared the frustration expressed by other commenters on the language itself. I have found my zen on GNU Guix, but more options are better :)

notthemessiah 2y ago
Tvix aims to be a modulary implementation of Nix, which could create a bridge between Nix and Guix, according to a claim by its author: "It should be possible to write an evaluator that plugs in the Guile language (for compatibility with GNU Guix), to use arbitrary builders, and to replace the store implementation." https://tvix.dev/ https://tvix.dev/

zarathustreal 2y ago

I’ve generally found the most interesting things are the ones that don’t feel intuitive at first. Familiarity bias can really hold you back. The language semantics of Nix are basically Haskell’s which.. is probably the One True Way to write correct code if we’re honest with ourselves.

hexo 2y ago
nix does not have haskells semantics
- zarathustreal 2y ago
  Yes it does? Purely functional, lazily evaluated.. declarative emphasis. Yea all it’s missing is strong typing, which is why I said it’s similar, not the same.
davidalayachew 2y ago
I strongly disagree. Haskell is great in a pure world, but it has an undue amount of friction when in the IO world. It does work in the IO world just fine, but languages like Java handle the actual concerns of the IO world better than Haskell. And for that reason, I say that no, Haskell is not the one true way to write correct code. It definitely is the One True Way to write happy path code though.

raffraffraff 2y ago

Reading the comments makes me think I'm turning into my dad. He was a simple man in terms of education and career. He was a fire man on a stream train in 1950, a farmer, barman, handyman. Out in the sticks, he was everyone's go-to guy when a vet couldn't be found, because he kept a "livestock first aid kit" including penicillin & syringes. He talked to vets and read books and knew about the most common ailments and how to treat them. He serviced his own car and tractor, including stripping the tractor engine down in his yard without proper tools, and getting cylinders bored and pistons sleeved, and putting it back together. He ran electric cable in farm building and sheds (after chatting to an electrician) and he did a ton of building work around the house. But at a certain age, maybe around 50, all new technology was out. Electricity was fine but electronics were out. Mechanical problems were fine because you could look straight at a thing and figure out how it clicked together, but once it had a microcontroller and a panel that abstracted away the actual working of the machine it just became some weird arbitrary "thing" that blocked his view of the machine. I'm sure if he was in his 20s when this stuff appeared he'd be all over it, but he was born in 1926, so I think the mighty micro was too late for him. He flatly refused to use the VCR, alarm systems, any type of computer. Show him a rubis cube, he's interested. Show him a microwave oven with more than 5 buttons and he'll make a sandwich instead. This is me with nix. I'm ok with all the stuff that's emerged in my field (infra/sysadmin) since the 90s. But nix makes me instantly glaze over. Several times I tried, but it's like my brain unplugs itself. I'm sure it does some wonderful stuff at an extremely high price that I'm not willing to pay. Nix has a huge mindshare, which I don't think you can win back if Nix keeps fixing the core issues mentored in these comments. Best of luck with this, though.

otabdeveloper4 2y ago
Nix solves the same problem we used to solve back in the day (before Docker taught us to run everything as root and disable firewalls, #yolo) ad-hoc with random poorly maintained scripts. If you never had to solve this problem then it probably isn't for you. If you did, then being able to finally replace the old contraptions with something properly engineered is a breath of fresh air.
- Dalewyn 2y ago

tkz1312 2y ago

Looks awesome! Congrats on the release. I would be curious to hear your thoughts on how this compares to garn [1], a typescript front end to the existing nixpkgs ecosystem. 1: https://garn.io/ https://garn.io/

kylewlacy 2y ago
I think garn is really cool! If I were still in the Nix ecosystem, garn would probably be my go-to Building something on Nix obviously has a ton of benefits: well-proven tooling, lots of mindshare already, TONS of packages. Everyone knows that Nix-the-language is fairly unintuitive for beginners, and tools like garn, Devenv, Devbox, etc. all seem to be trying to let folks use Nix without exposing them to Nix-the-language. I think this is a good goal to have and helps make Nix more accessible But I really thought it was worth exploring the Nix-free design space too. For example, I think derivations in Nix are... kinda complicated and weird! Having a fixed set of phases specifically designed for autotools builds always seemed a little weird to me. By ditching Nix, I was able to see what the world looks like when you ditch the idea of phases completely, for example Another interesting one was ditching the absolute path for the store path. Getting everything working by resolving relative paths was surprisingly tricky, but the end result is that you don't need to be root to install Brioche, and you can also take a Brioche-built binary (along with the resource dir it outputs) and ship it to another Linux machine and it'll just work. That's only something you can do if you see what happens if you explore outside the Nix space So I'm hoping Brioche can _at least_ be an interesting case study in how you can take a lot of Nix's ideas and get a lot of the same value without having to take Nix's approach directly (or who knows-- maybe some of those ideas end up back in Nix someday!)

doctor_phil 2y ago

I actually haven't tried Nix (or other nix-alternatives) yet. I've seen a lot of complaints about the language, but a lot of praise and love of the idea. I though Guix was the "main" competitor to nix, using a more mainstream language (Scheme). Is that still the case? How would Brioche compare against Guix if you are OK with both Scheme and Typescript as languages?

rgoulter 2y ago
> I've seen a lot of complaints about the language, but a lot of praise and love of the idea I think when people say they like the idea of nix, they're referring to how it builds packages from programmatic package formulae into an isolated store (so e.g. you can easily have different programs running against different versions of dependencies), and the use cases this allows for. -- With an expressive language for describing packaging, you get stuff like declarative developer environments, or even declared Linux configuration with quick/easy rollback. I think complaints about the language are partly from Nix being unusual enough that it takes time to get an intuition for the basics of how to write anything in it, but largely from the Nixpkgs (the large package repository the Nix package manager uses) being a large codebase, which grew organically with many contributors, with many similar-but-different solutions to problems, and sometimes the codebase is a bit too clever.
cmrdporcupine 2y ago
Guix is the main competitor but it's also focused on free software so mainly excludes things that don't fit that licensing philosophy. Yes there are I believe repositories that maintain non-free things but ... Nix already doesn't have declarations for a lot of things. Guix even less so.

Gud 2y ago

I’ve always wondered why no one ports FreeBSDs pkg/ports package management system. From all the operating systems I’ve used it’s the clear leader.

deleted 2y ago
[deleted]
justinsaccount 2y ago
https://pkgsrc.org/ https://pkgsrc.org/ from netbsd runs on many systems

amelius 2y ago

Can't we have something Python based, since it is already the lingua franca and seems to be the language that people are least frustrated about?

hisamafahri 2y ago

Wow, congrats for the release!!! But, can we revisit the name? “Brioche” sounds cool but (at least for me) it’s so awkward to type on the terminal and pronounce