3 ms·

Sounds like their main concern wasn't Nix but the complexity of the code itself.

by yawaramin 6y ago

addicted 6y ago
The hours long build system appears to be attributed to Nix. Is Nix really that slow?
- mason55 6y ago
  Dependencies can take awhile if you can’t used cached versions of the binaries. Building some Haskell tooling from scratch took ~3 hours on my MBP.
  - gmfawcett 6y ago
    There are few compilers that are slower than GHC. I suspect this issue has nothing to do with Nix, and the tools would have taken ages to build from scratch without Nix.
- geofft 6y ago
  If you're applying idiomatic Nix to modern JS, I wouldn't be surprised - modern JS tends to involve installing thousands of packages by just combining them into a directory, but Nix doesn't want you to edit existing directories. So your dependency graph turns into a build-dependency graph, with each JS package requiring a full build of everything it depends on, and the Nix build system is presumably not optimized for fast turnaround times on five-line JS modules that don't even have a compilation step. (Personally, I wouldn't try to apply idiomatic Nix to modern JS - I'd apply it to the major components like "my web server" and "my database library" and have "all the JS I depend on" as one big Nix package. That's not really a claim about Nix, I wouldn't try to apply idiomatic, say, Debian packaging to modern JS either. In both cases I'd still get about 90% of the benefit of using Nix/Debian as a delivery mechanism.) The other thing that could be slow is if you start compiling all your dependencies, including gcc and node.js, from scratch. While there's some security benefit in doing so, the reality is that just about nobody actually does that. You'd want to set things up to use the precompiled packages, or at least set up your own cache server and have compiled binaries you trust but only do it once.
  - joepie91_ 6y ago
    Hi, JS developer here who uses Nix and who liberally makes use of single-responsibility modules. No, Nix does not need "hours" to build a modern JS project. Something else is going on here. Nix is slow, but not that slow. (With the limited information provided here, my first guess would be "someone turned off the binary cache because they don't trust prebuilt binaries, and now it recompiles the world from scratch because there are no prebuilt binaries anymore".)
    - somatic 6y ago
      “who liberally makes use of single-responsibility modules” If your aim is precision of language, you should call this “400MB of unaudited and unauditable mystery code from the [total clowns]( https://github.com/babel/babel/pull/3646 https://github.com/babel/babel/pull/3646 )* who run the npm ecosystem.” Imagine [this]( https://medium.com/s/silicon-satire/i-peeked-into-my-node-modules-directory-and-you-wont-believe-what-happened-next-b89f63d21558 https://medium.com/s/silicon-satire/i-peeked-into-my-node-mo... )* but instead riddled with advertisers and intelligence agencies. Burn it down and start over. * Insert Markdown, etc.
      - yawaramin 6y ago
        Part of the removed code: test("guy", function() { assert.equal(typeof babel.guy, "string"); }); Facepalming so hard right now
        somatic 6y ago
        That’s a quality test. Don’t you feel more comfortable? Your browsing experience is in good hands.
        geofft 6y ago
        Thank you for heavily derailing the productive, technically meaningful conversation I was having. Next time I won't say anything since apparently productive and technically meaningful conversations aren't what this site is for.
- tomberek 6y ago
  I've built a combined python2/3,c,c++,javascript (VueJS) deployment along with hardware-in-the-loop integration testing and compiling for ARM on an Raspberry Pi with Nix. Should not take hours. Something else is wrong. Perhaps binary cache is not being used? Something is invalidating the builds like a "src = ./.;" which telling the build system to include every single file in the directory in the build rather than using a .gitignore or git-hash for reproducibility. Or someone was using npm2nix (deprecated)?
- floriol 6y ago
  No, nix can cache already built binaries with hydra, so on a normal system it simply downloads from there - same speed as apt or the like. During the audit they likely disabled it, so nix had to build everything from scratch (like even system libraries, the build tools, everything).

Sounds like their main concern wasn't Nix but the complexity of the code itself.

addicted 6y ago

The hours long build system appears to be attributed to Nix. Is Nix really that slow?

mason55 6y ago
Dependencies can take awhile if you can’t used cached versions of the binaries. Building some Haskell tooling from scratch took ~3 hours on my MBP.
- gmfawcett 6y ago
  There are few compilers that are slower than GHC. I suspect this issue has nothing to do with Nix, and the tools would have taken ages to build from scratch without Nix.
geofft 6y ago
If you're applying idiomatic Nix to modern JS, I wouldn't be surprised - modern JS tends to involve installing thousands of packages by just combining them into a directory, but Nix doesn't want you to edit existing directories. So your dependency graph turns into a build-dependency graph, with each JS package requiring a full build of everything it depends on, and the Nix build system is presumably not optimized for fast turnaround times on five-line JS modules that don't even have a compilation step. (Personally, I wouldn't try to apply idiomatic Nix to modern JS - I'd apply it to the major components like "my web server" and "my database library" and have "all the JS I depend on" as one big Nix package. That's not really a claim about Nix, I wouldn't try to apply idiomatic, say, Debian packaging to modern JS either. In both cases I'd still get about 90% of the benefit of using Nix/Debian as a delivery mechanism.) The other thing that could be slow is if you start compiling all your dependencies, including gcc and node.js, from scratch. While there's some security benefit in doing so, the reality is that just about nobody actually does that. You'd want to set things up to use the precompiled packages, or at least set up your own cache server and have compiled binaries you trust but only do it once.
- joepie91_ 6y ago
  Hi, JS developer here who uses Nix and who liberally makes use of single-responsibility modules. No, Nix does not need "hours" to build a modern JS project. Something else is going on here. Nix is slow, but not that slow. (With the limited information provided here, my first guess would be "someone turned off the binary cache because they don't trust prebuilt binaries, and now it recompiles the world from scratch because there are no prebuilt binaries anymore".)
  - somatic 6y ago
    “who liberally makes use of single-responsibility modules” If your aim is precision of language, you should call this “400MB of unaudited and unauditable mystery code from the [total clowns]( https://github.com/babel/babel/pull/3646 https://github.com/babel/babel/pull/3646 )* who run the npm ecosystem.” Imagine [this]( https://medium.com/s/silicon-satire/i-peeked-into-my-node-modules-directory-and-you-wont-believe-what-happened-next-b89f63d21558 https://medium.com/s/silicon-satire/i-peeked-into-my-node-mo... )* but instead riddled with advertisers and intelligence agencies. Burn it down and start over. * Insert Markdown, etc.
    - yawaramin 6y ago
      Part of the removed code: test("guy", function() { assert.equal(typeof babel.guy, "string"); }); Facepalming so hard right now
      - somatic 6y ago
        That’s a quality test. Don’t you feel more comfortable? Your browsing experience is in good hands.
        geofft 6y ago
        Thank you for heavily derailing the productive, technically meaningful conversation I was having. Next time I won't say anything since apparently productive and technically meaningful conversations aren't what this site is for.
tomberek 6y ago
I've built a combined python2/3,c,c++,javascript (VueJS) deployment along with hardware-in-the-loop integration testing and compiling for ARM on an Raspberry Pi with Nix. Should not take hours. Something else is wrong. Perhaps binary cache is not being used? Something is invalidating the builds like a "src = ./.;" which telling the build system to include every single file in the directory in the build rather than using a .gitignore or git-hash for reproducibility. Or someone was using npm2nix (deprecated)?
floriol 6y ago
No, nix can cache already built binaries with hydra, so on a normal system it simply downloads from there - same speed as apt or the like. During the audit they likely disabled it, so nix had to build everything from scratch (like even system libraries, the build tools, everything).