5 ms·

This is probably naive, but I'd love to see one of the Node.js competitors, i.e. Bun or Deno, innovate on the "it takes ~forever to load node_modules every time

This is probably naive, but I'd love to see one of the Node.js competitors, i.e. Bun or Deno, innovate on the "it takes ~forever to load node_modules every time I run a test/script/etc" problem. I.e. Bun is improving "bun install" time for projects with large node_modules, and that's great, but what about, after it's on disk, having to parse+eval all of it from scratch every time I run a test? As admitted, this is a very naive ask due to the interpreted/monkey-patching nature of the JS language, but I'll hand wave with "something something v8 isolate snapshots" + "if accomplishing this requires restrictions on what packages do during module load, deno is rebooting npm anyway..." and hope someone smarter than me can figure it out. :-)

qbasic_forever 4y ago
Deno doesn't use node_modules so you won't have a problem there. You specify each dependency as an import URL (either in the code or in an import map) and it grabs them directly (also using a local cache directory). There's no need in the deno world to even use a package manager.
- ch4s3 4y ago
  > There's no need in the deno world to even use a package manager That doesn't sound like I thing I would want.
  - int_19h 4y ago
    It is a package manager. It's just integrated into the runtime to the point where you import things and it "just works".
    - ch4s3 4y ago
      How does that pan out as an application grows and you're pulling in the same dep across numerous files?
      - int_19h 4y ago
        You still get one shared copy of that dependency. Why would it be any different?
        ch4s3 4y ago
        How do you ensure that across files on a large project?
- stephen 4y ago
  Ah yeah, you're right that Deno doesn't have a node_modules, but AFAIU it still downloads dependencies to "somewhere on disk" and then, every time your code runs, it re-evals all of them from scratch. So, admittedly I was using "node_modules" as a shorthand for "the code that makes up my dependencies", and that AFAIU Deno has not implemented this "use a v8 snapshot to cache preloaded/pre-evald dependencies" optimization. I.e. I want something like: https://danheidinga.github.io/Everyone_wants_fast_startup/ https://danheidinga.github.io/Everyone_wants_fast_startup/
  - MuffinFlavored 4y ago
    does V8 support snapshot + restore?
  - CapsAdmin 4y ago
    You could maybe load bytecode if that's a thing in javascript to avoid parsing at least, but outside of that you'd have to make sure the environment is actually the same. For example if I define an environment variable and some script deep inside the modules folder reads that variable and does something with it. You'd need to make sure the environment is identical before everything is loaded. There are other things as well like a script defining a global (perhaps for polyfill) and it needs to load before some other script. The Nix package manager is probably a good place to look at how this should be done. However it's all based on strict constraints and other functional principles that I don't think npm packages are anywhere near of satisfying.

qbasic_forever 4y ago

Deno doesn't use node_modules so you won't have a problem there. You specify each dependency as an import URL (either in the code or in an import map) and it grabs them directly (also using a local cache directory). There's no need in the deno world to even use a package manager.

ch4s3 4y ago
> There's no need in the deno world to even use a package manager That doesn't sound like I thing I would want.
- int_19h 4y ago
  It is a package manager. It's just integrated into the runtime to the point where you import things and it "just works".
  - ch4s3 4y ago
    How does that pan out as an application grows and you're pulling in the same dep across numerous files?
    - int_19h 4y ago
      You still get one shared copy of that dependency. Why would it be any different?
      - ch4s3 4y ago
        How do you ensure that across files on a large project?
stephen 4y ago
Ah yeah, you're right that Deno doesn't have a node_modules, but AFAIU it still downloads dependencies to "somewhere on disk" and then, every time your code runs, it re-evals all of them from scratch. So, admittedly I was using "node_modules" as a shorthand for "the code that makes up my dependencies", and that AFAIU Deno has not implemented this "use a v8 snapshot to cache preloaded/pre-evald dependencies" optimization. I.e. I want something like: https://danheidinga.github.io/Everyone_wants_fast_startup/ https://danheidinga.github.io/Everyone_wants_fast_startup/
- MuffinFlavored 4y ago
  does V8 support snapshot + restore?
- CapsAdmin 4y ago
  You could maybe load bytecode if that's a thing in javascript to avoid parsing at least, but outside of that you'd have to make sure the environment is actually the same. For example if I define an environment variable and some script deep inside the modules folder reads that variable and does something with it. You'd need to make sure the environment is identical before everything is loaded. There are other things as well like a script defining a global (perhaps for polyfill) and it needs to load before some other script. The Nix package manager is probably a good place to look at how this should be done. However it's all based on strict constraints and other functional principles that I don't think npm packages are anywhere near of satisfying.