4 ms·

I mentioned it in another answer, but we'll eject the packages that require postinstall scripts - so they will work as before, but will take extra time to be in

I mentioned it in another answer, but we'll eject the packages that require postinstall scripts - so they will work as before, but will take extra time to be installed. As for wasm, I'm curious to hear what you think isn't good enough. I think the two main issues are garbage collection and dynamic linking, and there's ongoing work on them to fix them.

jorangreef 8y ago
"I mentioned it in another answer, but we'll eject the packages that require postinstall scripts" Sure, but you also mentioned in the PDF that the other solution was just to deprecate native modules in the long term, and that's not acceptable. "As a data point, we encountered no problem at Facebook with adding --ignore-scripts to all of our Yarn invocations. The two main projects we’re aware of that still use postinstall scripts are fsevents (which is optional, and whose absence didn’t cause us any harm), and node-sass (which is currently working on a WebAssembly port)." I think you're too quick to sacrifice native modules because you're not really using them.
- arcatek 8y ago
  I did work on the asmjs/wasm bindings for Yoga, Text-Buffer, and a few other projects, so I'm biased :) Anyway, regardless of my own long term feelings, rest assured that postinstall scripts will be supported as well as they are now.
  - Benjamin_Dobell 8y ago
    Isn't native interoperability the entire point of Node.js though? Filesystem access etc. has to, at some point, cross the barrier from VM to native code. Sure Node.js has file-system access (and other stuff) built in, but willingly moving in a direction that's non-extensible seems odd. I know you've since said you'll just pull native modules out of the cache, and I'm perfectly happy with that. However, please don't overlook the importance of native interoperability because the code you tend to write abstracts this away from you. Native interoperability is what makes Node.js.
electroly 8y ago
Native code has two benefits over JavaScript: it's faster, and it can call the OS. WASM is faster, but it can't call the OS. It cannot replace all uses of native modules.
- mixedCase 8y ago
  Wasm on the browser can't call the OS, but I don't assume that to be the case on Node, just like it isn't for JS.
  - mhenr18 8y ago
    JS (and eventually wasm) calls the OS via requiring native modules, which is why we're concerned about keeping them around.
- zeroname 8y ago
  Calling the OS doesn't require native modules, it should be supported via a native cffi in node, like in Python.
  - electroly 8y ago
    Are you describing something you wish that node had, or something that actually exists? As far as I'm aware, native modules are the FFI for node, and that every package that involves presenting an FFI to the developer is using a native module to do it.
    - zeroname 8y ago
      Node doesn't have it natively like Python does, but it exists: https://github.com/node-ffi/node-ffi https://github.com/node-ffi/node-ffi
      - electroly 8y ago
        Yeah, that's a native module. If you look at the C source code in "src", the "NAN_METHOD" and stuff are Node native module macros. node-ffi will not work if they remove native modules.
jpambrun 8y ago
I don't think WAM achieve native performance yet. You need to copy data in and out of it instead of zero-copy. WASM can't link to native system libraries. WASM can't access most hardware. It's very neat, but I can't see it fully replacing native modules soon.
mhenr18 8y ago
Wasm is designed to deal with the VM sandbox, which if you're writing a server is fantastic compared to actual native code as there's better security properties. It's not fantastic if you're working on an Electron app where there might be one specific feature that requires a native module in order to hit an OS API that you can't do from a browser sandbox (and that Electron itself doesn't expose). Rather than pursue total deprecation/eventual removal of postinstall/native modules, I think a package.json "allowNativeDependencies": "never" | "devOnly" | "always" option that defaults to "devOnly" if not specified is the way to go.
- woodrowbarlow 8y ago
  i would say make it a boolean flag, default to false, with a comment that says "this must be enabled for packages which use the local filesystem or OS APIs". if wasm delivers on its potential, OS-level interactions would really be the only valid use case (and legacy code, which should be considered a little bit inherently unsafe anyway). perhaps yarn could even develop official packages which include native code that wraps all the most relevant OS APIs, and the default setting of the flag would allow only those native packages. this is all pretty far down the road anyway.
kabes 8y ago
Almost all the native modules I come across are about accessing system calls not available from within the sandbox. Only a few are there for speed, and since native calls add quite some overhead, they are only useful in specific use cases.

jorangreef 8y ago

"I mentioned it in another answer, but we'll eject the packages that require postinstall scripts" Sure, but you also mentioned in the PDF that the other solution was just to deprecate native modules in the long term, and that's not acceptable. "As a data point, we encountered no problem at Facebook with adding --ignore-scripts to all of our Yarn invocations. The two main projects we’re aware of that still use postinstall scripts are fsevents (which is optional, and whose absence didn’t cause us any harm), and node-sass (which is currently working on a WebAssembly port)." I think you're too quick to sacrifice native modules because you're not really using them.

arcatek 8y ago
I did work on the asmjs/wasm bindings for Yoga, Text-Buffer, and a few other projects, so I'm biased :) Anyway, regardless of my own long term feelings, rest assured that postinstall scripts will be supported as well as they are now.
- Benjamin_Dobell 8y ago
  Isn't native interoperability the entire point of Node.js though? Filesystem access etc. has to, at some point, cross the barrier from VM to native code. Sure Node.js has file-system access (and other stuff) built in, but willingly moving in a direction that's non-extensible seems odd. I know you've since said you'll just pull native modules out of the cache, and I'm perfectly happy with that. However, please don't overlook the importance of native interoperability because the code you tend to write abstracts this away from you. Native interoperability is what makes Node.js.

electroly 8y ago

Native code has two benefits over JavaScript: it's faster, and it can call the OS. WASM is faster, but it can't call the OS. It cannot replace all uses of native modules.

mixedCase 8y ago
Wasm on the browser can't call the OS, but I don't assume that to be the case on Node, just like it isn't for JS.
- mhenr18 8y ago
  JS (and eventually wasm) calls the OS via requiring native modules, which is why we're concerned about keeping them around.
zeroname 8y ago
Calling the OS doesn't require native modules, it should be supported via a native cffi in node, like in Python.
- electroly 8y ago
  Are you describing something you wish that node had, or something that actually exists? As far as I'm aware, native modules are the FFI for node, and that every package that involves presenting an FFI to the developer is using a native module to do it.
  - zeroname 8y ago
    Node doesn't have it natively like Python does, but it exists: https://github.com/node-ffi/node-ffi https://github.com/node-ffi/node-ffi
    - electroly 8y ago
      Yeah, that's a native module. If you look at the C source code in "src", the "NAN_METHOD" and stuff are Node native module macros. node-ffi will not work if they remove native modules.

jpambrun 8y ago

I don't think WAM achieve native performance yet. You need to copy data in and out of it instead of zero-copy. WASM can't link to native system libraries. WASM can't access most hardware. It's very neat, but I can't see it fully replacing native modules soon.

mhenr18 8y ago

Wasm is designed to deal with the VM sandbox, which if you're writing a server is fantastic compared to actual native code as there's better security properties. It's not fantastic if you're working on an Electron app where there might be one specific feature that requires a native module in order to hit an OS API that you can't do from a browser sandbox (and that Electron itself doesn't expose). Rather than pursue total deprecation/eventual removal of postinstall/native modules, I think a package.json "allowNativeDependencies": "never" | "devOnly" | "always" option that defaults to "devOnly" if not specified is the way to go.

woodrowbarlow 8y ago
i would say make it a boolean flag, default to false, with a comment that says "this must be enabled for packages which use the local filesystem or OS APIs". if wasm delivers on its potential, OS-level interactions would really be the only valid use case (and legacy code, which should be considered a little bit inherently unsafe anyway). perhaps yarn could even develop official packages which include native code that wraps all the most relevant OS APIs, and the default setting of the flag would allow only those native packages. this is all pretty far down the road anyway.

kabes 8y ago

Almost all the native modules I come across are about accessing system calls not available from within the sandbox. Only a few are there for speed, and since native calls add quite some overhead, they are only useful in specific use cases.