7 ms·
While this is clearly a bug, the default publish strategy is to publish all the contents of the package including dev/test files. If you look at the node_module
by kotxig 7y ago
While this is clearly a bug, the default publish strategy is to publish all the contents of the package including dev/test files. If you look at the node_modules tree on a typical server-side node project it's full of garbage files.
We still think it's a great idea to pull down hundreds if not thousands of copies of modules from a remote every time we generate a build. The community has always been lacking maturity around how it manages modules and releases code for production, preferring to preserve developer productivity and trinket features over best security practices and optimizing for the size and quality of production packages.
I don't want to install hundreds of modules every time I create a new build. I don't want your tests or your README in my production tarball. I don't want your browser compatibility code in my server code. I don't want my node_modules tree to be 9 layers deep. I don't want to have to dedupe multiple copies of modules by hand. I don't want to have to debug where shrinkwrap isn't respected. I don't want to play roulette with the package manager version to figure out which one does the right thing for my package. I don't want to run your dubious pre/post install scripts. I don't want to use npm.
The sad thing is, npm is a commercial entity that has its package manager published by default with an opensource community project. And this is why we can't have nice things.
- LeoNatan25 7y agoMaybe "we" should not be writing our server code using a terrible ecosystem? Just an idea.
- kotxig 7y agoIndeed. It's hard to avoid that conclusion, but you can't deprecate things overnight.
- halfmatthalfcat 7y agoNode has a great server-side ecosystem. I'd go as far as saying Node lends itself to certain layers of the backend stack because of it's history being more CPU-efficient then other server-side runtimes. Can you provide any examples where Node does not provide a good ecosystem for backend programming? Genuinely curious.
- zaarn 7y agoNode isn't great when you need not only CPU-efficient but also multithreaded. The multi-threading or even multi-processing story in node is a clusterfuck. Other than running two full fat Node.js processes, your options are limited to a few APIs that have more chance to break your kneecaps than run the code.
- halfmatthalfcat 7y agoSure but that's the whole premise of Node, there doesn't _need_ to be multithreading. Whats wrong with spinning up multiple workers (in something like Docker/Swarm/K8S) that split up the work and report back to whatever collector or service that wants the end result? So many problems have spawned out of the desire to thread everywhere that the safety of single-threadedness is a feature not a bug.
- zaarn 7y agoMultithreaded (or Multiprocess) can be done safely. I write a lot of stuff in Rust these days, where (unless you use unsafe) it doesn't even let you do anything that would break under multi-threading, even if you don't use it. As a result, it was incredibly easy to write a light-weight crawler that delivery results into a main worker queue and the data is processed by <cpu_cores> threads. The crawler largely uses async for IO advantages but the worker threads rely heavily on data crunching where your throughput is limited by CPU not by IO and async suffers a bit. And since crawler and worker are on the same process, the latency between a page being grabbed and starting to be processed is incredibly low, any kind of networking would likely cut throughput by a factor of 10 or more. Not even mentioning having to serialize the data instead of relying on references to get read-only copies and only making minimal copies on the stack.
- halfmatthalfcat 7y agoOk, fair point Rust makes multithreading easier but one could easily make the argument that fighting Rust's borrow checker is a huge burden to development. My point is Node isn't as bad as a server-side language as people claim. There are certain use cases where you can leverage Node's strengths and use another language somewhere else where its shines more. I use Node heavily in my backend stack and have been extremely pleased with it's performance and library ecosystem.
- drngdds 7y agoYeah, I don't know why npm defaults to including everything in the folder for the production build instead of having it be a whitelist. It's not that hard to specify "include package.json, package-lock.json, and ./build" in your package.json.
- kotxig 7y agoIt promotes certain developer-friendly ideals such as going into the module directly within node_modules or using npm explore. I think this is the wrong priority.
- halfmatthalfcat 7y agoThat's why it's a good idea to leverage bundlers and simply roll all of your desired dependencies into a single file and then just execute Node against that. You can easily do this with Webpack. edit: To clarify I'm talking about server-side components.
- andrew_ 7y agoFor anyone reading this comment, investigate Rollup. My own anecdotal experience is that it kicks webpack's behind when it comes to bundling/packaging server-side anything, or anything that'll be run via `node`
- shados 7y agoThats why we never replaced our homegrown, predates-npm package manager. It feels crazy to be maintaining our own when there's such a popular 3rd party one out there, but honestly...it really sucks. Migrating nearly a decade worth of code to something that is often objectively worse doesn't feel worth it. And so the homegrown package manager lives on...
- growtofill 7y ago‘We’ being?
- tracker1 7y ago`npm ci` will use a local cache, and only pull specifically identified versions.