8 ms·

> These kinds of dependencies are everywhere and nobody would even think that they could be harmful. Tons of people think these kind of micro dependencies are

> These kinds of dependencies are everywhere and nobody would even think that they could be harmful. Tons of people think these kind of micro dependencies are harmful and many of them have been saying it for years.

anonzzzies 1y ago
Yes. It is a bit painful this is not rather obvious by now. But I do have, every code review, whine about people who just include trivial outdated one function npms :(
SebastianKra 1y ago
Yeah, there's an entire community dedicated to cleaning up the js ecosystem. https://e18e.dev/ https://e18e.dev/ Micro-dependencies are not the only thing that went wrong here, but hopefully this is a wakeup call to do some cleaning.
- skydhash 1y ago
  Discord server? Is it that much work to create a forum or a mailing list with anonymous access. Especially with a community you can vet that easily?
balder1991 1y ago
Working for a bank did make me think much more about all the vulnerabilities that can go into certain tools. The company has a lot of bureaucracy to prevent installing anything or adding external dependencies.
- benoau 1y ago
  Working for a fintech and being responsible for the software made me very wary of dependencies and weeding out the deprecated and EOL'd stuff that had somehow already found its way into what was a young project when I joined. Left unrestrained, developers will add anything if it resolves their immediate needs like you could probably spread malware very well just by writing a fake-blog advocating a malicious module to solve certain scenarios.
  - esseph 1y ago
    > Left unrestrained, developers will add anything if it resolves their immediate needs Absolutely. A lot of developers work on a large Enterprise app for years and then scoot off to a different project or company. What's not fun is being the poor Ops staff that have to deal with supporting the library dependencies, JVM upgrades, etc for decades after.
stickfigure 1y ago
It wouldn't be a problem if there wasn't a culture of "just upgrade everything all the time" in the javascript ecosystem. We generally don't have this problem with Java libraries, because people pick versions and don't upgrade unless there's good reason.
- ilvez 1y ago
  From maintenance perspective both never and always seem like extremes though. Upgrading when falling off the train is serious drawback on moving fast..
  - 0xDEAFBEAD 1y ago
    Maybe we need two upgrade paths: An expedited auto-upgrade path which requires multi-key signoff from various trusted developers, and a standard upgrade path which is low-pressure.
- jcelerier 1y ago
  and then you get Log4Shell
Groxx 1y ago
I'm rather convinced that the next major language-feature wave will be permissions for libraries. It's painfully clear that we're well past the point where it's needed. I didn't think it'll make things perfect, not by a long shot. But it can make the exploits a lot harder to pull off.
- mbrevda1 1y ago
  yup, here is node's docs for it (WIP): https://nodejs.org/api/permissions.html https://nodejs.org/api/permissions.html
- crazygringo 1y ago
  Totally agreed, and I'm surprised this idea hasn't become more mainstream yet. If a package wants to access the filesystem, shell, OS API's, sockets, etc., those should be permissions you have to explicitly grant in your code.
  - crdrost 1y ago
    This was one of Doug Crockford's big bugaboos since The Good Parts and JSLint and Yahoo days—the idea that lexical scope aka closures give you an unprecedented ability to actually control I/O because you can say function main(io) { const result = somethingThatRequiresHttp(io.fetch); // ... } and as long as you don't put I/O in global scope (i.e. window.fetch) but do an injection into the main entrypoint, that entrypoint gets to control what everyone else can do. I could for example do function main(io) { const result = something(readonlyFetch(onlyOurAPI(io.fetch)) } function onlyOurAPI(fetch) { return (...args) => { const test = /^https:\/\/api.mydomain.example\//.exec(args[0]); if (test == null) { throw new ValueError("must only communicate with our API"); } return fetch(..args); } } function readonlyFetch(fetch) { /* similar but allowlist only GET/HEAD methods */ } I vaguely remember him being really passionate about "JavaScript lets you do this, we should all program in JavaScript" at the time... these days he's much more likely to say "JavaScript doesn't have any way to force you to do this and close off all the exploits from the now-leaked global scope, we should never program in JavaScript." Shoutout to Ryan Dahl and Deno, where you write `#!/usr/bin/env deno --allow-net=api.mydomain.example` at the start of your shell script to accomplish something similar. In my amateur programming-conlang hobby that will probably never produce anything joyful to anyone other than me, one of those programming languages has a notion of sending messages to "message-spaces" and I shamelessly steal Doug's idea -- message-spaces have handles that you can use to communicate with them, your I/O is a message sent to your main m-space containing a bunch of handles, you can then pattern-match on that message and make a new handle for a new m-space, provisioned with a pattern-matcher that only listens for, say, HTTP GET/HEAD events directed at the API, and forwards only those to the I/O handle. So then when I give this new handle to someone, they have no way of knowing that it's not fully I/O capable, requests they make to the not-API just sit there blackholed until you get an alert "there are too many unread messages in this m-space" and peek in to see why.
amarant 1y ago
Throwback to leftpad! Hey that was also on NPM iirc!
amysox 1y ago
What I'd like to know is why anyone thinks it's a good idea to have this level of granularity in libraries? Seriously? A library that only contains "a utility function that determines if its argument can be used like an array"? That's a lot of overhead in dependency management, which translates into a lot of cognitive load. Sooner or later, something's going to snap...and something did, here.
procaryote 1y ago
I've nixed javascript in the backend in several places, partly because of the weird culture around dependencies. Having to audit that for compliance, or keeping it actually secure, is a nightmare. Nixing javascript in the frontend is a harder sell, sadly
- christophilus 1y ago
  What did you switch to instead? I used to be a C# dev, and have done my fair share of Go. Both of those have decent enough standard libraries that I never found myself with a large 3rd party dependency tree. Ruby, Python, and Clojure, though? They weren’t any better than my npm projects, being roughly the same order of magnitude. Same seems to be true for Rust.
  - procaryote 1y ago
    You can get pretty far in python without a lot of dependencies, and the dependencies you do need tend to be more substantial blocks of functionality. Much easier to keep the tree small than npm. Same with Java, if you avoid springboot and similar everything frameworks, which admittedly is a bit of an uphill battle given the state of java developers. You can of course also keep dependencies small in javascript, but it's a very uphill fight where you'll have just a few options and most people you hire are used to including a library (that includes 10 libraries) to not have to so something like `if (x % 2 == 1)` Just started with golang... the language is a bit annoying but the dependency culture seems OK

anonzzzies 1y ago

Yes. It is a bit painful this is not rather obvious by now. But I do have, every code review, whine about people who just include trivial outdated one function npms :(

SebastianKra 1y ago

Yeah, there's an entire community dedicated to cleaning up the js ecosystem. https://e18e.dev/ https://e18e.dev/ Micro-dependencies are not the only thing that went wrong here, but hopefully this is a wakeup call to do some cleaning.

skydhash 1y ago
Discord server? Is it that much work to create a forum or a mailing list with anonymous access. Especially with a community you can vet that easily?

balder1991 1y ago

Working for a bank did make me think much more about all the vulnerabilities that can go into certain tools. The company has a lot of bureaucracy to prevent installing anything or adding external dependencies.

benoau 1y ago
Working for a fintech and being responsible for the software made me very wary of dependencies and weeding out the deprecated and EOL'd stuff that had somehow already found its way into what was a young project when I joined. Left unrestrained, developers will add anything if it resolves their immediate needs like you could probably spread malware very well just by writing a fake-blog advocating a malicious module to solve certain scenarios.
- esseph 1y ago
  > Left unrestrained, developers will add anything if it resolves their immediate needs Absolutely. A lot of developers work on a large Enterprise app for years and then scoot off to a different project or company. What's not fun is being the poor Ops staff that have to deal with supporting the library dependencies, JVM upgrades, etc for decades after.

stickfigure 1y ago

It wouldn't be a problem if there wasn't a culture of "just upgrade everything all the time" in the javascript ecosystem. We generally don't have this problem with Java libraries, because people pick versions and don't upgrade unless there's good reason.

ilvez 1y ago
From maintenance perspective both never and always seem like extremes though. Upgrading when falling off the train is serious drawback on moving fast..
- 0xDEAFBEAD 1y ago
  Maybe we need two upgrade paths: An expedited auto-upgrade path which requires multi-key signoff from various trusted developers, and a standard upgrade path which is low-pressure.
jcelerier 1y ago
and then you get Log4Shell

Groxx 1y ago

I'm rather convinced that the next major language-feature wave will be permissions for libraries. It's painfully clear that we're well past the point where it's needed. I didn't think it'll make things perfect, not by a long shot. But it can make the exploits a lot harder to pull off.

mbrevda1 1y ago
yup, here is node's docs for it (WIP): https://nodejs.org/api/permissions.html https://nodejs.org/api/permissions.html
crazygringo 1y ago
Totally agreed, and I'm surprised this idea hasn't become more mainstream yet. If a package wants to access the filesystem, shell, OS API's, sockets, etc., those should be permissions you have to explicitly grant in your code.
- crdrost 1y ago
  This was one of Doug Crockford's big bugaboos since The Good Parts and JSLint and Yahoo days—the idea that lexical scope aka closures give you an unprecedented ability to actually control I/O because you can say function main(io) { const result = somethingThatRequiresHttp(io.fetch); // ... } and as long as you don't put I/O in global scope (i.e. window.fetch) but do an injection into the main entrypoint, that entrypoint gets to control what everyone else can do. I could for example do function main(io) { const result = something(readonlyFetch(onlyOurAPI(io.fetch)) } function onlyOurAPI(fetch) { return (...args) => { const test = /^https:\/\/api.mydomain.example\//.exec(args[0]); if (test == null) { throw new ValueError("must only communicate with our API"); } return fetch(..args); } } function readonlyFetch(fetch) { /* similar but allowlist only GET/HEAD methods */ } I vaguely remember him being really passionate about "JavaScript lets you do this, we should all program in JavaScript" at the time... these days he's much more likely to say "JavaScript doesn't have any way to force you to do this and close off all the exploits from the now-leaked global scope, we should never program in JavaScript." Shoutout to Ryan Dahl and Deno, where you write `#!/usr/bin/env deno --allow-net=api.mydomain.example` at the start of your shell script to accomplish something similar. In my amateur programming-conlang hobby that will probably never produce anything joyful to anyone other than me, one of those programming languages has a notion of sending messages to "message-spaces" and I shamelessly steal Doug's idea -- message-spaces have handles that you can use to communicate with them, your I/O is a message sent to your main m-space containing a bunch of handles, you can then pattern-match on that message and make a new handle for a new m-space, provisioned with a pattern-matcher that only listens for, say, HTTP GET/HEAD events directed at the API, and forwards only those to the I/O handle. So then when I give this new handle to someone, they have no way of knowing that it's not fully I/O capable, requests they make to the not-API just sit there blackholed until you get an alert "there are too many unread messages in this m-space" and peek in to see why.

amarant 1y ago

Throwback to leftpad! Hey that was also on NPM iirc!

amysox 1y ago

What I'd like to know is why anyone thinks it's a good idea to have this level of granularity in libraries? Seriously? A library that only contains "a utility function that determines if its argument can be used like an array"? That's a lot of overhead in dependency management, which translates into a lot of cognitive load. Sooner or later, something's going to snap...and something did, here.

procaryote 1y ago

I've nixed javascript in the backend in several places, partly because of the weird culture around dependencies. Having to audit that for compliance, or keeping it actually secure, is a nightmare. Nixing javascript in the frontend is a harder sell, sadly

christophilus 1y ago
What did you switch to instead? I used to be a C# dev, and have done my fair share of Go. Both of those have decent enough standard libraries that I never found myself with a large 3rd party dependency tree. Ruby, Python, and Clojure, though? They weren’t any better than my npm projects, being roughly the same order of magnitude. Same seems to be true for Rust.
- procaryote 1y ago
  You can get pretty far in python without a lot of dependencies, and the dependencies you do need tend to be more substantial blocks of functionality. Much easier to keep the tree small than npm. Same with Java, if you avoid springboot and similar everything frameworks, which admittedly is a bit of an uphill battle given the state of java developers. You can of course also keep dependencies small in javascript, but it's a very uphill fight where you'll have just a few options and most people you hire are used to including a library (that includes 10 libraries) to not have to so something like `if (x % 2 == 1)` Just started with golang... the language is a bit annoying but the dependency culture seems OK