6 ms·

What’s the alternative? Ultimately, the issue isn’t “a dependency graph of a million tiny packages” but rather running other people’s code. If you read the atta

by rcme 3y ago

What’s the alternative? Ultimately, the issue isn’t “a dependency graph of a million tiny packages” but rather running other people’s code. If you read the attack details, this is a targeted campaign where people are being tricked into running untrusted code. I don’t see how removing small dependencies solves that problem.

jaredsohn 3y ago
I think a benefit of a smaller number of larger packages is reduces the number of parties you need to trust. The people in charge of each larger package can review contributions made to it and have the authority to reject code that includes attacks. If a larger package doesn't do a good job of preventing attacks, its reputation will decline and because there are fewer packages involved it is easy to remember these reputations.
- ljm 3y ago
  It's never as simple as finding exploits in code review though. Trusted accounts are compromised or the credentials to the package manager are compromised or there is some other supply chain attack. Larger packages tend to have a lot more community and governance around them than your typical JS-style function-as-a-dependency left-pad affair, so maybe they're safer if computer accounts are gatekeeping the published artifacts rather than having individuals embed their personal keys or tokens into the publishing pipeline. I've had this debate before and, as far as dependencies go, sometimes you have to rip the bandaid off and audit your shit, and understand why exactly you outsourced something to a package.
- thephyber 3y ago
  That seems intuitive. I agree in the sense that smaller supply chain surface area is easier to secure/verify. But it also makes the developers/maintainers of the few packages much brighter targets. I’m more of a fan of standardizing how libraries can be rated, evaluated, trusted, marketed, etc. we need more efforts analogous to Consumer Reports / Underwriters Laboratories. So far there are a few for-profit companies in this space, including Google.
ecnahc515 3y ago
With some other languages, you have big projects that basically are your secondary stdlib. Boost in C++, Apache Commons in Java, etc. These can reduce a lot of your dependencies down to a single umbrella project who stewards everything. The number of developers who work as maintainers on the packages within these projects is probably comparable to the number of developers working on the same sorts of packages in Node, but they're all working under a single banner, effectively. It's about having dependencies within projects as organizations vs the NPM approach where basically every dependency is it's own project and organization.
- noahtallen 3y ago
  The JS ecosystem has moved away from massive monolithic packages due to bundle size constraints. Unfortunately, JS code has a lot of extra constraints placed on it since it has to be instantly performant over the wire. Something desktop apps don't have to deal with! jQuery and Lodash could be considered "secondary stdlib" packages. Lodash for various data functions, and jQuery for dom manipulation functions. Now that other frameworks are more common and the basic stdlib is larger, many projects have migrated away from them to improve bundle size. (As an example, pulling in one lodash function tends to pull in lots of other functions from within lodash, unnecessarily inflating the amount of code sent to the browser.)
  - com2kid 3y ago
    Another advantage of this, at last for the backend, is that Node itself has a small footprint. Compared to the JVM, Node is tiny. It does some IO stuff and some eventing stuff and not much else. For the front end, ideally tree shaking is used, and even fancier build systems create code that doesn't pull down dependencies until they are actually needed at runtime.
  - bathtub365 3y ago
    Why not just strip out unused code? This has existed in other toolchains for years.
    - 3y ago
CharlieDigital 3y ago
Consider a runtime like Microsoft's .NET: - The standard base class libraries with .NET are quite comprehensive; I end up rarely adding "utility" libraries - For non-standard libraries, Microsoft is a first party publisher of many packages on nuget; many of their first party frameworks and tools are published this way - For third-party libraries, generally speaking, I end up using maybe a dozen or so very well-known and well-vetted libraries which have low dependencies of their own (again, because of the rich standard libraries) - You end up with far less surface area for dependency chain attacks According to the 2020 State of the Octoverse report [0]: - Node projects had nearly an order of magnitude more transitive dependencies than the next highest (683 to 70 for PHP) (p.11) - npm had the highest percentage of critical and high severity advisories (p.14) - 72.8% of JavaScript repos had dependabot alerts (well above the weighted average of 59%) compared to 6.5% for .NET and 40.2% for Java Some would say that Microsoft's stewardship of .NET and C# is a negative; it can never be truly OSS. The flip side of that is that there is a professional team of paid software engineers -- some of the best in the world -- maintaining, patching, and securing the platform. While the third party ecosystem doesn't have as many options for doing X, the few options for doing X on .NET tend to be more well-rounded by the tighter community focus on a few, well-known libraries. Maybe some variation of an Apache Foundation like approach to curating a set of OSS NPM libraries and some mechanism of establishing trust would be beneficial to the NPM ecosystem? Just my take. [0]: https://arxiv.org/ftp/arxiv/papers/2110/2110.10246.pdf https://arxiv.org/ftp/arxiv/papers/2110/2110.10246.pdf
- com2kid 3y ago
  And for literal decades the 3rd party library ecosystem around .NET was anemic. Importing 3rd party libraries was an afterthought in the language, typically it involved downloading a .dll from some server, dropping it in the same folder as your project, and then manually editing some XML. Things are better now, but for all its (many!) positives, .NET is not an example of how to create a good ecosystem. Of course back then Microsoft considered an ecosystem to be 3rd party partners selling libraries, of which quite a few successful companies did.
  - CharlieDigital 3y ago
mxkopy 3y ago
Normalize writing your own code. If you’re in an environment where importing GetFirstChar from an essentially random person is the norm, this is bound to happen. It seems like some developers write code only if the functionality isn’t provided by some package, when really they should be reaching for the imports only if writing the package is too much effort.
- thephyber 3y ago
  This is the opposite of specialization, so there are reasons to not do this. Examples: are you likely to write unit tests for your GetFirstChar? A specialized open source library that built this function is far more likely to have unit tests and documentation. Those unit tests and documentation come in handy during edge cases and when your coworker decides to extend the function with an `options` parameter. Also, general developers rarely understand the full breadth of character sets, for example. So if your function just grabs the 1st byte from a string and expects it to be a character, your end user might be angry when only the first byte of a multi byte character is returned. Similarly the difference between a character and a glyph and a rune, etc. are important and lesser understood.
  - boppo1 3y ago
    >difference between a character and a glyph and a rune What is the difference?
    - consilient 3y ago
      Characters are abstract entities, glyphs are the shapes that represent them. a and a are two distinct glyphs for the same character. "rune" is Go's name for a unicode code point.
  - mxkopy 3y ago
    The best option as always is somewhere in between and depends on context. If you’re Netflix or Walmart, localization is a high priority concern and these sort of things need to be considered. If you’re a N=50 company doing logistics for a fishing company, you probably only need to worry about your own internal Char representation that you have complete control over.
Aeolun 3y ago
Having a few big dependencies would make you more likely to check them all out individually? I can never personally check my entire 2000 package forest, but if I had only 5 I could reasonably audit them.

by rcme 3y ago

jaredsohn 3y ago

I think a benefit of a smaller number of larger packages is reduces the number of parties you need to trust. The people in charge of each larger package can review contributions made to it and have the authority to reject code that includes attacks. If a larger package doesn't do a good job of preventing attacks, its reputation will decline and because there are fewer packages involved it is easy to remember these reputations.

ljm 3y ago
It's never as simple as finding exploits in code review though. Trusted accounts are compromised or the credentials to the package manager are compromised or there is some other supply chain attack. Larger packages tend to have a lot more community and governance around them than your typical JS-style function-as-a-dependency left-pad affair, so maybe they're safer if computer accounts are gatekeeping the published artifacts rather than having individuals embed their personal keys or tokens into the publishing pipeline. I've had this debate before and, as far as dependencies go, sometimes you have to rip the bandaid off and audit your shit, and understand why exactly you outsourced something to a package.
thephyber 3y ago
That seems intuitive. I agree in the sense that smaller supply chain surface area is easier to secure/verify. But it also makes the developers/maintainers of the few packages much brighter targets. I’m more of a fan of standardizing how libraries can be rated, evaluated, trusted, marketed, etc. we need more efforts analogous to Consumer Reports / Underwriters Laboratories. So far there are a few for-profit companies in this space, including Google.

ecnahc515 3y ago

With some other languages, you have big projects that basically are your secondary stdlib. Boost in C++, Apache Commons in Java, etc. These can reduce a lot of your dependencies down to a single umbrella project who stewards everything. The number of developers who work as maintainers on the packages within these projects is probably comparable to the number of developers working on the same sorts of packages in Node, but they're all working under a single banner, effectively. It's about having dependencies within projects as organizations vs the NPM approach where basically every dependency is it's own project and organization.

noahtallen 3y ago
The JS ecosystem has moved away from massive monolithic packages due to bundle size constraints. Unfortunately, JS code has a lot of extra constraints placed on it since it has to be instantly performant over the wire. Something desktop apps don't have to deal with! jQuery and Lodash could be considered "secondary stdlib" packages. Lodash for various data functions, and jQuery for dom manipulation functions. Now that other frameworks are more common and the basic stdlib is larger, many projects have migrated away from them to improve bundle size. (As an example, pulling in one lodash function tends to pull in lots of other functions from within lodash, unnecessarily inflating the amount of code sent to the browser.)
- com2kid 3y ago
  Another advantage of this, at last for the backend, is that Node itself has a small footprint. Compared to the JVM, Node is tiny. It does some IO stuff and some eventing stuff and not much else. For the front end, ideally tree shaking is used, and even fancier build systems create code that doesn't pull down dependencies until they are actually needed at runtime.
- bathtub365 3y ago
  Why not just strip out unused code? This has existed in other toolchains for years.
  - 3y ago

CharlieDigital 3y ago

Consider a runtime like Microsoft's .NET: - The standard base class libraries with .NET are quite comprehensive; I end up rarely adding "utility" libraries - For non-standard libraries, Microsoft is a first party publisher of many packages on nuget; many of their first party frameworks and tools are published this way - For third-party libraries, generally speaking, I end up using maybe a dozen or so very well-known and well-vetted libraries which have low dependencies of their own (again, because of the rich standard libraries) - You end up with far less surface area for dependency chain attacks According to the 2020 State of the Octoverse report [0]: - Node projects had nearly an order of magnitude more transitive dependencies than the next highest (683 to 70 for PHP) (p.11) - npm had the highest percentage of critical and high severity advisories (p.14) - 72.8% of JavaScript repos had dependabot alerts (well above the weighted average of 59%) compared to 6.5% for .NET and 40.2% for Java Some would say that Microsoft's stewardship of .NET and C# is a negative; it can never be truly OSS. The flip side of that is that there is a professional team of paid software engineers -- some of the best in the world -- maintaining, patching, and securing the platform. While the third party ecosystem doesn't have as many options for doing X, the few options for doing X on .NET tend to be more well-rounded by the tighter community focus on a few, well-known libraries. Maybe some variation of an Apache Foundation like approach to curating a set of OSS NPM libraries and some mechanism of establishing trust would be beneficial to the NPM ecosystem? Just my take. [0]: https://arxiv.org/ftp/arxiv/papers/2110/2110.10246.pdf https://arxiv.org/ftp/arxiv/papers/2110/2110.10246.pdf

com2kid 3y ago
And for literal decades the 3rd party library ecosystem around .NET was anemic. Importing 3rd party libraries was an afterthought in the language, typically it involved downloading a .dll from some server, dropping it in the same folder as your project, and then manually editing some XML. Things are better now, but for all its (many!) positives, .NET is not an example of how to create a good ecosystem. Of course back then Microsoft considered an ecosystem to be 3rd party partners selling libraries, of which quite a few successful companies did.
- CharlieDigital 3y ago

mxkopy 3y ago

Normalize writing your own code. If you’re in an environment where importing GetFirstChar from an essentially random person is the norm, this is bound to happen. It seems like some developers write code only if the functionality isn’t provided by some package, when really they should be reaching for the imports only if writing the package is too much effort.

thephyber 3y ago
This is the opposite of specialization, so there are reasons to not do this. Examples: are you likely to write unit tests for your GetFirstChar? A specialized open source library that built this function is far more likely to have unit tests and documentation. Those unit tests and documentation come in handy during edge cases and when your coworker decides to extend the function with an `options` parameter. Also, general developers rarely understand the full breadth of character sets, for example. So if your function just grabs the 1st byte from a string and expects it to be a character, your end user might be angry when only the first byte of a multi byte character is returned. Similarly the difference between a character and a glyph and a rune, etc. are important and lesser understood.
- boppo1 3y ago
  >difference between a character and a glyph and a rune What is the difference?
  - consilient 3y ago
    Characters are abstract entities, glyphs are the shapes that represent them. a and a are two distinct glyphs for the same character. "rune" is Go's name for a unicode code point.
- mxkopy 3y ago
  The best option as always is somewhere in between and depends on context. If you’re Netflix or Walmart, localization is a high priority concern and these sort of things need to be considered. If you’re a N=50 company doing logistics for a fishing company, you probably only need to worry about your own internal Char representation that you have complete control over.

Aeolun 3y ago

Having a few big dependencies would make you more likely to check them all out individually? I can never personally check my entire 2000 package forest, but if I had only 5 I could reasonably audit them.