4 ms·

I think there’s a much stronger argument for policies that both limit the number and complexity of dependencies. Don’t add it unless it’s highly focused (no “ev

I think there’s a much stronger argument for policies that both limit the number and complexity of dependencies. Don’t add it unless it’s highly focused (no “everything libraries” that pull in entire universes of their own) and carries a high level of value. A project’s entire dependency tree should be small and clean. Libraries themselves should perhaps also take a page from the book of Linux distributions and offer LTS (long term support) releases that are feature frozen and include only security patches, which are much easier to reason about and periodically audit.

pengaru 11mo ago
I'd be willing to pay $100 to upvote your comment 100x.
- deleted 11mo ago
  [deleted]
- Y_Y 11mo ago
  How do you think dang puts bread on the table?
tcfhgj 11mo ago
Won't using highly focused dependencies increase the amount of dependencies? Limiting the number of dependencies, but then rewriting them in your own code, will also increase the maintenance burden and compile times
- skydhash 11mo ago
  A lot of projects are using dependencies, but are only using a small part of. Or are using them in a single place for a single usecase. Like bringing in formik (npm), but you only have one single form. Or moment, because you want to format a single date.
buu700 11mo ago
I think AI nudges the economics more in this direction as well. Adding a non-core dependency has historically bought short-term velocity in exchange for different long-term maintenance costs. With AI, there are now many more cases where a first-party implementation becomes cheaper/easier/faster in both the short term and the long term. Of course it's up to developers to weigh the tradeoffs and make reasonable choices, but now we have a lot more optionality. Reaching for a dependency no longer needs to be the default choice of a developer on a tight timeline/budget.
- xmodem 11mo ago
  Let's have AI generate the same vulnerable code across hundreds of projects, most of which will remain vulnerable forever, instead of having those projects all depend on a central copy of that code that can be fixed and distributed once the issue gets discovered. Great plan!
  - buu700 11mo ago
    You're attacking a straw man. No one said not to use dependencies.
    - xmodem 11mo ago
      At one stage in my career the startup I was working at was being acquired, and I was conscripted into the due-diligence effort. An external auditor had run a scanning tool over all of our repos and the team I was on was tasked with going through thousands of snippets across ~100 services and doing something about them. In many cases I was able to replace 10s of lines of code with a single function call to a dependency the project already had. In very few cases did I have to add a new dependency. But directly relevant to this discussion is the story of the most copied code snippet on stack overflow of all time [1]. Turns out, it was buggy. And we had more than once copy of it. If it hadn't been for the due diligence effort I'm 100% certain they would still be there. [1]: https://news.ycombinator.com/item?id=37674139 https://news.ycombinator.com/item?id=37674139
xmodem 11mo ago
I've seen this argument made frequently. It's clearly a popular sentiment, but I can't help feel that it's one of those things that sounds nice in theory if you don't think about it too hard. (Also, cards on the table, I personally really like being able to pull in a tried-and-tested implementation of code to solve a common problem that's also used by in some cases literally millions of other projects. I dislike having to re-solve the same problem I have already solved elsewhere.) Can you cite an example of a moderately-widely-used open source project or library that is pulling in code as a dependency that you feel it should have replicated itself? What are some examples of "everything libraries" that you view as problematic?
- skydhash 11mo ago
  Anything that pulled in chalk. You need a very good reason to emit escape sequences. The whole npm (and rust, python,..) ecosystem assumes that if it’s a tty, then it’s a full blown xterm-256color terminal. And then you need to pipe to cat or less to have sensible output. So if you’re adding chalk, that generally means you don’t know jack about terminals.
  - igregoryca 11mo ago
    Some people appreciate it when terminal output is easier to read. If chalk emits sequences that aren't supported by your terminal, then that's a deficiency in chalk, not the programs that wanted to produce colored output. It's easier to fix chalk than to fix 50,000 separate would-be dependents of chalk.
  - Dylan16807 11mo ago
    I appreciate your frustration but this isn't an answer to the question. The question is about implementing the same feature in two different ways, dependency or internal code. Whether a feature should be added is a different question.
  - zahlman 11mo ago
    In the Python world, people often enough use Rich so that they can put codes like [red] into a string that are translated into the corresponding ANSI. The end user pays several megabytes for this by default, as Rich will also pull in Pygments, which is basically a collection of lexers for various programming languages to enable syntax highlighting. They also pay for a rather large database of emoji names, a Markdown parser, logic for table generation and column formatting etc. all of which might go unused by someone who just doesn't want to remember \e[31m (or re-create the lookup table and substitution code).
igregoryca 11mo ago
Most of your supply chain attack surface is social engineering attack surface. Doesn't really matter if I use Lodash, or 20 different single-function libraries, if I end up trusting the exact same people to not backdoor my server. Of course, small libraries get a bad rap because they're often maintained by tons of different people, especially in less centralized ecosystems like npm. That's usually a fair assessment. But a single author will sometimes maintain 5, 10, or 20 different popular libraries, and adding another library of theirs won't really increase your social attack surface. So you're right about "pull[ing] in universes [of package maintainers]". I just don't think complexity or number of packages are the metrics we should be optimizing. They are correlates, though. (And more complex code can certainly contain more vulnerabilities, but that can be dealt with in the traditional ways. Complexity begets simplicity, yadda yadda; complexity that only begets complexity should obviously be eliminated)
mkoubaa 11mo ago
The lower level the dependency is, the more unjustifiable it is for it to have its own dependencies. This ought to be a point of competition between libraries and often is, at least in the c++ world

pengaru 11mo ago

I'd be willing to pay $100 to upvote your comment 100x.

deleted 11mo ago
[deleted]
Y_Y 11mo ago
How do you think dang puts bread on the table?

tcfhgj 11mo ago

Won't using highly focused dependencies increase the amount of dependencies? Limiting the number of dependencies, but then rewriting them in your own code, will also increase the maintenance burden and compile times

skydhash 11mo ago
A lot of projects are using dependencies, but are only using a small part of. Or are using them in a single place for a single usecase. Like bringing in formik (npm), but you only have one single form. Or moment, because you want to format a single date.

buu700 11mo ago

I think AI nudges the economics more in this direction as well. Adding a non-core dependency has historically bought short-term velocity in exchange for different long-term maintenance costs. With AI, there are now many more cases where a first-party implementation becomes cheaper/easier/faster in both the short term and the long term. Of course it's up to developers to weigh the tradeoffs and make reasonable choices, but now we have a lot more optionality. Reaching for a dependency no longer needs to be the default choice of a developer on a tight timeline/budget.

xmodem 11mo ago
Let's have AI generate the same vulnerable code across hundreds of projects, most of which will remain vulnerable forever, instead of having those projects all depend on a central copy of that code that can be fixed and distributed once the issue gets discovered. Great plan!
- buu700 11mo ago
  You're attacking a straw man. No one said not to use dependencies.
  - xmodem 11mo ago
    At one stage in my career the startup I was working at was being acquired, and I was conscripted into the due-diligence effort. An external auditor had run a scanning tool over all of our repos and the team I was on was tasked with going through thousands of snippets across ~100 services and doing something about them. In many cases I was able to replace 10s of lines of code with a single function call to a dependency the project already had. In very few cases did I have to add a new dependency. But directly relevant to this discussion is the story of the most copied code snippet on stack overflow of all time [1]. Turns out, it was buggy. And we had more than once copy of it. If it hadn't been for the due diligence effort I'm 100% certain they would still be there. [1]: https://news.ycombinator.com/item?id=37674139 https://news.ycombinator.com/item?id=37674139

xmodem 11mo ago

I've seen this argument made frequently. It's clearly a popular sentiment, but I can't help feel that it's one of those things that sounds nice in theory if you don't think about it too hard. (Also, cards on the table, I personally really like being able to pull in a tried-and-tested implementation of code to solve a common problem that's also used by in some cases literally millions of other projects. I dislike having to re-solve the same problem I have already solved elsewhere.) Can you cite an example of a moderately-widely-used open source project or library that is pulling in code as a dependency that you feel it should have replicated itself? What are some examples of "everything libraries" that you view as problematic?

skydhash 11mo ago
Anything that pulled in chalk. You need a very good reason to emit escape sequences. The whole npm (and rust, python,..) ecosystem assumes that if it’s a tty, then it’s a full blown xterm-256color terminal. And then you need to pipe to cat or less to have sensible output. So if you’re adding chalk, that generally means you don’t know jack about terminals.
- igregoryca 11mo ago
  Some people appreciate it when terminal output is easier to read. If chalk emits sequences that aren't supported by your terminal, then that's a deficiency in chalk, not the programs that wanted to produce colored output. It's easier to fix chalk than to fix 50,000 separate would-be dependents of chalk.
- Dylan16807 11mo ago
  I appreciate your frustration but this isn't an answer to the question. The question is about implementing the same feature in two different ways, dependency or internal code. Whether a feature should be added is a different question.
- zahlman 11mo ago
  In the Python world, people often enough use Rich so that they can put codes like [red] into a string that are translated into the corresponding ANSI. The end user pays several megabytes for this by default, as Rich will also pull in Pygments, which is basically a collection of lexers for various programming languages to enable syntax highlighting. They also pay for a rather large database of emoji names, a Markdown parser, logic for table generation and column formatting etc. all of which might go unused by someone who just doesn't want to remember \e[31m (or re-create the lookup table and substitution code).

igregoryca 11mo ago

Most of your supply chain attack surface is social engineering attack surface. Doesn't really matter if I use Lodash, or 20 different single-function libraries, if I end up trusting the exact same people to not backdoor my server. Of course, small libraries get a bad rap because they're often maintained by tons of different people, especially in less centralized ecosystems like npm. That's usually a fair assessment. But a single author will sometimes maintain 5, 10, or 20 different popular libraries, and adding another library of theirs won't really increase your social attack surface. So you're right about "pull[ing] in universes [of package maintainers]". I just don't think complexity or number of packages are the metrics we should be optimizing. They are correlates, though. (And more complex code can certainly contain more vulnerabilities, but that can be dealt with in the traditional ways. Complexity begets simplicity, yadda yadda; complexity that only begets complexity should obviously be eliminated)

mkoubaa 11mo ago

The lower level the dependency is, the more unjustifiable it is for it to have its own dependencies. This ought to be a point of competition between libraries and often is, at least in the c++ world