9 ms·
> It's not feasible for me to audit every single one of my dependencies, and every one of my dependencies' dependencies I think this is a good argument for red
by gameman144 1y ago
> It's not feasible for me to audit every single one of my dependencies, and every one of my dependencies' dependencies
I think this is a good argument for reducing your dependency count as much as possible, and keeping them to well-known and trustworthy (security-wise) creators.
"Not-invented-here" syndrome is counterproductive if you can trust all authors, but in an uncontrolled or unaudited ecosystem it's actually pretty sensible.
- Ajedi32 1y agoIf it's not feasible to audit every single dependency, it's probably even less feasible to rewrite every single dependency from scratch. Avoiding that duplicated work is precisely why we import dependencies in the first place.
- curtisf 1y agoThis is true to the extent that you actually _use_ all of the features of a dependency. You only need to rewrite what you use, which for many (probably most) libraries will be 1% or less of it
- zahlman 1y agoIndeed. About 26% of the disk space for a freshly-installed copy of pip 25.2 for Python 3.13 comes from https://pypi.org/project/rich/ https://pypi.org/project/rich/ (and its otherwise-unneeded dependency https://pypi.org/project/Pygments/ https://pypi.org/project/Pygments/), "a Python library for rich text and beautiful formatting in the terminal", hardly any of the features of which are relevant to pip. This is in spite of an apparent manual tree-shaking effort (mostly on Pygments) — a separate installed copy of rich+Pygments is larger than pip. But even with that attempt, for example, there are hundreds of kilobytes taken up for a single giant mapping of "friendly" string names to literally thousands of emoji. Another 20% or more is https://pypi.org/project/requests/ https://pypi.org/project/requests/ and its dependencies — this is an extremely popular project despite that the standard library already provides the ability to make HTTPS connections (people just hate the API that much). One of requests' dependencies is certifi, which is basically just a .pem file in Python package form. The vendored requests has not seen any tree-shaking as far as I can tell. This sort of thing is a big part of why I'll be able to make PAPER much smaller.
- AlecBG 1y agoNot sure I completely agree as you often use only a small part of a library
- lukan 1y ago"rewrite every single dependency from scratch" No need to. But also no need to pull in a dependency that could be just a few lines of own (LLM generated) code.
- brianleb 1y ago>>a few lines of own (LLM generated) code. ... and now you've switched the attack vector to a hostile LLM.
- zelphirkalt 1y agoThough you will see the code at least, when you are copy pasting it and if it is really only a few lines, you may be able to review it. Should review it of course.
- appreciatorBus 1y agoSure but that's a one time vector. If the attacker didn't infiltrate the LLM before it generated the code, then the code is not going to suddenly go hostile like an npm package can.
- lukan 1y agoI did not say to do blind copy paste. A few lines of code can be audited.
- gameman144 1y agoIt isn't feasible to audit every line of every dependency, just as it's not possible to audit the full behavior of every employee that works at your company. In both cases, the solution is similar: try to restrict access to vital systems only to those you trust,so that you have less need to audit their every move. Your system administrators can access the server room, but the on-site barista can't. Your HTTP server is trusted enough to run in prod, but a color-formatting library isn't.
- autoexec 1y ago> It isn't feasible to audit every line of every dependency, just as it's not possible to audit the full behavior of every employee that works at your company. Your employees are carefully vetted before hiring. You've got their names, addresses, and social security numbers. There's someone you're able to hold accountable if they steal from you or start breaking everything in the office. This seems more like having several random contractors who you've never met coming into your business in the middle of night. Contractors that were hired by multiple anonymous agencies you just found online somewhere with company names like gkz00d or 420_C0der69 who you've also never even spoken to and who have made it clear that they can't be held accountable for anything bad that happens. Agencies that routinely swap workers into or out of various roles at your company without asking or telling you, so you don't have any idea who the person working in the office is, what they're doing, or even if they're supposed to be there. "To make thing easier for us we want your stuff to require the use of a bunch of code (much of which does things you don't even need) that we haven't bothered looking at because that'd be too much work for us. Oh, and third parties we have no relationship with control a whole bunch of that code which means it can be changed at any moment introducing bugs and security issues we might not hear about for months/years" seems like it should be a hard sell to a boss or a client, but it's sadly the norm. Assuming that something is going to go wrong and trying to limit the inevitable damage is smart, but limiting the amount of untrustworthy code maintained by the whims of random strangers is even better. Especially when the reasons for including something that carries so much risk is to add something trivial or something you could have just written yourself in the first place.
- 1y ago
- zelphirkalt 1y agoMost dependencies do much more than we need from them. Often it means we only need one or a few functions from them. This means one doesn't need to rewrite whole dependencies usually. Don't use dependencies for things you can trivially write yourself, and use them for cases where it would be too much work to write yourself.
- btown 1y agoA brief but important point is that this primarily holds true in the context of rewriting/vendoring utilities yourself, not when discussing importing small vs. large dependencies. Just because dependencies do a lot more than you need, doesn't mean you should automatically reach for the smallest dependency that fits your needs. If you need 5 of the dozens of Lodash functions, for instance, it might be best to just install Lodash and let your build step shake out any unused code, rather than importing 5 new dependencies, each with far fewer eyes and release-management best practices than the Lodash maintainers have.
- jay_kyburz 1y agoYes, fewer, larger, trustworthy dependencies with tree shaking is the way to go if you ask me.
- _puk 1y agoAlmost like a standard library..
- jay_kyburz 1y agoYeah, but perhaps we could have different flavors. If you like functional style you could have a very functional standard library that doesn't mutate anything, or if you like object oriented stuff you could have classes of object with methods that mutate themselves. And the Typescript folks could have a strongly typed library.
- baq 1y ago
- bennyg 1y agoSounds like the job for an LLM tool to extract what's actually used from appropriately-licensed OSS modules and paste directly into codebases.
- philipwhiuk 1y agoDo you have any evidence it wouldn't just make up code.
- shakna 1y agoRequiring you to audit both security and robustness on the LLM generated code. Creating two problems, where there was one.
- bennyg 1y agoI didn't say generate :) - in all seriousness, I think you could reasonably have it copy the code for e.g. lodash.merge() and paste it into your codebase without the headaches you're describing. IMO, this method would be practical for a majority of npm deps in prod code. There are some I'd want to rely on the lib (and its maintenance over time), but also... a sort function is a sort function.
- shakna 1y agoLLMs don't copy and paste. They ingest and generate. The output will always be a generated something.
- kristianbrigman 1y agoOne interesting side effect of AI is that it makes it sometimes easy to just recreate the behavior, perhaps without even realizing it..
- reaperducer 1y agoit's probably even less feasible to rewrite every single dependency from scratch. When you code in a high-security environment, where bad code can cost the company millions of dollars in fines, somehow you find a way. The sibling commenter is correct. You write what you can. You only import from trusted, vetted sources.
- 8note 1y agois it that infeasible with LLMs? a lor of these dependencies are higher order function definitions, which never change, and could be copy/pasted around just fine. they're never gonna change
- motorest 1y ago> If it's not feasible to audit every single dependency, it's probably even less feasible to rewrite every single dependency from scratch. There is no need to rewrite dependencies. Sometimes it just so happens that a project can live without outputting fancy colorful text to stdout, or doesn't need to spread transitive dependencies on debug utilities. Perhaps these concerns should be a part of the standard library, perhaps these concerns are useless. And don't get me started on bullshit polyfill packages. That's an attack vector waiting to be exploited.
- smrtinsert 1y agoIts much more feasible these days. These days for my personal projects I just have CC create only a plain html file with raw JS and script links.
- EGreg 1y agoExactly. I always tried to keep the dependencies to a minimum. Another thing you can do is lock versions to a year ago (this is what linux distros do) and wait for multiple audits of something, or lack of reports in the wild, before updating.
- gameman144 1y agoI saw one of those word-substition browser plugins a few years back that swapped "dependency" for "liability", and it was basically never wrong. (Big fan of version pinning in basically every context, too)
- j1elo 1y agoI'm re-reading all these previous comments, replacing "dependency" for "liability" in my mind, and it's being quite fun to see how well everything still keeps meaning the same, but better
- 2muchcoffeeman 1y agoHave we all forgotten the left-pad incident? This is an eco system that has taken code reuse to the (unreasonable) extreme. When JS was becoming popular, I’m pretty sure every dev cocked an eyebrow at the dependency system and wondered how it’d be attacked.
- zelphirkalt 1y ago> This is an eco system that has taken code reuse to the (unreasonable) extreme. Not even that actually. Actually the wheel is reinvented over and over again in this exact ecosystem. Many packages are low quality, and not even suitable to be reused much.
- wongarsu 1y agoThe perfect storm of on the one side junior developers who are afraid of writing even trivial code and are glad if there's a package implementing functionality that can be done in a one-liner, and on the other side (often junior) developers who want to prove themselves and think the best way to do that is to publish a successful npm package
- bobthepanda 1y agoThe blessing and curse of frontend development is that there basically isn't a barrier to entry given that you can make some basic CSS/JS/HTML and have your browser render it immediately. There's also the flavor of frontend developer that came from the backend and sneers at actually having to learn frontend because "it's not real development"
- pxc 1y ago> There's also the flavor of frontend developer that came from the backend and sneers at actually having to learn frontend because "it's not real development" What kind of code does this developer write?
- 1y ago
- umvi 1y ago"A little copying is better than a little dependency" -- Go proverb (also applies to other programming languages)
- respondo2134 1y ago>> and keeping them to well-known and trustworthy (security-wise) creators. The true threat here isn't the immediate dependency though, it's the recursive supply chain of dependencies. "trustworthy" doesn't make any sese either when the root cause is almost always someone trustworthy getting phished. Finally if I'm not capable of auditing the dependencies it's unlikely I can replace them with my own code. That's like telling a vibe coder the solution to their brittle creations is to not use AI and write the code themselves.
- deleted 1y ago[deleted]
- autoexec 1y ago> Finally if I'm not capable of auditing the dependencies it's unlikely I can replace them with my own code. That's like telling a vibe coder the solution to their brittle creations is to not use AI and write the code themselves. In both cases, actually doing the work and writing a function instead of adding a dependency or asking an AI to write it for you will probably make you a better coder and one who is better able to audit code you want to blindly trust in the future.
- user34283 1y agoJust like it's going to make you a better engineer if you design the microchips in your workstation yourself instead of buying an x86 CPU. It's still neither realistic nor helpful advice.
- motorest 1y ago> I think this is a good argument for reducing your dependency count as much as possible, and keeping them to well-known and trustworthy (security-wise) creators. I wonder to which extent is the extreme dependency count a symptom of a standard library that is too minimalistic for the ecosystem's needs. Perhaps this issue could be addressed by a "version set" approach to bundling stable npm packages.
- DrewADesign 1y agoI remember people in the JS crowd getting really mad at the implication that this all was pretty much inevitable, like 10/15 years ago. Can’t say they didn’t do great things since then, but it’s not like nobody saw this coming.
- silon42 1y agoIMO, one thing I like in npm packages is that that usually they are small, and they should ideally converge towards stability (frozen)... If they are not, something is bad and the dependency should be "reduced" if at all possible.
- tjpnz 1y agoEasier said than done when your ecosystem of choice took the Unix philosophy of doing one thing well, misinterpreted it and then drove it off a cliff. The dependency tree of a simple Python service is incomparable to a Node service of similar complexity.