7 ms·
People really need to start thinking twice when adding a new dependency. So many supply chain attacks this year. This week, I needed to add a progress bar wit
by f311a 1y ago
People really need to start thinking twice when adding a new dependency.
So many supply chain attacks this year.
This week, I needed to add a progress bar with 8 stats counters to my Go project. I looked at the libraries, and they all had 3000+ lines of code. I asked LLM to write me a simple progress report tracking UI, and it was less than 150 lines. It works as expected, no dependencies needed. It's extremely simple, and everyone can understand the code. It just clears the terminal output and redraws it every second. It is also thread-safe. Took me 25 minutes to integrate it and review the code.
If you don't need a complex stats counter, a simple progress bar is like 30 lines of code as well.
This is a way to go for me now when considering another dependency. We don't have the resources to audit every package update.
- croes 1y agoWithout these dependencies there would be no training data so the AI can write your code
- wat10000 1y agoPart of the value proposition for bringing in outside libraries was: when they improve it, you get that automatically. Now the threat is: when they “improve” it, you get that automatically. left-pad should have been a major wake up call. Instead, the lesson people took away from it seems to have mostly been, “haha, look at those idiots pulling in an entire dependency for ten lines of code. I, on the other hand, am intelligent and thoughtful because I pull in dependencies for a hundred lines of code.”
- fluoridation 1y agoThe problem is less the size of a single dependency but the transitivity of adding dependencies. It used to be, library developers sought to not depend on other libraries if they could avoid it, because it meant their users had to make their build systems more complicated. It was unusual for a complete project to have a dependency graph more than two levels deep. Package managers let you easily build these gigantic dependency graphs with ease. Great for productivity, not so much for security.
- wat10000 1y agoThe size itself isn’t a problem, it’s just a rough indicator of the benefit you get. If it’s only replacing a hundred lines of code, is it really worth bringing in a dependency, and as you point out potentially many transitive dependencies, instead of writing your own? People understood this with left-pad but largely seemed unwilling to extrapolate it to somewhat larger libraries.
- 3036e4 1y agoYou are probably bringing in 10-1000 lines of code for every 1 line you did not have to write (I am sure some good estimate could be calculated?), since all the libraries support cases you do not need. This also tends to result in having to use APIs that are far more complex than they have to be. In addition to security risks.
- chuckadams 1y agoSo, what's the acceptable LOC count threshold for using a library? Maybe scolding and mocking people isn't a very effective security posture after all.
- tremon 1y agoScolding and mocking is all we're left with, since two decades worth of rational arguments against these types of hazards have been dismissed as fear-mongering.
- coldpie 1y ago> People really need to start thinking twice when adding a new dependency. So many supply chain attacks this year. I was really nervous when "language package managers" started to catch on. I work in the systems programming world, not the web world, so for the past decade, I looked from a distance at stuff like pip and npm and whatever with kind of a questionable side-eye. But when I did a Rust project and saw how trivially easy it was to pull in dozens of completely un-reviewed dependencies from the Internet with Cargo via a single line in a config file, I knew we were in for a bad time. Sure enough. This is a bad direction, and we need to turn back now. (We won't. There is no such thing as computer security.)
- rootnod3 1y agoFully agree. That is why I vendor all my dependencies. On the common lisp side a new tool emerged a while ago for that[1]. On top of that, I try to keep the dependencies to an absolute minimum. In my current project it's 15 dependencies, including the sub-dependencies. [1]: https://github.com/fosskers/vend https://github.com/fosskers/vend
- skydhash 1y agoVendoring is nice. Using the system version is nicer. If you can’t run on $current_debian, that’s very much a you problem. If postgres and nginx can do it, you can too.
- rootnod3 1y agoBut that would lock me in to say whatever $debian provides. And some dependencies only exist as source because they are not packaged for $distribution. Of course, if possible, just saying "hey, I need these dependencies from the system" is nicer, but also not error-free. If a system suddenly uses an older or newer version of a dependency, you might also run into trouble. In either case, you run into either an a) trust problem or b) a maintenance problem. And in that scenario I tend to prefer option b), at least I know exactly whom to blame and who is in charge of fixing it: me. Also comes down to the language I guess. Common Lisp has a tendency to use source packages anyway.
- christophilus 1y agoI’d like a package manager that essentially does a git clone, and a culture that says: “use very few dependencies, commit their source code in your repo, and review any changes when you do an update.” That would be a big improvement to the modern package management fiasco.
- hardwaregeek 1y agoThat’s called the original Go package manager and it was pretty terrible
- christophilus 1y agoI think it was only terrible because the tooling wasn't great. I think it wouldn't be too terribly hard to build a good tool around this approach, though I admittedly have only thought about it for a few minutes. I may try to put together a proof of concept, actually.
- jerf 1y agoIf you're working in Go, you don't need to put together a proof of concept. Very basic project tooling in conjunction with "go mod vendor", which takes care of copying in the dependencies in locally, will do what you're talking about. Go may not default to this operation, but using it this way is fairly easy.
- hvb2 1y agoIs that realistic though? What you're proposing is letting go of abstractions completely. Say you need compression, you're going to review changes in the compression code? What about encryption, a networking library, what about the language you're using itself? That means you need to be an expert on everything you run. Which means no one will be building anything non trivial.
- 3036e4 1y agoSmall, trivial, things, each solving a very specific problem, and that can be fully understood, sounds pretty amazing though. Much better than what we have now.
- skydhash 1y agoI actually loathe those progress trackers. They break emacs shell (looking at you expo and eas). Why not print a simple counter like: ..10%..20%..30% Or just: Uploading… Terminal codes should be for TUI or interactive-only usage.
- quotemstr 1y agoTry mistty
- sfink 1y agoCarriage returns are good enough for progress bars, and seem to work fine in my emacs shell at least: % echo -n "loading..."; sleep 1; echo -en "\rABORT ABORT"; sleep 1; echo -e "\rTerminated" works fine for me, and that's with TERM set to "dumb". (I'm actually not sure why it cleared the line automatically though. I'm used to doing "\rmessage " to clear out the previous line.) Admittedly, that'll spew a bunch of stuff if you're sending it to a pager, so I guess that ought to be % if [ -t 1 ]; then echo -n "loading..."; sleep 1; echo -en "\rABORT ABORT"; sleep 1; echo -e "\rTerminated"; fi but I still haven't made it to 15 dependencies or 200 lines of code! I don't get a full-screen progress bar out of it either, but that's where I agree with you. I don't want one.
- JdeBP 1y agoThe problem is that two pagers don't do everything that they should do in this regard. They are supposed to do things like ul utility does, but neither BSD more nor less handle when a CR is emitted to overstrike the line from the beginning. They only handle overstriking characters with BS. most handles overstriking with CR, though. Your output appears as intended when you page it with most. * https://jedsoft.org/most/ https://jedsoft.org/most/
- flexagoon 1y agoI feel like not properly supporting widely used escape codes is an issue with the shell, not with the program that uses them
- littlecranky67 1y agoWe are using NX heavily (and are not affected) in my teams in a larger insurance company. We have >10 standalone line of business apps and 25+ individual libraries in the same monorepo, managed by NX. I've toyed with other monorepo tools for these kind of complex setup in my career (lerna, rushjs, yarn workspaces) but not only did none came close, lerna is basically handed over to NX, and rushjs is unmaintained. If you have any proposal how to properly manage the complexity of a FE monorepo with dozens of daily developers involved and heavy CI/CD/Devops integration, please post alternatives - given that security incident many people are looking.
- threetonesun 1y agonpm workspaces and npm scripts will get you further than you might think. Plenty of people got along fine with Lerna, which didn't do much more than that, for years. I will say, I was always turned off by NX's core proposition when it launched, and more turned off by whatever they're selling as a CI/CD solution these days, but if it works for you, it works for you.
- crabmusket 1y agoI'd recommend pnpm over npm for monorepos. Forcing you to be explicit about each package's dependencies is good. I found npm's workspace features lacking in comparison and sparsely documented. It was also hard to find advice on the internet. I got the sense nobody was using npm workspaces for anything other than beginner articles.
- dboreham 1y agoAfter 10 years or so enduring the endless cycle of "new thing to replace npm", I'm using: npm. And I'm not creating monorepos.
- crabmusket 1y agoI was happily using npm until I outgrew it. pnpm seemed the smallest step towards what I needed after having evaluated nx, moonrepo etc.
- chrismustcode 1y agoI honestly find in go it’s easier and less code to just write whatever feature you’re trying to implement than use a package a lot of the time. Compared to typescript where it’s a package + code to use said package which always was more loc than anything comparative I have done in golang.
- legacynl 1y agoWell that's just the difference between a library and building custom. A library is by definition supposed to be somewhat generic, adaptable and configurable. That takes a lot of code.
- amelius 1y agoAnd do you know what type of code the LLM was trained on? How do you know its sources were not compromised?
- dakiol 1y agoEasier solution: you don’t need a progress bar.
- SoftTalker 1y agoEvery feature is also a potential vulnerability.
- deleted 1y ago[deleted]
- f311a 1y agoIt runs indefinitely to process small jobs. I could log stats somewhere, but it complicates things. Right now, it's just a single binary that automatically gets restarted in case of a problem.
- skydhash 1y agoWhy not print on stdout, then redirect it to a file?
- chairmansteve 1y agoOne of the wisest comments I've ever seen on HN.
- nicce 1y agoDepends on the purpose… but I guess if you replace it with estimated time left, may be good enough. Sometimes progress bar is just there to identify whether you need stop the job since it takes too much time.
- vendiddy 1y agoAnd if you really do? Print the percentage to stdout.
- cosmic_cheese 1y agoUsing languages and frameworks that take a batteries-included approach to design helps a lot here too, since you don’t need to pull in third party code or write your own for every little thing. It’s too bad that more robust languages and frameworks lost out to the import-world culture that we’re in now.
- sfink 1y agoI think something like cargo vet is the way forward: https://mozilla.github.io/cargo-vet/ https://mozilla.github.io/cargo-vet/ Yes, it's a ton of overhead, and an equivalent will be needed for every language ecosystem. The internet was great too, before it became too monetizable. So was email -- I have fond memories of cold-emailing random professors about their papers or whatever, and getting detailed responses back. Spam killed that one. Dependency chains are the latest victim of human nature. This is why we can't have nice things.
- kbrkbr 1y agoBut here's the catch. If you do that in a lot of places, you'll have a lot of extra code to manage. So your suggested approach does not seem to scale well.
- lxgr 1y agoThere's obviously a tradeoff there. At some level of complexity it probably makes sense to import (and pin to a specific version by hash) a dependency, but at least in the JavaScript ecosystem, that level seems to be "one expression of three tokens" (https://www.npmjs.com/package/is-even https://www.npmjs.com/package/is-even).
- andix 1y agonx is not a random dependency. It's a multi-project management tool, package manager, build tool, and much more. It's backed by a commercial offering. A lot of serious projects use it for managing a lot of different concerns. This is not something silly like leftpad or is-even.
- throwmeaway222 1y agoI've been saying this for a while, llms will get rid of a lot of libraries, rightly so.
- girvo 1y ago> People really need to start thinking twice when adding a new dependency I've been preaching this since ~2014 and had little luck getting people on board unless I have full control over a particular team (which is rare). The need to avoid "reinventing the wheel" seems so strong to so many.
- vendiddy 1y agoI find if I read the source code of a dependency I might add, it's common that the part that I actually need is like 100 LOC rather than 1500 LOC. Please keep preaching.
- myaccountonhn 1y agoIn pure functional programming like elm and Haskell, it is extremely easy to audit dependencies because any side effect must be explicitly listed, so you just search for those. That makes the risk way lower for dependencies, which is an underrated strength.