4 ms·
Somehow in my career I've never managed to develop the hatred for dependencies and upgrades that so many seem to have. What's the big deal?
by willsmith72 3y ago
Somehow in my career I've never managed to develop the hatred for dependencies and upgrades that so many seem to have.
What's the big deal?
- ozim 3y agoBecause with big enough project you have 2 ways: You update dependencies as a full time job and it bites you because something breaks big way. But most likely you have some help to fix it and as you are on top you might need to put work to fix it but yeah still full time job. You neglect updating dependencies and it bites you and you don’t even expect it just because you have to update something you did not expect will need it and you are now scrambling to do anything because you are messed up. Besides I like writing my own code more than dealing with some lib shenanigans.
- dspillett 3y agoOr you neglect updating them and get bitten by a missed security update, either just the rush to update (and potentially deal with breaking changes and other issues that come from not having done so in a long while) worse because it actually affects your projects instances in the wild.
- anonzzzies 3y agoI used to not have issues with Java or .net, but with npm it’s very painful; I need to pin everything as updates, even minor ones, break whole slews of things. And that’s with dependencies of dependencies of dependencies. If you are not a consulting outfit (I suspect consultancy outfits love the js ecosystem; every day something breaks so that’s another few $150/hr hours), but have a company with 100+ applications installed that actively make or help you make your bottom line, you want them to run in peace for years or decades. If no new features or security issues arrive, you don’t want to spend $0.01 on them as what’s the point of that? And that’s where this new fangled crap falls down: we have Django and .net apps running that are 10 years old, php and java ones 15-20 years old that all survived without much or any work. We have apps in js/typescript from 1 year ago that have bucketloads of deprecated, no longer supported, mandatory update warnings. It’s a pita so we are looking to move back to php or/and Django. I can see that in a startup with a dedicated team, this would be no issue; you are on top of all and working daily. Note as well that for some products (the more regulated of our products; financial/health/gov) have strict dependencies we all have to vet; in our experience of the past 4 years, it’s not a hallucination that npms change so often without any useful new features or fixes; just some useless dep updates that sometimes break things and weren’t needed, at all (you do not always need the latest version of everything ya know); I suspect (but maybe someone here knows) is that npms often update because it gives the authors more kudos somehow (as in; if posting many videos on TikTok ups your engagement; does that work for npm updates and new useless npms too?).
- holoduke 3y agoWhat are really painful if i may ask? We have some very old node express servers running. I think 20 years old. They still work after so many years. Occasionally I update them. But never really run into issues. For those old php/puthon projects. Do you really have package manager used for those? Most of those old projects are just folders with scripts without any managers.
- curtisblaine 3y agoIt's painful when you have a lot of peer dependencies in your tree and then you are forced to update one dependency for security reasons. That dependency may have updated peer dependencies that are not compatible with the existing ones, which forces you to upgrade other dependencies and so on, in a domino effect. If the dependencies you have upgraded to are not compatible with your code or, worse, you configuration, you might end up refactoring big chunks of your application. This happens a lot with the frontend ecosystem (where everything is a peer dependency of either react or webpack / rollup / whatever build system) but can happen on the backend too (where many things are plugins / middlewares of express or whatever server you're using and carry a peer dependency on express). This is made more evident by the fact that, in its latest iterations, the npm project decided to install peer dependencies by default and break if their collective ranges are not compatible (before it just warned), essentially breaking CI if peer deps are broken (I'm not saying it's a bad move, before that a lot of applications essentially failed less but more obscurely). Another issue is the fact that the Node project moves quite fastly and new versions of dependencies that you must support for security reasons may be dependant on newer Node engines. A simple dependency migration, in that case, might become a migration towards a new version of Node, that 1) in corporate environments might not be available 2) might clash with other dependencies being still dependent on earlier versions of the Node engine 3) might deprecate or remove some of the APIs you were using. All this makes sense per se: upgrading dependencies is a complex topic that shouldn't be taken lightly. What makes less sense, imho, is the extreme customizability of a Node project: millions of available small dependencies that you can combine however you like, fractally (your deps do this as well, and their deps and so on). The ecosystem would probably benefit from more monolithic dependencies that are less customizable. Rome was a project in this sense - a monolithic package of pre-configured, opinionated dev tooling - but now the cat is out of the bag. Less experienced developers will bikeshed if they can and npm is the perfect ecosystem to bikeshed your way to thousands of dependencies that are not upgraded regularly and after a couple of years become a nightmare to upgrade in bloc. That's why people are starting to write zero-deps code.
- fifticon 3y agoI manage a 4year old 'vanilla' typescript app at work. It doesnt't really do anything 'wild', looking at it you would think 'that could be built with anything/nothing'. But if you check node-modules, it contains ~1500 npm packages! Your second thought might be 'oh, its because they let themselves use a huge amount of random gizmos!'. Nope.. the actual root dependencies are relatively small in number, maybe 30-40 core dependencies. The problem is, that the prevailing attitude in the ecosystem is to 'just use dependencies', for anything, like lpad. As 'I know, we'll just use the creditcard my daddy gave me'. As long as your daddy is rich, you won't see a problem. If your daddy isnt rich, you will notice that every 10 days as you run npm-build, it will proclaim 'hooray, there are now again 10-15 high-medium security in your 1500 dependencies'. Combine this need to npm update continuously, with frontends happy disregard for typechecking and backwards compatibility, and you have a recipe for the plague,which we are currently living :-(
- ivanjermakov 3y ago> small in number, maybe 30-40 core dependencies I consider 40 to be a lot, even for a huge enterprise project. I think without major refactoring half of them can be replaced with simple utility functions.
- deleted 3y ago[deleted]
- glutamate 3y agoI think it's also a question of education. You can read these sorts of blog posts and tinker with this type of development without using it at work. A lot of web developers are confused about basic things like the request cycle and I think the use of magic easy frameworks is partly responsible (also this is stuff you don't learn in CS school). Seeing how things work without dependencies or frameworks really helps.
- solatic 3y agoIf you upgrade regularly, maintain sufficient business-logic testing to catch breakage, and regularly audit your dependencies for those which are no longer maintained so that they can be ripped out and replaced with maintained alternatives, then you're fine. If you skip any one of those, then your full-time job is longer about shipping features/value but just trying to get the damned thing to work.
- subtra3t 3y agoI think doing just a few of your stated precautions would be more than enough to provide sufficient reason to hate dependencies
- timrichard 3y agoOut of interest, I just pasted a package.json into ChatGPT and asked if any dependencies were deprecated or no longer maintained. The results looked promising, but there seems to be a cap on the time it can spend browsing. So perhaps it could be done in chunks, or maybe the API version doesn't have the same restriction (I haven't tried it). I also tried prompting to ask if any dependencies have been eclipsed by more popular libraries, which seemed to produce results. Might be worth experimenting to see if it can take some grunt work out of it. Or an automated report as a CI stage. Hmm.
- subtra3t 3y agoI'd imagine many people would be willing to pay for such a product.
- prmph 3y agoAnd what is the guarantee that a no-longer-maintained dependency will have a suitable and exact alternative? Also, tests catching breakage doesn't solve the breakage, so regular upgrades can be quite time consuming, From my experience, dependencies introduce complexity in a super-linear manner, and one should be very careful in introducing them. This is a general principle that goes beyond NPM.
- tobyhinloopen 3y agoInstalling a few basic packages in a Node project can quickly result in 1000s of packages. That's 1000s of packages you need to maintain (update), 1000s of packages that can be or become compromised, or get deleted. 1000s of packages that will spam "this package is deprecated" messages in your logs or just break because NodeJS decides that importing and requiring files should work differently again for some reason. 1000s of packages with backwards-incompatible changes that seems to happen every few years in NodeJS land. Some packages have backwards-incompatible changes and you don't even know because you're not using the package yourself and suddenly the behavior of your code changes in some subtle way you'll find out about a few weeks after upgrading. I think the problem is that NodeJS's standard library is either too barebones or nobody wants to use it, since there are libraries for all kinds of basic stuff and everyone ends up using different libraries (or different versions of the same library) for the same thing. HTTP clients, array/string manipulation and date/time handling are some notable examples.
- JodieBenitez 3y ago> 1000s of packages that will spam "this package is deprecated" messages in your logs Well... if only it was just deprecation messages... but some devs think the log is a good place for proselytism or adverts.
- marcus_holmes 3y agoI learnt this lesson dealing with Windows DLLs back in the VB days. More dependencies == less sanity. I've written Node/React web applications that break completely after a few months of not being updated, and after a year or more they need refactoring to remove deprecated/dead dependencies. And they always come with half a dozen security warnings that are impossible to get rid of. I've seen the default Go linter aggregator have to drop half the linters because no-one updated them to use generics. It's annoying, but not so much that I want to dive in and fix someone else's linter. I've spent hours learning the interface for ${useful library}, only to find they changed it in the latest release and they're not maintaining the old version. And I'm certain that the massive use of dependencies will cause a huge security problem this decade. Best account I've read of this: https://medium.com/hackernoon/im-harvesting-credit-card-numbers-and-passwords-from-your-site-here-s-how-9a8cb347c5b5 https://medium.com/hackernoon/im-harvesting-credit-card-numb... I use as few dependencies as possible these days.
- devjab 3y agoThe maintenance required, and often the poor development choices for something that can be really basic, or tailored to what you actually need. That being said, I’m not against dependencies, but I do think you should think about them quite a lot. I can’t imagine not using Typescript, eslint or ts-jest as an example. Similarly I can’t imagine interacting with many Microsoft APIs without using their SDKs (even as painful as that can be at times). Or using not using React(or similar) for the frontend. But we needed an OData client, and there were some available through NPM, but they were terrible. You could use them, but you wouldn’t have a good time. So we wrote our own, which is on NPM actually, but also a really good example of dangerous packages as we made a lot of breaking changes through the different versions as we were building it. If you had blindly used it, you would have had to put in a ton of maintenance. Something which isn’t really exclusive to the Node package environment. We have a C# application that uses RestSharp instead of the Microsoft SDKs (for God knows what reason) and it’s a few versions behind by now because they made some rather big changes that nobody wanted to spend time on. On the flip side, we’ve also used Dapper as an ORM and been very happy with that. So often it’s also a question of using dependencies where and when it makes sense. But blindly going into them is a maintenance nightmare. The less usage packages see, and the fewer maintainers, and so on, also has a big impact on whether you can trust them or not. We update quite a few of our packages automatically after a one-two week period, and others we only update manually after vetting the code. So it’s always a resource question.
- thomasfromcdnjs 3y agoNeither. I've also used pretty much every package manager out there, and this may hurt some, but I've always loved the npm ecosystem the most.
- Aeolun 3y agoThe black hole that is node_modules? Like, it’s fine for my personal projects, but every time I install the project I implicitly trust some 2000 authors to not have added “rm -rf /“ to their installation scripts. Ideally I wouldn’t have to worry about that.
- ivanjermakov 3y agoYou have to have this trust with any software you're using. Is not feasible to inspect source and install from source every package/library you use. Any of them can hypothetically run rm rf and include it even in a minor update.
- subtra3t 3y agoBut that risk is relatively small when you're using 2-3 packages that are very popular compared to 200-300 packages, most of which will be esoteric one liners.
- ivanjermakov 3y agoTrue, but only if these popular packages treat their own dependencies cautiously.
- r3trohack3r 3y agoFor me, personally, my "hatred" developed while working on a team that shipped software living in other people's process spaces. Classical examples of this are metrics clients, tracing clients, loggers, HTTP frameworks, routers, etc. Tools that are hard/impossible to spin out into a dedicated microservice but generic enough to warrant trying to share that code across many services. These dependencies can have measurable impact on the user experience. Some teams just toss the code over the wall so "it's someone else's problem." But I'd take a "every instance of my code in a production server is a production deployment" approach. Being able to test your changes against real production traffic, knowing when you've made a change that is going to break services, owning that, being able to roll it back yourself. All of that is stuff I want in my dependency manager. A dependency is more than something down inside a node_modules directory. It's a code path that executes alongside, or even directly handles incoming customer requests. Trying to track down and upgrade my library/framework/w.e. when it's deployed across 100,000+ instances for 10,000+ services is... not fun. Even less fun is trying to surgically update _only_ my code and not rebuild the entire world pulling in a huge pile of changes along with my changes. Package management failed me when I tried to model deployments as a collection of stakeholders shipping code to production. This isn't just an npm problem. This sucks in pretty much every package manager I've worked with. Don't get me started on apt... unpack an ar archive into a global namespace, run some Rube Goldberg scripts, and hope for the best. The Nix paper gives me hope, but I never got around to using Nix in production before I moved on from large-scale deployments.
- jwells89 3y agoThe thing about dependencies (regardless of ecosystem) is that it’s easy for them to spiral out of control, making it difficult to have a solid grip on all the things going on under the hood. Over time this can be a real problem because it makes tracking down breakages and fixing them that much more difficult, especially when dealing with deep fractals of dependencies. In short, they’re fine in moderation but each added is another liability. As such, it can be beneficial to take a judicious approach and only add them when the value they bring outweighs that liability.
- recursive 3y agoDifferent combinations of versions break in novel ways. They each come with their own build constraints. Some of them have bugs that go unfixed for years. It's find to submit a PR, but sometimes they're not accepting any. And it's a transitive dependency anyway, so using my fork wouldn't even help. And so on.