7 ms·

I am beginning to think that the terrible situation with dependency management in traditional C and C++ is a good thing. Now, with systems like npm, maven or c

by GuB-42 1y ago

I am beginning to think that the terrible situation with dependency management in traditional C and C++ is a good thing. Now, with systems like npm, maven or cargo, all you need to do to get a package is to add a line in a configuration file, and it fetches all the dependencies you need automatically from a central repository. Very convenient, however, you can quickly find yourself with 100+ packages from who knows where and 100s of MB of code. In C, traditionally, every library you include requires some consideration. There is no auto-download, and the library the user has may be a different version from the one you worked with, and you have to accommodate it, and so does the library publisher. Or you may have to ship is with your own code. Anyways, it is so messy that the simplest solution is often not to use a library at all and write the thing yourself, or even better, realize that you don't need the feature you would have used that library for. Bad reason, and reinventing the wheel comes with its own set of problems, but at least, the resulting code is of a manageable size.

klysm 1y ago
Unfortunately that comes with the baggage of terrible memory safety. I do agree with the sentiment though, that deps should be taken with more consideration.
- codr7 1y ago
  Given how much critical software is written in C, and the number of problems we run into; I don't see a reason to keep repeating that line outside of the Rust marketing department. Some people will always prefer C to Rust, might as well learn to live with that fact.
  - klysm 1y ago
    I never mentioned rust. I’m just saying C and C++ have terrible memory safety.
    - codr7 1y ago
      And what's the alternative then, from your perspective? What did you have in mind when you wrote the comment?
      - klysm 1y ago
        I had no alternative in mind. The topic at hand is security and bloat, and C/C++ might be leaner apps in practice but they are generally going to have memory safety bugs which is a security problem.
        codr7 1y ago
        It is, but there are very good tools and plenty of experience with dealing with the problem. It's been blown way the fuck out of proportion lately by the Rust mob.
  - udev4096 1y ago
    Remember how cloudflare (in 2017) leaked pretty much everyone's secret tokens in search engine cache due to a simple buffer overflow? Yeah, that wouldn't have happened with Rust
    - codr7 1y ago
      Yeah I know, if only we could rewrite the entire world in Rust everything would be rainbows and unicorns. But it's not going to happen, deal with it.
    - lelanthran 1y ago
      Remember that the most expensive exploit the world has ever seen was in a memory safe GC language? My argument is that you are missing the point: the point is that a larger attack surface enables more exploits regardless of language. When using a language that has tremendous friction in expanding the attack surface you tend to have a small attack surface as a result. Theres obviously a crossover point where you'd be safer with a memory safe language and a larger attack surface than with a memory unsafe language and a minuscule attack surface.
      - lmm 1y ago
        > Remember that the most expensive exploit the world has ever seen was in a memory safe GC language? No I don't, which exploit are you talking about? The most expensive exploit I can think of was caused by heartbleed which was in a memory unsafe language. The "most expensive software bug" (not an exploit) caused by turning off the safe overflow handler in the language being used can hardly be considered an indictment of language level safety either. So what exploit are you talking about?
        throw1111221 1y ago
        Not the person you replied to, but they're probably talking about Log4j. It's a Java logging library that had a helpful feature where logging a special format string would pull code from a remote URL and execute it. So anywhere you can get a Java server to log something you can run arbitrary code. (Ex: by setting a malicious User-Agent.) Estimates say 93% of enterprise cloud environments where affected. I suppose Stuxnet could also count, where the initial infection depends on the human curiosity of plugging an unknown usb drive into an air gapped system.
- privong 1y ago
  > Unfortunately that comes with the baggage of terrible memory safety. Isn't this unrelated to the parent post's thoughts about the benefit's of the C/C++ ecosystem (or lack thereof) for dependency management? I.e., a Rust-like language could still exist with a dependency management system similar to what C/C++ have now -- that isn't predicated on how the language handles memory.
deleted 1y ago
[deleted]
deleted 1y ago
[deleted]
grg0 1y ago
This is something that I think about constantly and I have come to the same conclusion. While the idea of being able to trivially share code worldwide is appealing, so far it seems to encourage shittier software more than anything else, and the benefit of sharing trivially seems to be defeated by the downsides that bloat and bad software bring with it. Adding friction to code re-use (by means of having to manually download shit from a website and compile it yourself like it's 1995) seems to be a good thing for now until a better package management system is figured out. The friction forces you to think seriously where you actually need that shit or you can write the subset of the functionality you need yourself. To be clear, I also think C++ projects suffer a lot from re-inventing the wheel, particularly in the gamedev world, but that seems to be less worse than, e.g., initializing some nodejs framework project and starting with 100+ dependencies when you haven't even started to write shit.
- pixl97 1y ago
  When doing SBOM/SCA we see apps with 1000+ deps. It's insane. It's so often we see large packages pulled in because a single function/behavior is needed and ends up massively increasing the risk profile.
  - deleted 1y ago
    [deleted]
  - 1over137 1y ago
    Holy cow. What domain is this? Web-based probably?
    - pixl97 1y ago
      Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system.
      - thewebguyd 1y ago
        > Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system. You can, but I think this thread speaks volumes about the problem with the JavaScript/NPM ecosystem as a whole vs. pretty much any other. We need something else for the web. The only reason we have 200+ NPM packages for a blank project is because JavaScript is atrocious and has almost nothing built-in. We got crap like LeftPad, isodd, is-array, etc. because of the language. Most of what NPM will pull in on any new web front end project is likely already part of the standard library in C#/dotnet, or Java, Go, etc. But you could go further back and say it's not javascript's fault, it's the fault of trying to hammer the web into doing things it was never designed to do in the first place. But, we insisted on making it an application delivery platform, and now we're suffering the consequences of that. I'm hopeful for WASM, but ideally I'd love to see a resurgence of native apps.
ChrisSD 1y ago
In my experience every developer, company, team, sub-team, etc has their own "library" of random functions, utilities, classes, etc that just end up being included into new projects sooner or later (and everyone and their dog has their own bespoke string handling libraries). Copy/pasting large chunks of code from elsewhere is also rampant. I'm not so sure C/C++ solves the actual problem. Only sweeps it under a carpet so it's much less visible.
- achierius 1y ago
  It definitely does solve one problem. Like it or not, you can't be hit by supply chain attacks if you don't have a supply chain.
  - dgfitz 1y ago
    I mirror all deps locally and only build from the mirror. It isn’t an issue. C/C++ is my dayjob
    - procaryote 1y ago
      at some point you could mirror a supply chain attack... xz was a pretty long game and only found by accident for example
      - dgfitz 1y ago
        I’m sure I will.
    - josephg 1y ago
      This runs the risk of shipping C/C++ libraries with known vulnerabilities. How do you keep track of that? At least with npm / cargo / etc, updating dependencies is a single command away.
      - dgfitz 1y ago
        Pull, update, build?
socalgal2 1y ago
100, ha! The official rust docs, built in rust, use ~750 dependencies - queue the apoligists
nradov 1y ago
There are no absolute good or bad reasons here, it depends on the problem domain and usage environment. If you're writing code where safety or security matters then of course you need to carefully manage the software supply chain. On the other hand, if you're writing an internal utility for limited use with no exposure then who cares, pull in all the dependencies you need and git 'er done.
udev4096 1y ago
Go has the most lean and simple dependency management. It's far better than npm or pypi dumpster fire
- watermelon0 1y ago
  It's also worth mentioning the extensive standard library and golang.org/x/, which means that you generally don't even need that many 3rd party packages.
  - udev4096 1y ago
    Also the extensive measures to combat supply chain security for packages [0] [0] - https://go.dev/blog/supply-chain https://go.dev/blog/supply-chain
- guappa 1y ago
  Go is a dumpster fire as well as those. edit: lol at the downvotes. Go developers showing how insecure they are once again.
  - staunton 1y ago
    Since you're apparently interested in downvotes (why?), I'm pretty sure it's not due to criticism of Go but rather the fact that your criticism is entirely non-specific and therefore doesn't add anything to the discussion...
    - guappa 1y ago
      Because the comment I replied to was so specific? There's plenty of perfectly good libraries on npm and pypi, and there's awful ones. Likewise for go which pulls from "the internet". Must I really demonstrate that bad code exists in go? You want examples? There's plenty of bad libraries in go, and pinning to a commit is a terrible practice in any language. Encourages unstable APIs and unfixable bugs.
    - lmm 1y ago
atoav 1y ago
Bloat might be correlated with the ease of bloating software and it indeed easier to do precisely that if you don't have to write it yourself. Bloat is uncontrolled complexity and making it harder to manage complexity reduces bloat. But it also makes it harder to write software that has to be complex for legitimate reasons. Not everybody should write their own library handling SSL, SQL or regex for example. But those libraries are rarely the problem, things like leftpad are. Or: you can use package systems for good and for evil. The only real way to fight bloat is to be diciplined an vet your dependencies. It must cost you something to pull them in. If you have to read and understand everything you pull in, pulling in everybody and their dog suddenly becomes less desireable. Also I think this is much more an issue off the quality of dependencies than it is about using dependencies themselves (it would be stupid to write 1000 implementations of HTTP for a language, one that works really well is better).
- RetroTechie 1y ago
  > But it also makes it harder to write software that has to be complex for legitimate reasons. Might have stolen this quote somewhere, but imho: Simple things should be easy, complex things should be possible. Related: software (binary) size should reflect the complexity of the problem domain. Some time ago, ran down the size of apps on my phone. Smallest one? ~2MB. What does that app do? Calculate some hash on a file. Select a file, it does its thing, shows the hash (and/or copy to clipboard). What the ..!?!#$ 2,000,000+ bytes for that? This is on Android, 'batteries included'. Selecting / opening a file should be a couple (or couple dozen) lines of source code, a function call to the OS, and presto. Same with reading file contents, and display output / clipboard copy. Which leaves... computing the hash. I'm not an expert, but what hash functions are so complex that you'd need a MB+ of code to calculate? (answer: none). Note that this app was the least worse offender. Conclusion: Android app model is broken. Or SDKs used to build Android apps are crap. Or other reasons / some combination thereoff. Regardless, ~2MB to compute a file hash is ridiculous. Full-blown Graphical User Interfaces (GUI) have been done in less. I'd be interested to know what that 2MB consists of, though. And where the hash function is at. And what (minute) % of overall binary size. And what all the rest of that binary does.
account-5 1y ago
I'm not a professional Dev but thought this is was tree-shaking is about? Certainly this happens in flutter, whatever you feel about flutter/dart. Or is this a sticking plaster? Genuinely don't know as I only develop personal projects.
- victorNicollet 1y ago
  Tree-shaking is able to remove code that will never be called. And it's not necessarily good at it: we can detect some situations where a function is never called, and remove that function, but it's mostly the obvious situations such as "this function is never referenced". It cannot detect a case such as: if the string argument to this function contains a substring shaped like XYZ, then replace that substring with a value from the environment variables (the Log4j vulnerability), or from the file system (the XML Entity Extension vulnerability). From the point of view of tree-shaking, this is legitimate code that could be called. This is the kind of vulnerable bloat that comes with importing large libraries (large in the sense of "has many complex features", rather than of megabytes).
  - account-5 1y ago
    Thanks for the explanations, much appreciated. I suppose the options are then: 1. Write everything yourself, time consuming and hard, less likely to lead to these types of vulnerabilities. 2. Import others code, easy and takes no time, can lead to vulnerabilities. 3. Use others code, but only what you actually need. Maybe less time consuming than 1 but more than 2, adds a different sort of complexity, done correctly less likely to lead to these vulnerabilities. Not sure if there's any other options here?
    - victorNicollet 1y ago
      I would say 4. grab individual code files (as opposed to entire libraries) and manually edit them, removing unnecessary features and adding new ones where needed.
ozim 1y ago
Writing everything from scratch by hand is an insane take. It is not just reinventing the wheel but there are whole frameworks one should use because writing that thing on your own will take you a lifetime. Yes you should not just pull as dependency thing that kid in his parents basement wrote for fun or to get OSS maintainer on his CV. But there are tons of legitimate libraries and frameworks from people who are better than you at that specific domain.
- actionfromafar 1y ago
  I feel you are arguing a bit of a strawman. The take is much more nuanced than write everything from scratch.
  - ozim 1y ago
    ... simplest solution is often not to use a library at all and write the thing yourself, or even better, realize that you don't need the feature you would have used that library for ... the resulting code is of a manageable size.. I don't see the nuance there, that is my take of the comment, those are pretty much strongest statements and points about using libraries are minimal. That is why I added mine strongly pointing that real world systems are not going to be "managable size" unless they are really small or a single person is working on the.
    - actionfromafar 1y ago
      For me "realize that you don't need the feature" is strong and also hits home. I sometimes prototype in C because it makes me think really hard about "what does this thing really have to do? What can I omit for now?" While in for instance C# I tend to think "this would be simple to implement with whatever-fancy-thing-is-just-a-package-away". Neither way is impossible to judge as good or bad on its own. A real world system is almost always part of a larger system or system of systems. Making one thing simple can make another complex. The world is messy.
- barrkel 1y ago
  That's not how it works. Here's a scenario. You pull in some library - maybe it resizes images or something. It in turn pulls in image decoders and encoders that you may or may not need. They in turn pull in metadata readers, and those pull in XML libraries to parse metadata, and before you know it a fairly simple resize is costing you 10s of MB. Worse, you pull in different libraries and they all pull in different versions of their own dependencies, with lots of duplication of similar but slightly different code. Node_modules usually ends up like this. The point is not writing the resize code yourself. It's the cultural effect of friction. If pulling in the resize library means you need to chase down the dependencies yourself, first, you're more aware of the cost, and second, the library author will probably give you knobs to eliminate dependencies. Perhaps you only pull in a JPEG decoder because that's all you need, and you exclude the metadata functionality. It's an example, but can you see how adding friction to pulling in every extra transitive dependency would have the effect of librabry authors giving engineers options to prune the dependency tree? The easier a library is to use, the more popular it will be, and a library that has you chasing dependencies won't be easy to use.
SkiFire13 1y ago
How many vulnerabilities were due to badly reinventing the wheel in C/C++ though? Also, people often complain about "bloat", but don't realize that C/C++ are often the most bloated ones precisely because importing libraries is a pain, so they try to include everything in a single library, even though you only need to use less than 10% of it. Look for example at Qt, it is supposed to be a UI framework but it ends up implementing vectors, strings, json parser and who knows how much more stuff. But it's just 1 dependency so it's fine, right?
- reaperducer 1y ago
  How many vulnerabilities were due to badly reinventing the wheel in C/C++ though? I don't know. Suppose you tell us.
- phkahler 1y ago
  >> Look for example at Qt, it is supposed to be a UI framework but it ends up implementing vectors, strings, json parser and who knows how much more stuff. But it's just 1 dependency so it's fine, right? Qt is an application development framework, not a GUI toolkit. This is one reason I prefer GTK (there are things I dislike about it too).
  - r0ze-at-hn 1y ago
    I remember back in the early 2000's that discussion, but now with the the tonnage that systems like npm can pull in I laugh that we ever thought it wouldn't get worse.
- GuB-42 1y ago
  There is still an advantage to using Qt over dozens of libraries that offer the same functionality. Qt is backed by a single company, so all you have to watch out for is that company. Also, Qt is generally high quality, I have worked with it, read the source code, etc... and I generally liked what I saw. So I can reasonably assume that quality is consistent overall. When you have many libraries from many independent developers, it doesn't work. The JSON parser may be good, but it doesn't tell me anything about the library that deal with internationalization for instance, and if I wanted to keep track of everything, that's several time the work compared to a single vendor. I agree that Qt is bloated though, but multiplatform UI frameworks are hard to keep light. There is a lot going on in a desktop UIs that people only notice when it isn't there. I tend to treat them like I treat the standard libraries, the OS, and for web apps, the browser. Big components, but you reasonably can't do without.
otikik 1y ago
I thought about this several years ago and I think I hit the right balance with these 2 rules of thumb: * The closer something is to your core business, the less you externalize. * You always externalize security (unless security is your exclusive core business) Say you are building a tax calculation web app. You use dependencies for things like the css generation or database access. You do not rely on an external library for tax calculation. You maintain your own code. You might use an external library for handling currencies properly, because it's a tricky math problem. But you may want to use your own fork instead, as it is close to your core business. On the security side, unless that's your speciality, there's guys out there smarter than you and/or who have dedicated more time and resources than you to figure that stuff out. If you are programming a tax calculation web app you shouldn't be implementing your own authentication algorithm, even if having your tax information secure is one of your core needs. The exception to this is that your core business is literally implementing authentication and nothing else.
- the__alchemist 1y ago
  I would like to dig into point 2 a bit. Do you think this is a matter of degree, or of kind? Does security, in this, imply a network connection, or some other way that exposes your application to vulnerabilities, or is it something else? Are there any other categories that you would treat in a similar way as security, but to a lesser degree, or that almost meet that threshold for a special category, but don't?
- cogman10 1y ago
  I think this helps, but I also think the default for any dev (particularly library authors) should be to minimize dependencies as much as possible. Dependencies have both a maintenance and a security cost. Bad libraries have deep and sprawling trees. I've seen devs pull in frameworks just to get access to single simple to write functions.
- pphysch 1y ago
  There have been major F-ups in recent history with Okta, CrowdStrike, and so on. Keycloak had some major long-standing vulnerabilities. I've had PRs accepted in popular open-source IAM libraries a bit too easily. Yeah, we shouldn't roll our own cryptography, but security isn't as clean cut as this comment implies. It also frequently bleeds into your business logic. Don't confuse externalizing security with externalizing liability.
BrouteMinou 1y ago
When you "Reinvent the wheel", you implement only what you need in an optimized way. This gives a couple of advantages: you own your code, no bloat, usually simpler due to not having all the bells and whistles, less abstraction, so faster because there is no free lunch, minimize the attack surface for supply chain attacks... For fun, the next time you are tempted to install a BlaZiNg FaSt MaDe in RuSt software: get the source, install cargo audit and run the cargo audit on that project. See how many vulnerabilities there are. So far, in my experience, all the software I checked come with their list of vulnerabilities from transitive dependencies. I don't know about npm, I only know by reputation and it's enough for me to avoid.
- nebula8804 1y ago
  That wheel is only as good as your skill in making it. For many people (the majority i'd guess) someone else making that wheel will have a better end result.
  - doublerabbit 1y ago
    The skill is produced by carving the wheel. You've got to start somewhere. Whether a mess or not the returned product is a product of your own. By relying on dependencies you're forever reaching for a goal you'll never achieve.
jajko 1y ago
Yeah everybody should reimplement their own security for example, that's a really smart fool-proof approach especially down the line, no real cases for any contrarian opinions. I do get what you mean, but it works only on some very specific types of projects, when you & potentially comparably (very) good & skilled peers are maintaining and evolving it long term. This was never the case in my 20 years of dev career. This sort of shared well tested libraries -> gradually dependency hell is in some form shared across all similar languages since its pretty basic use case of software development as an engineering discipline. I haven't seen a good silver bullet so far, and ie past 14 years of my work wouldn't be possible with approach you describe.
matheusmoreira 1y ago
> There is no auto-download There is. Linux distributions have package managers whose entire purpose is to distribute and manage applications and their dependencies. The key difference between Linux distribution package managers and programming language package managers is the presence of maintainers. Any random person can push packages to the likes of npm or PyPI. To push packages to Debian or Arch Linux, you must be known and trusted. Programming language package managers are made for developers who love the convenience of pushing their projects to the world whenever they want. Linux distribution package managers are made for users who prefer to trust the maintainers not to let malware into the repositories. Some measured amount of elitism can be a force for good.
reaperducer 1y ago
Now, with systems like npm, maven or cargo, all you need to do to get a package is to add a line in a configuration file They can't hack what doesn't exist. Reducing surface area is sometimes the easiest security measure one can take.
hinkley 1y ago
Within reason, we need to be able to promote third party libraries into the standard library. A small standard library pairs well with an easy mechanism to download code, but at some point it's probably a crutch. There are maybe 5 functions in lodash at this point that show up routinely in production code but cannot be sufficed by existing editions to EcmaScript - sortBy, recursive get, and recursive merges being among the most useful. We could just have these and be done.
buggerme 1y ago
[dead]

by GuB-42 1y ago

klysm 1y ago

Unfortunately that comes with the baggage of terrible memory safety. I do agree with the sentiment though, that deps should be taken with more consideration.

codr7 1y ago
Given how much critical software is written in C, and the number of problems we run into; I don't see a reason to keep repeating that line outside of the Rust marketing department. Some people will always prefer C to Rust, might as well learn to live with that fact.
- klysm 1y ago
  I never mentioned rust. I’m just saying C and C++ have terrible memory safety.
  - codr7 1y ago
    And what's the alternative then, from your perspective? What did you have in mind when you wrote the comment?
    - klysm 1y ago
      I had no alternative in mind. The topic at hand is security and bloat, and C/C++ might be leaner apps in practice but they are generally going to have memory safety bugs which is a security problem.
      - codr7 1y ago
        It is, but there are very good tools and plenty of experience with dealing with the problem. It's been blown way the fuck out of proportion lately by the Rust mob.
- udev4096 1y ago
  Remember how cloudflare (in 2017) leaked pretty much everyone's secret tokens in search engine cache due to a simple buffer overflow? Yeah, that wouldn't have happened with Rust
  - codr7 1y ago
    Yeah I know, if only we could rewrite the entire world in Rust everything would be rainbows and unicorns. But it's not going to happen, deal with it.
  - lelanthran 1y ago
    Remember that the most expensive exploit the world has ever seen was in a memory safe GC language? My argument is that you are missing the point: the point is that a larger attack surface enables more exploits regardless of language. When using a language that has tremendous friction in expanding the attack surface you tend to have a small attack surface as a result. Theres obviously a crossover point where you'd be safer with a memory safe language and a larger attack surface than with a memory unsafe language and a minuscule attack surface.
    - lmm 1y ago
      > Remember that the most expensive exploit the world has ever seen was in a memory safe GC language? No I don't, which exploit are you talking about? The most expensive exploit I can think of was caused by heartbleed which was in a memory unsafe language. The "most expensive software bug" (not an exploit) caused by turning off the safe overflow handler in the language being used can hardly be considered an indictment of language level safety either. So what exploit are you talking about?
      - throw1111221 1y ago
        Not the person you replied to, but they're probably talking about Log4j. It's a Java logging library that had a helpful feature where logging a special format string would pull code from a remote URL and execute it. So anywhere you can get a Java server to log something you can run arbitrary code. (Ex: by setting a malicious User-Agent.) Estimates say 93% of enterprise cloud environments where affected. I suppose Stuxnet could also count, where the initial infection depends on the human curiosity of plugging an unknown usb drive into an air gapped system.
privong 1y ago
> Unfortunately that comes with the baggage of terrible memory safety. Isn't this unrelated to the parent post's thoughts about the benefit's of the C/C++ ecosystem (or lack thereof) for dependency management? I.e., a Rust-like language could still exist with a dependency management system similar to what C/C++ have now -- that isn't predicated on how the language handles memory.

deleted 1y ago

[deleted]

deleted 1y ago

[deleted]

grg0 1y ago

This is something that I think about constantly and I have come to the same conclusion. While the idea of being able to trivially share code worldwide is appealing, so far it seems to encourage shittier software more than anything else, and the benefit of sharing trivially seems to be defeated by the downsides that bloat and bad software bring with it. Adding friction to code re-use (by means of having to manually download shit from a website and compile it yourself like it's 1995) seems to be a good thing for now until a better package management system is figured out. The friction forces you to think seriously where you actually need that shit or you can write the subset of the functionality you need yourself. To be clear, I also think C++ projects suffer a lot from re-inventing the wheel, particularly in the gamedev world, but that seems to be less worse than, e.g., initializing some nodejs framework project and starting with 100+ dependencies when you haven't even started to write shit.

pixl97 1y ago
When doing SBOM/SCA we see apps with 1000+ deps. It's insane. It's so often we see large packages pulled in because a single function/behavior is needed and ends up massively increasing the risk profile.
- deleted 1y ago
  [deleted]
- 1over137 1y ago
  Holy cow. What domain is this? Web-based probably?
  - pixl97 1y ago
    Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system.
    - thewebguyd 1y ago
      > Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system. You can, but I think this thread speaks volumes about the problem with the JavaScript/NPM ecosystem as a whole vs. pretty much any other. We need something else for the web. The only reason we have 200+ NPM packages for a blank project is because JavaScript is atrocious and has almost nothing built-in. We got crap like LeftPad, isodd, is-array, etc. because of the language. Most of what NPM will pull in on any new web front end project is likely already part of the standard library in C#/dotnet, or Java, Go, etc. But you could go further back and say it's not javascript's fault, it's the fault of trying to hammer the web into doing things it was never designed to do in the first place. But, we insisted on making it an application delivery platform, and now we're suffering the consequences of that. I'm hopeful for WASM, but ideally I'd love to see a resurgence of native apps.

ChrisSD 1y ago

In my experience every developer, company, team, sub-team, etc has their own "library" of random functions, utilities, classes, etc that just end up being included into new projects sooner or later (and everyone and their dog has their own bespoke string handling libraries). Copy/pasting large chunks of code from elsewhere is also rampant. I'm not so sure C/C++ solves the actual problem. Only sweeps it under a carpet so it's much less visible.

achierius 1y ago
It definitely does solve one problem. Like it or not, you can't be hit by supply chain attacks if you don't have a supply chain.
- dgfitz 1y ago
  I mirror all deps locally and only build from the mirror. It isn’t an issue. C/C++ is my dayjob
  - procaryote 1y ago
    at some point you could mirror a supply chain attack... xz was a pretty long game and only found by accident for example
    - dgfitz 1y ago
      I’m sure I will.
  - josephg 1y ago
    This runs the risk of shipping C/C++ libraries with known vulnerabilities. How do you keep track of that? At least with npm / cargo / etc, updating dependencies is a single command away.
    - dgfitz 1y ago
      Pull, update, build?

socalgal2 1y ago

100, ha! The official rust docs, built in rust, use ~750 dependencies - queue the apoligists

nradov 1y ago

There are no absolute good or bad reasons here, it depends on the problem domain and usage environment. If you're writing code where safety or security matters then of course you need to carefully manage the software supply chain. On the other hand, if you're writing an internal utility for limited use with no exposure then who cares, pull in all the dependencies you need and git 'er done.

udev4096 1y ago

Go has the most lean and simple dependency management. It's far better than npm or pypi dumpster fire

watermelon0 1y ago
It's also worth mentioning the extensive standard library and golang.org/x/, which means that you generally don't even need that many 3rd party packages.
- udev4096 1y ago
  Also the extensive measures to combat supply chain security for packages [0] [0] - https://go.dev/blog/supply-chain https://go.dev/blog/supply-chain
guappa 1y ago
Go is a dumpster fire as well as those. edit: lol at the downvotes. Go developers showing how insecure they are once again.
- staunton 1y ago
  Since you're apparently interested in downvotes (why?), I'm pretty sure it's not due to criticism of Go but rather the fact that your criticism is entirely non-specific and therefore doesn't add anything to the discussion...
  - guappa 1y ago
    Because the comment I replied to was so specific? There's plenty of perfectly good libraries on npm and pypi, and there's awful ones. Likewise for go which pulls from "the internet". Must I really demonstrate that bad code exists in go? You want examples? There's plenty of bad libraries in go, and pinning to a commit is a terrible practice in any language. Encourages unstable APIs and unfixable bugs.
  - lmm 1y ago

atoav 1y ago

Bloat might be correlated with the ease of bloating software and it indeed easier to do precisely that if you don't have to write it yourself. Bloat is uncontrolled complexity and making it harder to manage complexity reduces bloat. But it also makes it harder to write software that has to be complex for legitimate reasons. Not everybody should write their own library handling SSL, SQL or regex for example. But those libraries are rarely the problem, things like leftpad are. Or: you can use package systems for good and for evil. The only real way to fight bloat is to be diciplined an vet your dependencies. It must cost you something to pull them in. If you have to read and understand everything you pull in, pulling in everybody and their dog suddenly becomes less desireable. Also I think this is much more an issue off the quality of dependencies than it is about using dependencies themselves (it would be stupid to write 1000 implementations of HTTP for a language, one that works really well is better).

RetroTechie 1y ago
> But it also makes it harder to write software that has to be complex for legitimate reasons. Might have stolen this quote somewhere, but imho: Simple things should be easy, complex things should be possible. Related: software (binary) size should reflect the complexity of the problem domain. Some time ago, ran down the size of apps on my phone. Smallest one? ~2MB. What does that app do? Calculate some hash on a file. Select a file, it does its thing, shows the hash (and/or copy to clipboard). What the ..!?!#$ 2,000,000+ bytes for that? This is on Android, 'batteries included'. Selecting / opening a file should be a couple (or couple dozen) lines of source code, a function call to the OS, and presto. Same with reading file contents, and display output / clipboard copy. Which leaves... computing the hash. I'm not an expert, but what hash functions are so complex that you'd need a MB+ of code to calculate? (answer: none). Note that this app was the least worse offender. Conclusion: Android app model is broken. Or SDKs used to build Android apps are crap. Or other reasons / some combination thereoff. Regardless, ~2MB to compute a file hash is ridiculous. Full-blown Graphical User Interfaces (GUI) have been done in less. I'd be interested to know what that 2MB consists of, though. And where the hash function is at. And what (minute) % of overall binary size. And what all the rest of that binary does.

account-5 1y ago

I'm not a professional Dev but thought this is was tree-shaking is about? Certainly this happens in flutter, whatever you feel about flutter/dart. Or is this a sticking plaster? Genuinely don't know as I only develop personal projects.

victorNicollet 1y ago
Tree-shaking is able to remove code that will never be called. And it's not necessarily good at it: we can detect some situations where a function is never called, and remove that function, but it's mostly the obvious situations such as "this function is never referenced". It cannot detect a case such as: if the string argument to this function contains a substring shaped like XYZ, then replace that substring with a value from the environment variables (the Log4j vulnerability), or from the file system (the XML Entity Extension vulnerability). From the point of view of tree-shaking, this is legitimate code that could be called. This is the kind of vulnerable bloat that comes with importing large libraries (large in the sense of "has many complex features", rather than of megabytes).
- account-5 1y ago
  Thanks for the explanations, much appreciated. I suppose the options are then: 1. Write everything yourself, time consuming and hard, less likely to lead to these types of vulnerabilities. 2. Import others code, easy and takes no time, can lead to vulnerabilities. 3. Use others code, but only what you actually need. Maybe less time consuming than 1 but more than 2, adds a different sort of complexity, done correctly less likely to lead to these vulnerabilities. Not sure if there's any other options here?
  - victorNicollet 1y ago
    I would say 4. grab individual code files (as opposed to entire libraries) and manually edit them, removing unnecessary features and adding new ones where needed.

ozim 1y ago

Writing everything from scratch by hand is an insane take. It is not just reinventing the wheel but there are whole frameworks one should use because writing that thing on your own will take you a lifetime. Yes you should not just pull as dependency thing that kid in his parents basement wrote for fun or to get OSS maintainer on his CV. But there are tons of legitimate libraries and frameworks from people who are better than you at that specific domain.

actionfromafar 1y ago
I feel you are arguing a bit of a strawman. The take is much more nuanced than write everything from scratch.
- ozim 1y ago
  ... simplest solution is often not to use a library at all and write the thing yourself, or even better, realize that you don't need the feature you would have used that library for ... the resulting code is of a manageable size.. I don't see the nuance there, that is my take of the comment, those are pretty much strongest statements and points about using libraries are minimal. That is why I added mine strongly pointing that real world systems are not going to be "managable size" unless they are really small or a single person is working on the.
  - actionfromafar 1y ago
    For me "realize that you don't need the feature" is strong and also hits home. I sometimes prototype in C because it makes me think really hard about "what does this thing really have to do? What can I omit for now?" While in for instance C# I tend to think "this would be simple to implement with whatever-fancy-thing-is-just-a-package-away". Neither way is impossible to judge as good or bad on its own. A real world system is almost always part of a larger system or system of systems. Making one thing simple can make another complex. The world is messy.
barrkel 1y ago
That's not how it works. Here's a scenario. You pull in some library - maybe it resizes images or something. It in turn pulls in image decoders and encoders that you may or may not need. They in turn pull in metadata readers, and those pull in XML libraries to parse metadata, and before you know it a fairly simple resize is costing you 10s of MB. Worse, you pull in different libraries and they all pull in different versions of their own dependencies, with lots of duplication of similar but slightly different code. Node_modules usually ends up like this. The point is not writing the resize code yourself. It's the cultural effect of friction. If pulling in the resize library means you need to chase down the dependencies yourself, first, you're more aware of the cost, and second, the library author will probably give you knobs to eliminate dependencies. Perhaps you only pull in a JPEG decoder because that's all you need, and you exclude the metadata functionality. It's an example, but can you see how adding friction to pulling in every extra transitive dependency would have the effect of librabry authors giving engineers options to prune the dependency tree? The easier a library is to use, the more popular it will be, and a library that has you chasing dependencies won't be easy to use.

SkiFire13 1y ago

How many vulnerabilities were due to badly reinventing the wheel in C/C++ though? Also, people often complain about "bloat", but don't realize that C/C++ are often the most bloated ones precisely because importing libraries is a pain, so they try to include everything in a single library, even though you only need to use less than 10% of it. Look for example at Qt, it is supposed to be a UI framework but it ends up implementing vectors, strings, json parser and who knows how much more stuff. But it's just 1 dependency so it's fine, right?

reaperducer 1y ago
How many vulnerabilities were due to badly reinventing the wheel in C/C++ though? I don't know. Suppose you tell us.
phkahler 1y ago
>> Look for example at Qt, it is supposed to be a UI framework but it ends up implementing vectors, strings, json parser and who knows how much more stuff. But it's just 1 dependency so it's fine, right? Qt is an application development framework, not a GUI toolkit. This is one reason I prefer GTK (there are things I dislike about it too).
- r0ze-at-hn 1y ago
  I remember back in the early 2000's that discussion, but now with the the tonnage that systems like npm can pull in I laugh that we ever thought it wouldn't get worse.
GuB-42 1y ago
There is still an advantage to using Qt over dozens of libraries that offer the same functionality. Qt is backed by a single company, so all you have to watch out for is that company. Also, Qt is generally high quality, I have worked with it, read the source code, etc... and I generally liked what I saw. So I can reasonably assume that quality is consistent overall. When you have many libraries from many independent developers, it doesn't work. The JSON parser may be good, but it doesn't tell me anything about the library that deal with internationalization for instance, and if I wanted to keep track of everything, that's several time the work compared to a single vendor. I agree that Qt is bloated though, but multiplatform UI frameworks are hard to keep light. There is a lot going on in a desktop UIs that people only notice when it isn't there. I tend to treat them like I treat the standard libraries, the OS, and for web apps, the browser. Big components, but you reasonably can't do without.

otikik 1y ago

I thought about this several years ago and I think I hit the right balance with these 2 rules of thumb: * The closer something is to your core business, the less you externalize. * You always externalize security (unless security is your exclusive core business) Say you are building a tax calculation web app. You use dependencies for things like the css generation or database access. You do not rely on an external library for tax calculation. You maintain your own code. You might use an external library for handling currencies properly, because it's a tricky math problem. But you may want to use your own fork instead, as it is close to your core business. On the security side, unless that's your speciality, there's guys out there smarter than you and/or who have dedicated more time and resources than you to figure that stuff out. If you are programming a tax calculation web app you shouldn't be implementing your own authentication algorithm, even if having your tax information secure is one of your core needs. The exception to this is that your core business is literally implementing authentication and nothing else.

the__alchemist 1y ago
I would like to dig into point 2 a bit. Do you think this is a matter of degree, or of kind? Does security, in this, imply a network connection, or some other way that exposes your application to vulnerabilities, or is it something else? Are there any other categories that you would treat in a similar way as security, but to a lesser degree, or that almost meet that threshold for a special category, but don't?
cogman10 1y ago
I think this helps, but I also think the default for any dev (particularly library authors) should be to minimize dependencies as much as possible. Dependencies have both a maintenance and a security cost. Bad libraries have deep and sprawling trees. I've seen devs pull in frameworks just to get access to single simple to write functions.
pphysch 1y ago
There have been major F-ups in recent history with Okta, CrowdStrike, and so on. Keycloak had some major long-standing vulnerabilities. I've had PRs accepted in popular open-source IAM libraries a bit too easily. Yeah, we shouldn't roll our own cryptography, but security isn't as clean cut as this comment implies. It also frequently bleeds into your business logic. Don't confuse externalizing security with externalizing liability.

BrouteMinou 1y ago

When you "Reinvent the wheel", you implement only what you need in an optimized way. This gives a couple of advantages: you own your code, no bloat, usually simpler due to not having all the bells and whistles, less abstraction, so faster because there is no free lunch, minimize the attack surface for supply chain attacks... For fun, the next time you are tempted to install a BlaZiNg FaSt MaDe in RuSt software: get the source, install cargo audit and run the cargo audit on that project. See how many vulnerabilities there are. So far, in my experience, all the software I checked come with their list of vulnerabilities from transitive dependencies. I don't know about npm, I only know by reputation and it's enough for me to avoid.

nebula8804 1y ago
That wheel is only as good as your skill in making it. For many people (the majority i'd guess) someone else making that wheel will have a better end result.
- doublerabbit 1y ago
  The skill is produced by carving the wheel. You've got to start somewhere. Whether a mess or not the returned product is a product of your own. By relying on dependencies you're forever reaching for a goal you'll never achieve.

jajko 1y ago

Yeah everybody should reimplement their own security for example, that's a really smart fool-proof approach especially down the line, no real cases for any contrarian opinions. I do get what you mean, but it works only on some very specific types of projects, when you & potentially comparably (very) good & skilled peers are maintaining and evolving it long term. This was never the case in my 20 years of dev career. This sort of shared well tested libraries -> gradually dependency hell is in some form shared across all similar languages since its pretty basic use case of software development as an engineering discipline. I haven't seen a good silver bullet so far, and ie past 14 years of my work wouldn't be possible with approach you describe.

matheusmoreira 1y ago

> There is no auto-download There is. Linux distributions have package managers whose entire purpose is to distribute and manage applications and their dependencies. The key difference between Linux distribution package managers and programming language package managers is the presence of maintainers. Any random person can push packages to the likes of npm or PyPI. To push packages to Debian or Arch Linux, you must be known and trusted. Programming language package managers are made for developers who love the convenience of pushing their projects to the world whenever they want. Linux distribution package managers are made for users who prefer to trust the maintainers not to let malware into the repositories. Some measured amount of elitism can be a force for good.

reaperducer 1y ago

Now, with systems like npm, maven or cargo, all you need to do to get a package is to add a line in a configuration file They can't hack what doesn't exist. Reducing surface area is sometimes the easiest security measure one can take.

hinkley 1y ago

Within reason, we need to be able to promote third party libraries into the standard library. A small standard library pairs well with an easy mechanism to download code, but at some point it's probably a crutch. There are maybe 5 functions in lodash at this point that show up routinely in production code but cannot be sufficed by existing editions to EcmaScript - sortBy, recursive get, and recursive merges being among the most useful. We could just have these and be done.

buggerme 1y ago

[dead]