4 ms·

This is something that I think about constantly and I have come to the same conclusion. While the idea of being able to trivially share code worldwide is appeal

by grg0 1y ago

This is something that I think about constantly and I have come to the same conclusion. While the idea of being able to trivially share code worldwide is appealing, so far it seems to encourage shittier software more than anything else, and the benefit of sharing trivially seems to be defeated by the downsides that bloat and bad software bring with it. Adding friction to code re-use (by means of having to manually download shit from a website and compile it yourself like it's 1995) seems to be a good thing for now until a better package management system is figured out. The friction forces you to think seriously where you actually need that shit or you can write the subset of the functionality you need yourself. To be clear, I also think C++ projects suffer a lot from re-inventing the wheel, particularly in the gamedev world, but that seems to be less worse than, e.g., initializing some nodejs framework project and starting with 100+ dependencies when you haven't even started to write shit.

pixl97 1y ago
When doing SBOM/SCA we see apps with 1000+ deps. It's insane. It's so often we see large packages pulled in because a single function/behavior is needed and ends up massively increasing the risk profile.
- deleted 1y ago
  [deleted]
- 1over137 1y ago
  Holy cow. What domain is this? Web-based probably?
  - pixl97 1y ago
    Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system.
    - thewebguyd 1y ago
      > Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system. You can, but I think this thread speaks volumes about the problem with the JavaScript/NPM ecosystem as a whole vs. pretty much any other. We need something else for the web. The only reason we have 200+ NPM packages for a blank project is because JavaScript is atrocious and has almost nothing built-in. We got crap like LeftPad, isodd, is-array, etc. because of the language. Most of what NPM will pull in on any new web front end project is likely already part of the standard library in C#/dotnet, or Java, Go, etc. But you could go further back and say it's not javascript's fault, it's the fault of trying to hammer the web into doing things it was never designed to do in the first place. But, we insisted on making it an application delivery platform, and now we're suffering the consequences of that. I'm hopeful for WASM, but ideally I'd love to see a resurgence of native apps.
      - 1over137 1y ago
        The problem with native apps is they are (mostly) locked behind "stores" run by Big Tech.
        hombre_fatal 1y ago
        And they're opaque. You have to mitm proxy to even see if they're making requests to who knows where. They run with too many privileges. You can't block ads. You can't link to them. Meanwhile, the web runs in a web browser. You have a network bar. You can inspect element. You can inject Javascript. You can run your own code inside people's apps. The former scenario isn't better than the latter scenario just because some people build their website with too many NPM dependencies.
        1over137 1y ago
        I take your point, but a lot of that needn't be the case. Native apps can have lesser privileges with sandboxing, you can inject code if your OS doesn't forbid it, etc. A lot of this is just in how many of are native apps are, not how they must be.
  - whstl 1y ago
    Could be a Hello World React app using the legacy creator-tool :/ Check this out: https://news.ycombinator.com/item?id=39019001 https://news.ycombinator.com/item?id=39019001 Of course, this is the whole environment except for Node.js itself. And Vite has improved it. But there are definitely some tools that are worse than others.
HPsquared 1y ago
The phrase "cheap and nasty" comes to mind. Over time, some markets tend towards the cheap and nasty.
- TeMPOraL 1y ago
  Some? Almost all. That's the default end state if there's actual competition on the market.
staunton 1y ago
> While the idea of being able to trivially share code worldwide is appealing, so far it seems to encourage shittier software more than anything else, and the benefit of sharing trivially seems to be defeated by the downsides that bloat and bad software bring with it. A lot of projects would simply not exist without it. Linux, comes to mind. I guess one might take the position that "Windows is fine" but would there ever have been even competition for Windows? Another example, everyone would be rolling their own crypto without openssl, and that would mean software that's yet a lot more insecure than what we have. Writing software with any cryptography functionality in mind would be the privilege of giant companies only (and still suck a lot more than what we have). There's a lot more things. The internet and software in general would be set back ~20years. Even with all the nostalgia I can muster, that seems like a much worse situation than today.
- rgavuliak 1y ago
  I agree fully, most users care about making their lives easier, not about development purity. If you can't do both, the puritanistic approach loses.
- grg0 1y ago
  All those projects existed long before package managers in programming languages were a thing (although you could consider the distro's package manager to fulfill that purpose, I guess), so I don't think your point really takes away from mine. And for sure, there are critical dependencies like openssl that better be a shared endeavour. But whether you pull those dependencies in manually or through a package manager is somewhat tangential.
rglullis 1y ago
Cathedrals vs Bazaars. Cathedrals are conservative. Reactionary, even. You can measure the rate of change by generations. Bazaars are accessible and universal. The whole system is chaotic. Changes happen every day. No single agent is in control. We need both to make meaningful progress, and it's the job of engineers to take any given problem and see where to look for the solution.
crabbone 1y ago
This is all heuristic (read "guessing") and not a real solution to the problem. The ground truth is that software bloat isn't bad enough of a problem for software developers to try and fight it. We already know how to prevent this, if really want to. And if the problem was really hurting so much, we'd have automated ways of slimming down the executables / libraries. In my role in creating CI for Python libraries, I did more hands-on dependency management. My approach was to first install libraries with pip, see what was installed, research why particular dependencies have been pulled in, then, if necessary, modify the packages in such a way that unnecessary dependencies would've been removed, and "vendor" the third party code (i.e. store it in my repository, at the version I need). This, obviously, works better for programs, where you typically end up distributing the program with its dependencies anyways. Less so for libraries, but in the context of CI this saved some long minutes of reinstalling dependencies afresh for every CI run. In the end, it was a much better experience than what you usually get with CI targeting Pyhon. But, in the end, nobody really cared. If CI took less than a minute to complete instead of twenty minutes, very little was actually gained. The project didn't have enough CI traffic for this to have any actual effect. So, it was a nice proof of concept, but ended up being not all that useful.
- ryandrake 1y ago
  The reason bloat doesn't get fixed is that it's a problem that doesn't really harm software developers. It is a negative externality whose pain is spread uniformly across users. Every little dependency developers add to make their work more convenient might increase the download size over the user's network by 100MB, or use another 0.5% of the user's CPU, or another 50MB of the user's RAM. The user gets hit, ever so slightly, but the developer sees only upside.

by grg0 1y ago

pixl97 1y ago

When doing SBOM/SCA we see apps with 1000+ deps. It's insane. It's so often we see large packages pulled in because a single function/behavior is needed and ends up massively increasing the risk profile.

deleted 1y ago
[deleted]
1over137 1y ago
Holy cow. What domain is this? Web-based probably?
- pixl97 1y ago
  Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system.
  - thewebguyd 1y ago
    > Npm/node_modules is typically one of the worst offenders, but programmers can do this with any import/library based system. You can, but I think this thread speaks volumes about the problem with the JavaScript/NPM ecosystem as a whole vs. pretty much any other. We need something else for the web. The only reason we have 200+ NPM packages for a blank project is because JavaScript is atrocious and has almost nothing built-in. We got crap like LeftPad, isodd, is-array, etc. because of the language. Most of what NPM will pull in on any new web front end project is likely already part of the standard library in C#/dotnet, or Java, Go, etc. But you could go further back and say it's not javascript's fault, it's the fault of trying to hammer the web into doing things it was never designed to do in the first place. But, we insisted on making it an application delivery platform, and now we're suffering the consequences of that. I'm hopeful for WASM, but ideally I'd love to see a resurgence of native apps.
    - 1over137 1y ago
      The problem with native apps is they are (mostly) locked behind "stores" run by Big Tech.
      - hombre_fatal 1y ago
        And they're opaque. You have to mitm proxy to even see if they're making requests to who knows where. They run with too many privileges. You can't block ads. You can't link to them. Meanwhile, the web runs in a web browser. You have a network bar. You can inspect element. You can inject Javascript. You can run your own code inside people's apps. The former scenario isn't better than the latter scenario just because some people build their website with too many NPM dependencies.
        1over137 1y ago
        I take your point, but a lot of that needn't be the case. Native apps can have lesser privileges with sandboxing, you can inject code if your OS doesn't forbid it, etc. A lot of this is just in how many of are native apps are, not how they must be.
- whstl 1y ago
  Could be a Hello World React app using the legacy creator-tool :/ Check this out: https://news.ycombinator.com/item?id=39019001 https://news.ycombinator.com/item?id=39019001 Of course, this is the whole environment except for Node.js itself. And Vite has improved it. But there are definitely some tools that are worse than others.

HPsquared 1y ago

The phrase "cheap and nasty" comes to mind. Over time, some markets tend towards the cheap and nasty.

TeMPOraL 1y ago
Some? Almost all. That's the default end state if there's actual competition on the market.

staunton 1y ago

> While the idea of being able to trivially share code worldwide is appealing, so far it seems to encourage shittier software more than anything else, and the benefit of sharing trivially seems to be defeated by the downsides that bloat and bad software bring with it. A lot of projects would simply not exist without it. Linux, comes to mind. I guess one might take the position that "Windows is fine" but would there ever have been even competition for Windows? Another example, everyone would be rolling their own crypto without openssl, and that would mean software that's yet a lot more insecure than what we have. Writing software with any cryptography functionality in mind would be the privilege of giant companies only (and still suck a lot more than what we have). There's a lot more things. The internet and software in general would be set back ~20years. Even with all the nostalgia I can muster, that seems like a much worse situation than today.

rgavuliak 1y ago
I agree fully, most users care about making their lives easier, not about development purity. If you can't do both, the puritanistic approach loses.
grg0 1y ago
All those projects existed long before package managers in programming languages were a thing (although you could consider the distro's package manager to fulfill that purpose, I guess), so I don't think your point really takes away from mine. And for sure, there are critical dependencies like openssl that better be a shared endeavour. But whether you pull those dependencies in manually or through a package manager is somewhat tangential.

rglullis 1y ago

Cathedrals vs Bazaars. Cathedrals are conservative. Reactionary, even. You can measure the rate of change by generations. Bazaars are accessible and universal. The whole system is chaotic. Changes happen every day. No single agent is in control. We need both to make meaningful progress, and it's the job of engineers to take any given problem and see where to look for the solution.

crabbone 1y ago

This is all heuristic (read "guessing") and not a real solution to the problem. The ground truth is that software bloat isn't bad enough of a problem for software developers to try and fight it. We already know how to prevent this, if really want to. And if the problem was really hurting so much, we'd have automated ways of slimming down the executables / libraries. In my role in creating CI for Python libraries, I did more hands-on dependency management. My approach was to first install libraries with pip, see what was installed, research why particular dependencies have been pulled in, then, if necessary, modify the packages in such a way that unnecessary dependencies would've been removed, and "vendor" the third party code (i.e. store it in my repository, at the version I need). This, obviously, works better for programs, where you typically end up distributing the program with its dependencies anyways. Less so for libraries, but in the context of CI this saved some long minutes of reinstalling dependencies afresh for every CI run. In the end, it was a much better experience than what you usually get with CI targeting Pyhon. But, in the end, nobody really cared. If CI took less than a minute to complete instead of twenty minutes, very little was actually gained. The project didn't have enough CI traffic for this to have any actual effect. So, it was a nice proof of concept, but ended up being not all that useful.

ryandrake 1y ago
The reason bloat doesn't get fixed is that it's a problem that doesn't really harm software developers. It is a negative externality whose pain is spread uniformly across users. Every little dependency developers add to make their work more convenient might increase the download size over the user's network by 100MB, or use another 0.5% of the user's CPU, or another 50MB of the user's RAM. The user gets hit, ever so slightly, but the developer sees only upside.