5 ms·

Granted Microsoft sets a particularly egregious example here, but the way we (don't) track dependencies is f'ed up across the whole software industry. It shoul

Granted Microsoft sets a particularly egregious example here, but the way we (don't) track dependencies is f'ed up across the whole software industry. It should be (nay, needs to be) possible - in an automated way, with a visual UI - to see a piece of software as a cloud of features and click and drag away or cut+paste and get JUST the parts of the code the feature ACTUALLY requires. Yes, like dependency inversion, but I'm talking about all the way down to the assembly instructions level. Before you object that that's just not possible: yes, but only because we've already scrambled the eggs and can't unscramble them. The problem is the way we develop(ed) the whole stack (down to the assembly language and kernel calls level) is bass ackwards. Consider that the foundations of how everything is organized (vis a vis dependency injection or lack thereof in low level libraries) dates back to a time when computers were many orders of magnitude smaller and you were expected to have a good idea what was in the libraries you included. Now, the situation is reversed - you have no idea what's in the libraries you include and no idea what they include, etc. But we still program as if it should be the responsibility of the user to understand the library / program and what it uses transitively.

tenaciousDaniel 8y ago
> a visual UI to see a piece of software as a cloud of features and ... get just the parts of the code the feature actually requires oooohhh I love that idea. I wonder if this could be utilized for npm.
- fit2rule 8y ago
  I'm seeing it being at least a bit feasible with package managers that allow the ability to config/build packages and install them locally according to needs by the developer .. something like Gobolinux with its bundle garden, combined with a configuration-management tool that can build every package with every possible configuration .. Bit of a goose, though. As in, not sure it'll fly in its current (scrambled eggs) state. Perhaps a rewrite is really what's needed .. or maybe HaikuOS can deliver this without much ruffling of feathers?
  - tenaciousDaniel 8y ago
    Yeah I agree. What OP is describing would require an architectural shift in the way npm packages are built, IMO. Sidenote, this is related, and pretty interesting: https://npm.anvaka.com https://npm.anvaka.com
  - phaedrus 8y ago
    Imagine if the provenance of every code line or machine instruction was tracked all the way from what the human wrote to what the machine executes. Combined with data flow analysis, itself combined with runtime analysis via a feedback process like Profile Guided Optimization. The outcome would be somewhat like a package management tool, but one which can subdivide & prune a package even if the original author wrote it monolithically. It could also provide feedback in the IDE showing which lines are "dead code" that don't contribute to the features which you've declared (via tests) are important.
- phaedrus 8y ago
  I've been thinking a lot about Software Archeology, trying to envision what that might be. I think we don't even have the tools yet to do a Site Survey. Somehow I can't fully envision what those tools would look like, but this is my best attempt to describe the vague picture in my head.
- ep103 8y ago
  Visual Studio's Code Architecture Diagrams are a good idea for how this could look and work, I think. Its basically just missing a "Crop all code outside this dependency chain" for any given function
- Ajedi32 8y ago
  Isn't that pretty much what tree shaking does? Webpack has had that for quite some time now (except fully automated, without the UI).
  - tenaciousDaniel 8y ago
    I think tree-shaking is made for finding dead code? Seems like the goals are intertwined though.
DrScientist 8y ago
Personally I don't like tooling that magically does transitive dependency management - it creates the problem of bloat by making it too easy to add stuff - stuff that typically never gets removed. Doing dependency management at the shared library level rather than source level, also results in massive bloat - you might only need one call for a library, but the whole library and all it's dependences, and their dependencies etc etc get pulled in - a ridiculous mess - then you have the whole different versions of libraries problem. Things like npm, maven etc are the problem, not a solution in my view.
- kjeetgill 8y ago
  Dependency management is a tremendous quality of life improvement, but you're not wrong: deep webs of dependency suck. The Java ecosystem is a better about this than npm, but good public libraries need to make the aesthetic choice to keep minimal dependencies. Helper/toolkit libraries like Guava or Lodash should pretty much be end user only. Your gzip library doesn't need em. I get they're convenient, but if you're going to use 3 helpers in 5 places just fork those little bits and add them to your own codebase. DRY is for applications not libraries. I like to see dependencies no more than 5 deep which is usually at the edge of manageability. For Java the critical path looks like: Internal framework -> internal client lib -> RPC lib -> serialization -> bytecode hackery lib

tenaciousDaniel 8y ago

> a visual UI to see a piece of software as a cloud of features and ... get just the parts of the code the feature actually requires oooohhh I love that idea. I wonder if this could be utilized for npm.

fit2rule 8y ago
I'm seeing it being at least a bit feasible with package managers that allow the ability to config/build packages and install them locally according to needs by the developer .. something like Gobolinux with its bundle garden, combined with a configuration-management tool that can build every package with every possible configuration .. Bit of a goose, though. As in, not sure it'll fly in its current (scrambled eggs) state. Perhaps a rewrite is really what's needed .. or maybe HaikuOS can deliver this without much ruffling of feathers?
- tenaciousDaniel 8y ago
  Yeah I agree. What OP is describing would require an architectural shift in the way npm packages are built, IMO. Sidenote, this is related, and pretty interesting: https://npm.anvaka.com https://npm.anvaka.com
- phaedrus 8y ago
  Imagine if the provenance of every code line or machine instruction was tracked all the way from what the human wrote to what the machine executes. Combined with data flow analysis, itself combined with runtime analysis via a feedback process like Profile Guided Optimization. The outcome would be somewhat like a package management tool, but one which can subdivide & prune a package even if the original author wrote it monolithically. It could also provide feedback in the IDE showing which lines are "dead code" that don't contribute to the features which you've declared (via tests) are important.
phaedrus 8y ago
I've been thinking a lot about Software Archeology, trying to envision what that might be. I think we don't even have the tools yet to do a Site Survey. Somehow I can't fully envision what those tools would look like, but this is my best attempt to describe the vague picture in my head.
ep103 8y ago
Visual Studio's Code Architecture Diagrams are a good idea for how this could look and work, I think. Its basically just missing a "Crop all code outside this dependency chain" for any given function
Ajedi32 8y ago
Isn't that pretty much what tree shaking does? Webpack has had that for quite some time now (except fully automated, without the UI).
- tenaciousDaniel 8y ago
  I think tree-shaking is made for finding dead code? Seems like the goals are intertwined though.

DrScientist 8y ago

Personally I don't like tooling that magically does transitive dependency management - it creates the problem of bloat by making it too easy to add stuff - stuff that typically never gets removed. Doing dependency management at the shared library level rather than source level, also results in massive bloat - you might only need one call for a library, but the whole library and all it's dependences, and their dependencies etc etc get pulled in - a ridiculous mess - then you have the whole different versions of libraries problem. Things like npm, maven etc are the problem, not a solution in my view.

kjeetgill 8y ago
Dependency management is a tremendous quality of life improvement, but you're not wrong: deep webs of dependency suck. The Java ecosystem is a better about this than npm, but good public libraries need to make the aesthetic choice to keep minimal dependencies. Helper/toolkit libraries like Guava or Lodash should pretty much be end user only. Your gzip library doesn't need em. I get they're convenient, but if you're going to use 3 helpers in 5 places just fork those little bits and add them to your own codebase. DRY is for applications not libraries. I like to see dependencies no more than 5 deep which is usually at the edge of manageability. For Java the critical path looks like: Internal framework -> internal client lib -> RPC lib -> serialization -> bytecode hackery lib