4 ms·

The holy grail of package management is absence of packages and versions, but distribution via signature-addressable code. In the end it’s all just functions t

The holy grail of package management is absence of packages and versions, but distribution via signature-addressable code. In the end it’s all just functions that take arguments and return values. They should be the minimal unit of distribution. Potentially even with precompiled bytecode signed by some trusted compilation provider.

saagarjha 6y ago
You still need to keep track of things that are mutually incompatible.
- keymone 6y ago
  Not really, because you ship every required version of every function. Functions aren’t incompatible with functions they don’t use.
cousin_it 6y ago
It's always fun to solve half of a problem perfectly and ignore the other half. But then you wonder why it doesn't take the world by storm. (Or sometimes it does, if you were pushy enough, and everyone has to live with your half-solution.) The problem is not only about units of distribution, but also about units of interoperability. The unit should be a group of functions that share common types, naming conventions and design decisions, and have no weird edge cases when used together, unlike some mishmash cobbled together from the internet. In other words, a library. It's like Coase's "Nature of the firm": libraries exist to lower interoperability costs internally, same as firms exist to lower transaction costs internally. The right size and version policy for libraries is a non-trivial question. I think the npm culture has settled on the wrong answer to that question, and that's why we end up with these complicated dependency trees.
- keymone 6y ago
  You only think it needs to be this cohesive thing with types and global definitions and conventions because that’s the current way to approach software development. There is more than one way to do it. In your code a library is completely ephemeral concept, it is only made concrete due to the fact that you have a reference to it in your package manager’s file. In your code however, that library is defined by functions you import and use. Do what stops you stripping the rest away? And if stripping is fine, what’s wrong with turning that upside down and just not having a concrete library as a unit of distribution? That set of functions can be built out of single repository and share a component in some unique reference scheme, but any subset of it is no less functional - each function still has a spec of what other functions it needs to do it’s job and those requirements are satisfied by your build system and everybody is happy. As for types, in vast majority of circumstances you don’t need anything beyond what’s described in EDN spec, which is basically json but more sensible. And if you do need more types - you just require functions that produce and manipulate them and you refer to those types using that same uniquq reference scheme to describe your function signatures.
  - cousin_it 6y ago
    Removing unused code is a long solved problem. For example, when you compile and link a C program that uses a library, you can make the executable contain only the functions that you used, not the whole library. Similar tools exist for JS. Exposed data isn't the only way functions interoperate. For example, a database library can give you an opaque connection object which you can pass to other code in the library. Sometimes that object will have parts that are visible to other code in the library, but not to you. Similarly, a UI library can give you UI objects that play nicely with other code in the library, and so on.
    - keymone 6y ago
      > For example, a database library can give you an opaque connection object which you can pass to other code in the library. Sometimes that object will have parts that are visible to other code in the library, but not to you. Similarly, a UI library can give you UI objects that play nicely with other code in the library, and so on. Right, and all those objects are consumed by functions of that “library”, so you just refer to those functions in your import statements and have build system take care of fetching them for you. No reason to download every single artifact the library provides to achieve that. Why bother with stripping if you can not bloat your app in the first place?
      - cousin_it 6y ago
        It's a non-problem. For example, three.js (a library that covers all your 3D rendering needs) is 600kb minified. You download it once and drop it in your project. Update it every month if you want, or keep using the same version, it won't break. Doesn't have any transitive dependencies either. If you don't need all its functionality, tree-shake your project at build time. A small set of such libraries can cover everything your project needs. The whole affair is trivial. Though if you come from a "many small dependencies" culture, things look different. You'll try to cobble the same functionality from twenty libraries, each of which uses twenty others, and not the same versions. Soon you're asking for optimization - how to make download size smaller? How to build a whiz system for versioning and dependency resolution? Maybe some clever hashing and caching will help? But if you'd used a small set of comprehensive libraries from the start, none of that would be needed.
devurand 6y ago
Sounds like you want Unison: https://www.unisonweb.org/docs/tour/ https://www.unisonweb.org/docs/tour/
- keymone 6y ago
  Interesting, yeah that sounds like what I was thinking about. Thanks for the link.
- j1mr10rd4n 6y ago
  I was just racking my memory and searching through my library of interesting links to find exactly this! Paul Chiusano gave a nice introductory talk at strangeloop last year: https://www.youtube.com/watch?v=gCWtkvDQ2ZI https://www.youtube.com/watch?v=gCWtkvDQ2ZI
j1mr10rd4n 6y ago
Rich Hickey's talk about this is enlightening https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk

saagarjha 6y ago

You still need to keep track of things that are mutually incompatible.

keymone 6y ago
Not really, because you ship every required version of every function. Functions aren’t incompatible with functions they don’t use.

cousin_it 6y ago

It's always fun to solve half of a problem perfectly and ignore the other half. But then you wonder why it doesn't take the world by storm. (Or sometimes it does, if you were pushy enough, and everyone has to live with your half-solution.) The problem is not only about units of distribution, but also about units of interoperability. The unit should be a group of functions that share common types, naming conventions and design decisions, and have no weird edge cases when used together, unlike some mishmash cobbled together from the internet. In other words, a library. It's like Coase's "Nature of the firm": libraries exist to lower interoperability costs internally, same as firms exist to lower transaction costs internally. The right size and version policy for libraries is a non-trivial question. I think the npm culture has settled on the wrong answer to that question, and that's why we end up with these complicated dependency trees.

keymone 6y ago
You only think it needs to be this cohesive thing with types and global definitions and conventions because that’s the current way to approach software development. There is more than one way to do it. In your code a library is completely ephemeral concept, it is only made concrete due to the fact that you have a reference to it in your package manager’s file. In your code however, that library is defined by functions you import and use. Do what stops you stripping the rest away? And if stripping is fine, what’s wrong with turning that upside down and just not having a concrete library as a unit of distribution? That set of functions can be built out of single repository and share a component in some unique reference scheme, but any subset of it is no less functional - each function still has a spec of what other functions it needs to do it’s job and those requirements are satisfied by your build system and everybody is happy. As for types, in vast majority of circumstances you don’t need anything beyond what’s described in EDN spec, which is basically json but more sensible. And if you do need more types - you just require functions that produce and manipulate them and you refer to those types using that same uniquq reference scheme to describe your function signatures.
- cousin_it 6y ago
  Removing unused code is a long solved problem. For example, when you compile and link a C program that uses a library, you can make the executable contain only the functions that you used, not the whole library. Similar tools exist for JS. Exposed data isn't the only way functions interoperate. For example, a database library can give you an opaque connection object which you can pass to other code in the library. Sometimes that object will have parts that are visible to other code in the library, but not to you. Similarly, a UI library can give you UI objects that play nicely with other code in the library, and so on.
  - keymone 6y ago
    > For example, a database library can give you an opaque connection object which you can pass to other code in the library. Sometimes that object will have parts that are visible to other code in the library, but not to you. Similarly, a UI library can give you UI objects that play nicely with other code in the library, and so on. Right, and all those objects are consumed by functions of that “library”, so you just refer to those functions in your import statements and have build system take care of fetching them for you. No reason to download every single artifact the library provides to achieve that. Why bother with stripping if you can not bloat your app in the first place?
    - cousin_it 6y ago
      It's a non-problem. For example, three.js (a library that covers all your 3D rendering needs) is 600kb minified. You download it once and drop it in your project. Update it every month if you want, or keep using the same version, it won't break. Doesn't have any transitive dependencies either. If you don't need all its functionality, tree-shake your project at build time. A small set of such libraries can cover everything your project needs. The whole affair is trivial. Though if you come from a "many small dependencies" culture, things look different. You'll try to cobble the same functionality from twenty libraries, each of which uses twenty others, and not the same versions. Soon you're asking for optimization - how to make download size smaller? How to build a whiz system for versioning and dependency resolution? Maybe some clever hashing and caching will help? But if you'd used a small set of comprehensive libraries from the start, none of that would be needed.

devurand 6y ago

Sounds like you want Unison: https://www.unisonweb.org/docs/tour/ https://www.unisonweb.org/docs/tour/

keymone 6y ago
Interesting, yeah that sounds like what I was thinking about. Thanks for the link.
j1mr10rd4n 6y ago
I was just racking my memory and searching through my library of interesting links to find exactly this! Paul Chiusano gave a nice introductory talk at strangeloop last year: https://www.youtube.com/watch?v=gCWtkvDQ2ZI https://www.youtube.com/watch?v=gCWtkvDQ2ZI

j1mr10rd4n 6y ago

Rich Hickey's talk about this is enlightening https://www.youtube.com/watch?v=oyLBGkS5ICk https://www.youtube.com/watch?v=oyLBGkS5ICk