8 ms·

Why do we need modules at all? (2011)

116 points by signa11 3y ago

dools 3y ago
I think this is the only time I've read the word "database" and thought "blockchain might be a superior solution". One could imagine replacing proof of work or whatever the "confirmation" process is with some sort of testing or peer review consensus protocol that would append reviewed code to the chain. A blockchain contains a full history of all modificiations, which sounds a bit like version control if you look at it funny.
- alex_young 3y ago
  Or, you know, git. It’s version control, distributed, and doesn’t need multiple power plants dedicated to its existence.
  - tetsuhamu 3y ago
    I get your point, and I agree with it for most cryptocurrencies. In this case, the proof-of-work would be peer-review instead of computation, and the "blocks" would be as frequent as "minor releases". The need for computation would be pretty light. Not that I'd actually support it in practice lol
    - junofan 3y ago
      As it is, I go on npmjs.com, look at the number of downloads, and hope for the best. Then ‘npm audit’ runs automatically, spitting out anywhere from a few dozen to several hundred warnings which I then ignore, still hoping for the best.
      - pests 3y ago
        Nothing better than starting a brand new project from template and that's the first thing you see. Looking at you CRA.
    - quickthrower2 3y ago
      Sounds like GitFlow
- ben-schaaf 3y ago
  This is what's known as a Merkle tree. It's how git works and was invented in 1979.
  - dools 3y ago
    It's also how blockchains work: "A blockchain is a distributed ledger with growing lists of records (blocks) that are securely linked together via cryptographic hashes.[1][2][3][4] Each block contains a cryptographic hash of the previous block, a timestamp, and transaction data (generally represented as a Merkle tree, where data nodes are represented by leaves)" https://en.wikipedia.org/wiki/Blockchain https://en.wikipedia.org/wiki/Blockchain
    - ben-schaaf 3y ago
      Yes, but without the "distributed ledger" and thus consensus protocol it's no longer a blockchain.
- DaiPlusPlus 3y ago
  > I think this is the only time I've read the word "database" and thought "blockchain might be a superior solution". ...how is this relevant to the linked mailing-list thread? They're talking about compilation units.
  - dools 3y ago
    At the start of the article: - all functions go into a global (searchable) Key-value database ... - there are no "open source projects" - only "the open source Key-Value database of all functions"
- zubairq 3y ago
  This is exactly what I am working on. Code is stored as a hash of the source code and stored in IPFS and commits can be optionally confirmed by writing a confirmation to a blockchain
  - dools 3y ago
    Based on how agressively I was downvoted for my comment I think you have a tough row to hoe getting it adopted ;)
macintux 3y ago
Joe was such a creative thinker, and he was never (that I could tell) embarrassed by the prospect of an idea of his proving to be a poor one, so it was always interesting to hear him talk. Genuinely curious, thinking outside the box (writing a new programming language on top of Prolog?), and he treated people with utmost respect. I was a nobody lucky enough to escort him around Chicago one day when he was attending a conference, and we spent a couple of hours talking about art, Erlang, Riak, and man I wish I could remember what else.
- piokoch 3y ago
  "all functions have unique distinct names" Really? And every developer will create custom, broken, non-standard "namespace" system: admin_get_user vs. readers_get_user, or maybe admin.get_user vs. readers.get_user, or maybe get_user_admin, get_user_readers, etc. Surely, this will stir a lot of creativity, but I am not sure we need that.
pyrolistical 3y ago
Only for us needy humans to be able to use short names. Modules can be thought of as prefixes to function names. You already have this global key value of names to functions. The name just happens to be module+function name
- Kinrany 3y ago
  Everything on top of machine code exists solely for the benefit of human understanding.
  - tripleo1 3y ago
    And program optimization, no? Am interesting theory.
- yakubin 3y ago
  Except now you can handle name collisions by renaming imports. If you have two modules that share a name, you can rename them in the import line (e.g. in Haskell or Go). Without modules though if you have two libraries which used the same prefix for their functions, now the compiler will complain about redefinitions and the linker will complain about symbol collisions.
  - mostlylurks 3y ago
    Modules are obviously the best solution for this purpose, but not the only one. Even in C, which has neither modules nor namespaces, you can avoid prefix collisions with a little bit of macro usage by making the prefix configurable in such a way that the consumer of the library only has to define a preprocessor constant before including the library if they're unlucky enough to have stumbled upon two different libraries with the same prefix, with no action required otherwise. Even for the author of an library this approach doesn't require any unreasonable levels of macro usage. In practice this approach is rare, but I've seen a couple of libraries use it, so it's not completely theoretical.
    - yakubin 3y ago
chriseidhof 3y ago
Other people know a lot more about this, but IIRC Unison has some ideas that go in this direction: https://www.unison-lang.org https://www.unison-lang.org
- whalesalad 3y ago
  This language was on the tip of my tongue, thanks for sharing. I thought it was relevant too. Specifically: https://www.unison-lang.org/learn/the-big-idea/ https://www.unison-lang.org/learn/the-big-idea/ > Each Unison definition is identified by a hash of its syntax tree. > Put another way, Unison code is content-addressed.
Jtsummers 3y ago
Three past submissions with comments: https://news.ycombinator.com/item?id=8572600 https://news.ycombinator.com/item?id=8572600 - Nov 7, 2014 (76 comments) https://news.ycombinator.com/item?id=10409507 https://news.ycombinator.com/item?id=10409507 - Oct 18, 2015 (46 comments) https://news.ycombinator.com/item?id=20808000 https://news.ycombinator.com/item?id=20808000 - Aug 27, 2019 (103 comments) EDIT: I also highly recommend reading the original mailing list thread. A lot of interesting discussion there that I don't recall having read before.
calebh 3y ago
I've thought a lot about storing program structures in distributed hash tables and came to the conclusion that the only viable languages that can be safely stored are purely functional. If you consider OOP languages, there are many stateful dependencies, for example a method may rely on a constructor initializing a private variable in some specific way, so the smallest unit of modularity cannot be a method. Similarly an entire class could perhaps rely on methods being called in a certain order. Even though classes are designed to be self contained, stateful behavior really mucks up the ability to separate things into constituent parts. Purely functional on the other hand has none of these problems. This is the approach taken by the Unison Language people, which I think makes the right design decisions.
- TylerE 3y ago
  Purely functional introduces lots of issues on it's own. (Look at all the monad insanity Haskell has to do to get the equivalent of a print statement).
  - docandrew 3y ago
    I thought the D language had a really nice take on this with its notion of “weak” purity, allowing for straightforward allocations and I/O: https://klickverbot.at/blog/2012/05/purity-in-d/ https://klickverbot.at/blog/2012/05/purity-in-d/ I think the idea is that if a function always gives the same result with the same inputs, it is considered “pure enough.”
    - tome 3y ago
      It's very hard to do that if you want to use higher-order functions ergonomically. In fact that's the entire reason this area of language design is difficult at all. Specifically, if you define a higher order function (e.g. `f :: (a -> b) -> c` in Haskell notation) and you want to specify that it is "pure" if and only if its argument is "pure". How do you do that in D? I bet you can't!
  - the-smug-one 3y ago
    What's insane about monads :-)?
skybrian 3y ago
Wikipedia is probably the most prominent example of a flat namespace that works at scale. But those names are pretty long, particularly when they need to disambiguate. Also, it's an encyclopedia where article editors are forced to collaborate, and it's leveraging large vocabularies that already exist. (One for each language.) For programming languages, even with a flat module namespace, you get a land rush where good names get taken early by packages that might end up unpopular or abandoned. Leveraging DNS seems like the answer. Java did it badly, but Go's approach seems fine, perhaps because it leverages GitHub's namespace too (for most modules).
- PaulDavisThe1st 3y ago
  What's the difference between /Wheel of Time /TV Series/Wheel of Time /Video Game/Wheel of Time and /Wheel of Time /Wheel of Time (TV Series) /Wheel of Time (Video Game)
  - Terr_ 3y ago
    Yeah, the non-flat hierarchy/tagging may be informal, but it still exists. (And for good reason.)
  - dgb23 3y ago
    I like the second one better. In programming, it’s useful to have namespaces for overall organization and conflict resolution. But it’s a trade off: you are nudged into a hierarchical ontology, with all the implied issues. Wikipedia titles are free text (more or less). So they can afford not to introduce hierarchical naming and still have nice, easily addressable names without conflicts. This avoids all sorts of problems. Most things simply can’t be categorized in a strict, hierarchical manner.
  - aragonite 3y ago
    I'd argue that "mental anguish" is probably less of a problem on the latter design, since the additional parenthetical material is used pure for disambiguation (and so is optional), not for categorization. So, on the latter design we can have: /Francis Bacon /Francis Bacon (artist) without having to answer anguish-inducing questions that would be raised by /Francis Bacon /artist/Francis Bacon e.g. questions such as: 1. "Maybe we should put Lord Bacon under a category also, maybe `philosopher/Francis Bacon`" 2. "But he's also a statesman ... what about `statesman/Francis Bacon`? Is he more of a philosopher or a statesman"? 3. What about ordinary people (who happen to be involved in historical events) that have wikipedia entries, like George Floyd? Should he be assigned something like `person/George Floyd`? If so, should the two Francis Bacons be assigned `person/philosopher/Francis Bacon` and `person/artist/Francis Bacon` instead? And so on
osdever 3y ago
Very well put post, thank you @signa11!
dgreensp 3y ago
There’s a spectrum of what different programming languages call “modules” and what they use them for, with languages like ML having “true” modules in the conceptual sense, where a module is a unit of abstraction, the way that classes and interfaces in many languages are units of abstraction, though when you dig into it, I believe modules are more fundamental and more powerful… then on the other hand you have the situation of, “Programs are collections of text files, people don’t want to put all their code in one file, let’s call a file a module.” Point being, some ways that programming languages use the concept of a module are deep and let you do things you couldn’t otherwise do, some are more dispensable. Then there’s Smalltalk where I believe programs aren’t collections of text files, you actually browse your code in a sort of code browser and it’s stored as runtime objects in a virtual machine image, basically! Then there’s the matter of how code is released and distributed… packaging and libraries. (In practice, the words “module” and “package” are used in overlapping ways by different languages.) Are the single-function modules/packages of NPM a good thing? In practice it doesn’t seem that way. It’s sort of analogous to trying to “giggify” all the jobs. Every individual function outsourced.
TheAceOfHearts 3y ago
Has NPM been the closest attempt at something similar to what's described here? Lots of single-function modules like is-even, is-odd, and left-pad. Although given potential problems with function deletion, maybe there's a case to be made for careful curation of these functions in a single place, we could call it the standard library.
- quickthrower2 3y ago
  They have a single function but don’t have to be, so package managers can’t assume they are single function. It they could tree shaking might not be needed?
taberiand 3y ago
I suppose it's only natural, having solved distributed systems with Erlang, Joe would move on to tackling the hardest problem in computer science - Naming Things.
crabmusket 3y ago
> contribution to open source can be as simple as contributing a single function Are there meaningful contributions to open source that are the size of a single function? Isn't this what leads to the terrible failure modes of NPM (hello left-pad)? If you want to contribute a single function, what's wrong with a blog post? Also, has the browser not solved this with ESM imports from URLs? I can write a web page containing import {someFunction} from "https://your-website.com/your-script.js"; This also solves the namespacing issues - at least to the extent that the domain name system solves namespacing issues. (Deno, of course, inherits this ability to import code from any URL.) I'm being provocative. I've heard people make similar arguments over time, but never understood why anyone would want open source contributions to be a single function.
- dfee 3y ago
  What’s wrong with a function? Here’s an interesting discussion on the topic, btw: https://github.com/sindresorhus/ama/issues/10 https://github.com/sindresorhus/ama/issues/10
  - tsimionescu 3y ago
    Code is much harder to maintain than it is to create, overall. Adding lines of code to a project is a net negative in the long run if they are not actively maintained by the original author. So as a project maintainer, you don't really want lots of small contributions from random people who don't stick around, at least not when they are adding new behavior to your project. And having thousands of one-line dependencies is actually much worse than having one thousand-lines dependency, since you now depend on the whims of each of those creators not to stop distributing their work, not to change their license, not to inject malicious code, to address zero-days etc.. Ultimately external dependencies are a form of collaboration, and it's very hard work to efficiently collaborate with thousands of people.
    - dfee 3y ago
      Ok. So you don’t like single line node modules. I don’t think I’ve ever used one. Maybe as a transient dependency, though. But a function could be written in multiple ways, and I’m not sure we actually mean “single line” here. You’re referring to something of arbitrarily small complexity. But, getting back to the original comment, what’s to say that one exported function isn’t supported by 100 more non-exported functions? Or, worse case, a 1000 line function that could be decomposed into smaller blocks. Now, I’m sure I’ve leaned on these sorts of dependencies. Especially ones that have a peer dependency on a broken release, where they monkey-patch the problem. Why would I invest in maintaining short lived code like that myself? My version manager would certainly tell me when things break.
waynesonfire 3y ago
here is elib1_misc.erl, https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl https://github.com/joearms/elib1/blob/master/lib/src/elib1_m...
sriku 3y ago
This was also part of the motivation behind (plug) https://GitHub.com/srikumarks/inai https://GitHub.com/srikumarks/inai. While the current incarnation focuses on REST packaged services as a unit, earlier versions did it at function level. Inai's service code is stored and updated through Redis but equivalent KV stores will do as well.
Joker_vD 3y ago
> (( managing namespaces seems really tricky, a lot of peoople seem > to thing that the problem goes away by adding "." 's to the name Well, it becomes someone else's problem... which is about as far as a technical solution to this problem can go: the namespace problem is actually a social problem. > but how do we discover > the initial name www.a.b? - there are two answers - a) we are given the name > (ie we click on a link) - we do not know the name but we search fo it )) The third way is that you look through the list of everything that "www.a" provides on a hunch there should be something like "b" in it. Usually this list is short enough that you can do this and also recognize that "ph" is a synonym so you can use it. Sure, renaming-at-import would help with globally unique names being unwieldy to use but we already can do this with module imports in pretty much any languages, even in C (preprocessor macros or, when you ran into by identical symbol names at the ABI level, symbol renaming). So I think modules is one of those "worse is better" things: when you imagine how different and wonderful the world could be, it kinda obvious that modules are pretty mediocre. But they work, right here and now, and simple things are possible and complex things require hacks on top but still are mostly possible too.
samjohnation111 3y ago
[dead]
37638894breeze 3y ago
I have to listen to the Joe's talk on this but from OP this is more accurately "Why do we need Erlang modules at all?" or generously "Modules in FP languages". A key motivating pattern (fib/3) is a pattern in FP. More directly in terms of Joe's brainstorming: - I don't see how the versioning matter is simplified by a flat space of functions. Before you had Nm modules to track and now you have Nf functions to track, with Nf >> Nm. Aggregating functions in library/modules to version is actually helping with versioning effort, not hindering it. More generally, the versioning of multi-component systems are complex affairs that can only be addressed by constraints - general engineering systems have standards + catalogs as the means of addressing this general engineering issue. - Broadly I disagree with conflating modules and libraries. They are distinct conceptually. Modules could have state (and meta-data state), conceptually. Modules potentially could also have active elements internally. Modules can have life-cycles. To sum: modules conceputally are not just collections of (related) functions. So the general question is 'can we live with just libraries of functions?' I think PLT excitement here is not 'a k/v bag of richly annotated functions' -- the guaranteed end result of that approach is n variants of elaborate 'structure' encoded into the metadata Joe is talking about -- but rather pushing modules to extreme to make the distinction from libraries crystal clear.
- jimbokun 3y ago
  Could you version based on some metadata tags, instead of modules?
  - coldtea 3y ago
    Still a mess for discoverability, does not play well with regular code completion, and so on. So except if you also make a Smalltalk like IDE...
  - 37638894breeze 3y ago
    You could go the route of having a zoo of many small libraries and just version the library, so mod-x becomes lib_x. That, dependency management, is not a convincing argument for something 'new' called "module". You can do it with libraries as well. The question (then) remains: are modules really just libraries? Was it always just about coexistence of related functions?
coldtea 3y ago
>Now I find this very convenient when I write a new small utility function I stick in in elib1_misc.erl - no mental anguish in choosing a module name is involved. No, just the anguish of navigating 1000 non-namespaced functions for the one you need. We might not need modules, but it's good to have namespaces - and just having metadata that are not part of the name doesn't cover that aspect.
- cryptonector 3y ago
  They would be namespaced by "metadata" (tags). You still have to pick tags and/or create nw ones, so it's not necessarily easier than picking or creating modules for your functions. But it might be useful for consumers -- or not, because it might only increase their cognitive load. Still, it's an interesting idea.
cryptonector 3y ago
| - do away with modules | - all functions have unique distinct names | - all functions have (lots of) meta data | - all functions go into a global (searchable) Key-value database So like MP3 tags for functions, and then you get to import by theme/genre/whatever. The system can automatically generate any dependency information not fully made explicit in the give metadata. It's... tempting. I could see this for jq.
zubairq 3y ago
This is actually a good point. On the project I am working on we do exactly that, we just give all functions a unique name, which is the hash of its source code and it works well

116 points by signa11 3y ago

I think this is the only time I've read the word "database" and thought "blockchain might be a superior solution". One could imagine replacing proof of work or whatever the "confirmation" process is with some sort of testing or peer review consensus protocol that would append reviewed code to the chain. A blockchain contains a full history of all modificiations, which sounds a bit like version control if you look at it funny.

alex_young 3y ago
Or, you know, git. It’s version control, distributed, and doesn’t need multiple power plants dedicated to its existence.
- tetsuhamu 3y ago
  I get your point, and I agree with it for most cryptocurrencies. In this case, the proof-of-work would be peer-review instead of computation, and the "blocks" would be as frequent as "minor releases". The need for computation would be pretty light. Not that I'd actually support it in practice lol
  - junofan 3y ago
    As it is, I go on npmjs.com, look at the number of downloads, and hope for the best. Then ‘npm audit’ runs automatically, spitting out anywhere from a few dozen to several hundred warnings which I then ignore, still hoping for the best.
    - pests 3y ago
      Nothing better than starting a brand new project from template and that's the first thing you see. Looking at you CRA.
  - quickthrower2 3y ago
    Sounds like GitFlow
ben-schaaf 3y ago
This is what's known as a Merkle tree. It's how git works and was invented in 1979.
- dools 3y ago
  It's also how blockchains work: "A blockchain is a distributed ledger with growing lists of records (blocks) that are securely linked together via cryptographic hashes.[1][2][3][4] Each block contains a cryptographic hash of the previous block, a timestamp, and transaction data (generally represented as a Merkle tree, where data nodes are represented by leaves)" https://en.wikipedia.org/wiki/Blockchain https://en.wikipedia.org/wiki/Blockchain
  - ben-schaaf 3y ago
    Yes, but without the "distributed ledger" and thus consensus protocol it's no longer a blockchain.
DaiPlusPlus 3y ago
> I think this is the only time I've read the word "database" and thought "blockchain might be a superior solution". ...how is this relevant to the linked mailing-list thread? They're talking about compilation units.
- dools 3y ago
  At the start of the article: - all functions go into a global (searchable) Key-value database ... - there are no "open source projects" - only "the open source Key-Value database of all functions"
zubairq 3y ago
This is exactly what I am working on. Code is stored as a hash of the source code and stored in IPFS and commits can be optionally confirmed by writing a confirmation to a blockchain
- dools 3y ago
  Based on how agressively I was downvoted for my comment I think you have a tough row to hoe getting it adopted ;)

macintux 3y ago

Joe was such a creative thinker, and he was never (that I could tell) embarrassed by the prospect of an idea of his proving to be a poor one, so it was always interesting to hear him talk. Genuinely curious, thinking outside the box (writing a new programming language on top of Prolog?), and he treated people with utmost respect. I was a nobody lucky enough to escort him around Chicago one day when he was attending a conference, and we spent a couple of hours talking about art, Erlang, Riak, and man I wish I could remember what else.

piokoch 3y ago
"all functions have unique distinct names" Really? And every developer will create custom, broken, non-standard "namespace" system: admin_get_user vs. readers_get_user, or maybe admin.get_user vs. readers.get_user, or maybe get_user_admin, get_user_readers, etc. Surely, this will stir a lot of creativity, but I am not sure we need that.

pyrolistical 3y ago

Only for us needy humans to be able to use short names. Modules can be thought of as prefixes to function names. You already have this global key value of names to functions. The name just happens to be module+function name

Kinrany 3y ago
Everything on top of machine code exists solely for the benefit of human understanding.
- tripleo1 3y ago
  And program optimization, no? Am interesting theory.
yakubin 3y ago
Except now you can handle name collisions by renaming imports. If you have two modules that share a name, you can rename them in the import line (e.g. in Haskell or Go). Without modules though if you have two libraries which used the same prefix for their functions, now the compiler will complain about redefinitions and the linker will complain about symbol collisions.
- mostlylurks 3y ago
  Modules are obviously the best solution for this purpose, but not the only one. Even in C, which has neither modules nor namespaces, you can avoid prefix collisions with a little bit of macro usage by making the prefix configurable in such a way that the consumer of the library only has to define a preprocessor constant before including the library if they're unlucky enough to have stumbled upon two different libraries with the same prefix, with no action required otherwise. Even for the author of an library this approach doesn't require any unreasonable levels of macro usage. In practice this approach is rare, but I've seen a couple of libraries use it, so it's not completely theoretical.
  - yakubin 3y ago

chriseidhof 3y ago

Other people know a lot more about this, but IIRC Unison has some ideas that go in this direction: https://www.unison-lang.org https://www.unison-lang.org

whalesalad 3y ago
This language was on the tip of my tongue, thanks for sharing. I thought it was relevant too. Specifically: https://www.unison-lang.org/learn/the-big-idea/ https://www.unison-lang.org/learn/the-big-idea/ > Each Unison definition is identified by a hash of its syntax tree. > Put another way, Unison code is content-addressed.

Jtsummers 3y ago

Three past submissions with comments: https://news.ycombinator.com/item?id=8572600 https://news.ycombinator.com/item?id=8572600 - Nov 7, 2014 (76 comments) https://news.ycombinator.com/item?id=10409507 https://news.ycombinator.com/item?id=10409507 - Oct 18, 2015 (46 comments) https://news.ycombinator.com/item?id=20808000 https://news.ycombinator.com/item?id=20808000 - Aug 27, 2019 (103 comments) EDIT: I also highly recommend reading the original mailing list thread. A lot of interesting discussion there that I don't recall having read before.

calebh 3y ago

I've thought a lot about storing program structures in distributed hash tables and came to the conclusion that the only viable languages that can be safely stored are purely functional. If you consider OOP languages, there are many stateful dependencies, for example a method may rely on a constructor initializing a private variable in some specific way, so the smallest unit of modularity cannot be a method. Similarly an entire class could perhaps rely on methods being called in a certain order. Even though classes are designed to be self contained, stateful behavior really mucks up the ability to separate things into constituent parts. Purely functional on the other hand has none of these problems. This is the approach taken by the Unison Language people, which I think makes the right design decisions.

TylerE 3y ago
Purely functional introduces lots of issues on it's own. (Look at all the monad insanity Haskell has to do to get the equivalent of a print statement).
- docandrew 3y ago
  I thought the D language had a really nice take on this with its notion of “weak” purity, allowing for straightforward allocations and I/O: https://klickverbot.at/blog/2012/05/purity-in-d/ https://klickverbot.at/blog/2012/05/purity-in-d/ I think the idea is that if a function always gives the same result with the same inputs, it is considered “pure enough.”
  - tome 3y ago
    It's very hard to do that if you want to use higher-order functions ergonomically. In fact that's the entire reason this area of language design is difficult at all. Specifically, if you define a higher order function (e.g. `f :: (a -> b) -> c` in Haskell notation) and you want to specify that it is "pure" if and only if its argument is "pure". How do you do that in D? I bet you can't!
- the-smug-one 3y ago
  What's insane about monads :-)?

skybrian 3y ago

Wikipedia is probably the most prominent example of a flat namespace that works at scale. But those names are pretty long, particularly when they need to disambiguate. Also, it's an encyclopedia where article editors are forced to collaborate, and it's leveraging large vocabularies that already exist. (One for each language.) For programming languages, even with a flat module namespace, you get a land rush where good names get taken early by packages that might end up unpopular or abandoned. Leveraging DNS seems like the answer. Java did it badly, but Go's approach seems fine, perhaps because it leverages GitHub's namespace too (for most modules).

PaulDavisThe1st 3y ago
What's the difference between /Wheel of Time /TV Series/Wheel of Time /Video Game/Wheel of Time and /Wheel of Time /Wheel of Time (TV Series) /Wheel of Time (Video Game)
- Terr_ 3y ago
  Yeah, the non-flat hierarchy/tagging may be informal, but it still exists. (And for good reason.)
- dgb23 3y ago
  I like the second one better. In programming, it’s useful to have namespaces for overall organization and conflict resolution. But it’s a trade off: you are nudged into a hierarchical ontology, with all the implied issues. Wikipedia titles are free text (more or less). So they can afford not to introduce hierarchical naming and still have nice, easily addressable names without conflicts. This avoids all sorts of problems. Most things simply can’t be categorized in a strict, hierarchical manner.
- aragonite 3y ago
  I'd argue that "mental anguish" is probably less of a problem on the latter design, since the additional parenthetical material is used pure for disambiguation (and so is optional), not for categorization. So, on the latter design we can have: /Francis Bacon /Francis Bacon (artist) without having to answer anguish-inducing questions that would be raised by /Francis Bacon /artist/Francis Bacon e.g. questions such as: 1. "Maybe we should put Lord Bacon under a category also, maybe `philosopher/Francis Bacon`" 2. "But he's also a statesman ... what about `statesman/Francis Bacon`? Is he more of a philosopher or a statesman"? 3. What about ordinary people (who happen to be involved in historical events) that have wikipedia entries, like George Floyd? Should he be assigned something like `person/George Floyd`? If so, should the two Francis Bacons be assigned `person/philosopher/Francis Bacon` and `person/artist/Francis Bacon` instead? And so on

osdever 3y ago

Very well put post, thank you @signa11!

dgreensp 3y ago

There’s a spectrum of what different programming languages call “modules” and what they use them for, with languages like ML having “true” modules in the conceptual sense, where a module is a unit of abstraction, the way that classes and interfaces in many languages are units of abstraction, though when you dig into it, I believe modules are more fundamental and more powerful… then on the other hand you have the situation of, “Programs are collections of text files, people don’t want to put all their code in one file, let’s call a file a module.” Point being, some ways that programming languages use the concept of a module are deep and let you do things you couldn’t otherwise do, some are more dispensable. Then there’s Smalltalk where I believe programs aren’t collections of text files, you actually browse your code in a sort of code browser and it’s stored as runtime objects in a virtual machine image, basically! Then there’s the matter of how code is released and distributed… packaging and libraries. (In practice, the words “module” and “package” are used in overlapping ways by different languages.) Are the single-function modules/packages of NPM a good thing? In practice it doesn’t seem that way. It’s sort of analogous to trying to “giggify” all the jobs. Every individual function outsourced.

TheAceOfHearts 3y ago

Has NPM been the closest attempt at something similar to what's described here? Lots of single-function modules like is-even, is-odd, and left-pad. Although given potential problems with function deletion, maybe there's a case to be made for careful curation of these functions in a single place, we could call it the standard library.

quickthrower2 3y ago
They have a single function but don’t have to be, so package managers can’t assume they are single function. It they could tree shaking might not be needed?

taberiand 3y ago

I suppose it's only natural, having solved distributed systems with Erlang, Joe would move on to tackling the hardest problem in computer science - Naming Things.

crabmusket 3y ago

> contribution to open source can be as simple as contributing a single function Are there meaningful contributions to open source that are the size of a single function? Isn't this what leads to the terrible failure modes of NPM (hello left-pad)? If you want to contribute a single function, what's wrong with a blog post? Also, has the browser not solved this with ESM imports from URLs? I can write a web page containing import {someFunction} from "https://your-website.com/your-script.js"; This also solves the namespacing issues - at least to the extent that the domain name system solves namespacing issues. (Deno, of course, inherits this ability to import code from any URL.) I'm being provocative. I've heard people make similar arguments over time, but never understood why anyone would want open source contributions to be a single function.

dfee 3y ago
What’s wrong with a function? Here’s an interesting discussion on the topic, btw: https://github.com/sindresorhus/ama/issues/10 https://github.com/sindresorhus/ama/issues/10
- tsimionescu 3y ago
  Code is much harder to maintain than it is to create, overall. Adding lines of code to a project is a net negative in the long run if they are not actively maintained by the original author. So as a project maintainer, you don't really want lots of small contributions from random people who don't stick around, at least not when they are adding new behavior to your project. And having thousands of one-line dependencies is actually much worse than having one thousand-lines dependency, since you now depend on the whims of each of those creators not to stop distributing their work, not to change their license, not to inject malicious code, to address zero-days etc.. Ultimately external dependencies are a form of collaboration, and it's very hard work to efficiently collaborate with thousands of people.
  - dfee 3y ago
    Ok. So you don’t like single line node modules. I don’t think I’ve ever used one. Maybe as a transient dependency, though. But a function could be written in multiple ways, and I’m not sure we actually mean “single line” here. You’re referring to something of arbitrarily small complexity. But, getting back to the original comment, what’s to say that one exported function isn’t supported by 100 more non-exported functions? Or, worse case, a 1000 line function that could be decomposed into smaller blocks. Now, I’m sure I’ve leaned on these sorts of dependencies. Especially ones that have a peer dependency on a broken release, where they monkey-patch the problem. Why would I invest in maintaining short lived code like that myself? My version manager would certainly tell me when things break.

waynesonfire 3y ago

here is elib1_misc.erl, https://github.com/joearms/elib1/blob/master/lib/src/elib1_misc.erl https://github.com/joearms/elib1/blob/master/lib/src/elib1_m...

sriku 3y ago

This was also part of the motivation behind (plug) https://GitHub.com/srikumarks/inai https://GitHub.com/srikumarks/inai. While the current incarnation focuses on REST packaged services as a unit, earlier versions did it at function level. Inai's service code is stored and updated through Redis but equivalent KV stores will do as well.

Joker_vD 3y ago

> (( managing namespaces seems really tricky, a lot of peoople seem > to thing that the problem goes away by adding "." 's to the name Well, it becomes someone else's problem... which is about as far as a technical solution to this problem can go: the namespace problem is actually a social problem. > but how do we discover > the initial name www.a.b? - there are two answers - a) we are given the name > (ie we click on a link) - we do not know the name but we search fo it )) The third way is that you look through the list of everything that "www.a" provides on a hunch there should be something like "b" in it. Usually this list is short enough that you can do this and also recognize that "ph" is a synonym so you can use it. Sure, renaming-at-import would help with globally unique names being unwieldy to use but we already can do this with module imports in pretty much any languages, even in C (preprocessor macros or, when you ran into by identical symbol names at the ABI level, symbol renaming). So I think modules is one of those "worse is better" things: when you imagine how different and wonderful the world could be, it kinda obvious that modules are pretty mediocre. But they work, right here and now, and simple things are possible and complex things require hacks on top but still are mostly possible too.

samjohnation111 3y ago

[dead]

37638894breeze 3y ago

I have to listen to the Joe's talk on this but from OP this is more accurately "Why do we need Erlang modules at all?" or generously "Modules in FP languages". A key motivating pattern (fib/3) is a pattern in FP. More directly in terms of Joe's brainstorming: - I don't see how the versioning matter is simplified by a flat space of functions. Before you had Nm modules to track and now you have Nf functions to track, with Nf >> Nm. Aggregating functions in library/modules to version is actually helping with versioning effort, not hindering it. More generally, the versioning of multi-component systems are complex affairs that can only be addressed by constraints - general engineering systems have standards + catalogs as the means of addressing this general engineering issue. - Broadly I disagree with conflating modules and libraries. They are distinct conceptually. Modules could have state (and meta-data state), conceptually. Modules potentially could also have active elements internally. Modules can have life-cycles. To sum: modules conceputally are not just collections of (related) functions. So the general question is 'can we live with just libraries of functions?' I think PLT excitement here is not 'a k/v bag of richly annotated functions' -- the guaranteed end result of that approach is n variants of elaborate 'structure' encoded into the metadata Joe is talking about -- but rather pushing modules to extreme to make the distinction from libraries crystal clear.

jimbokun 3y ago
Could you version based on some metadata tags, instead of modules?
- coldtea 3y ago
  Still a mess for discoverability, does not play well with regular code completion, and so on. So except if you also make a Smalltalk like IDE...
- 37638894breeze 3y ago
  You could go the route of having a zoo of many small libraries and just version the library, so mod-x becomes lib_x. That, dependency management, is not a convincing argument for something 'new' called "module". You can do it with libraries as well. The question (then) remains: are modules really just libraries? Was it always just about coexistence of related functions?

coldtea 3y ago

>Now I find this very convenient when I write a new small utility function I stick in in elib1_misc.erl - no mental anguish in choosing a module name is involved. No, just the anguish of navigating 1000 non-namespaced functions for the one you need. We might not need modules, but it's good to have namespaces - and just having metadata that are not part of the name doesn't cover that aspect.

cryptonector 3y ago
They would be namespaced by "metadata" (tags). You still have to pick tags and/or create nw ones, so it's not necessarily easier than picking or creating modules for your functions. But it might be useful for consumers -- or not, because it might only increase their cognitive load. Still, it's an interesting idea.

cryptonector 3y ago

| - do away with modules | - all functions have unique distinct names | - all functions have (lots of) meta data | - all functions go into a global (searchable) Key-value database So like MP3 tags for functions, and then you get to import by theme/genre/whatever. The system can automatically generate any dependency information not fully made explicit in the give metadata. It's... tempting. I could see this for jq.

zubairq 3y ago

This is actually a good point. On the project I am working on we do exactly that, we just give all functions a unique name, which is the hash of its source code and it works well