6 ms·
> textual requires and module system could’ve been much cleaner This irritated me a lot when I was a ruby programmer. Ruby's require is analogous to C's #inclu
by matheusmoreira 3y ago
> textual requires and module system could’ve been much cleaner
This irritated me a lot when I was a ruby programmer. Ruby's require is analogous to C's #include: modules are just files that get evaluated in order to add stuff to a global interpreter state.
It got to the point I went around asking people on IRC what they thought the perfect module system looked like. Studied long forgotten languages to see how they did it. To this day I'm obsessed with this. I feel like it's one of those things you just have to get right since day one because it's gonna be impossible to change it later.
- dexwiz 3y agoJavaScript’s modules only changed because the toolchain rapidly iterated several times over the course of a decade. Most languages don’t see this kind of churn. Even then the migration is at best partial.
- Zanfa 3y agoI've been fighting JS/TS because I went with ESM over CJS for one of my new projects. It's an endless minefield of configuration issues, crappy-to-none support from widely used libraries and half-baked tooling. There's also generally no (up to date at least) documentation, so most knowledge comes from random Github issues with magical incantations to copy-paste into your configs. I must have wasted weeks worth of time on this busywork and there's no end in sight since any new library might accidentally trigger some new edge case. So yeah, I'd agree with saying the migration is partial at best.
- thaumasiotes 3y ago> Ruby's require is analogous to C's #include: modules are just files that get evaluated in order to add stuff to a global interpreter state. What's the theoretical alternative? Adding stuff to a global interpreter state is the goal. The weird part of #include, to me, is not that it ends up modifying the global interpreter state. That's the point of calling it! The weird part, what C libraries tie themselves into endless knots to work around, is that when you #include something in more than one place, it will execute more than once.
- riffraff 3y agoThere's at least a few? E.g. * Lexical scoped imports a-la python * Pass state to modules explicitly, which I think is what newspeak was doing (i.e there's no global state) Either way you'd lose ruby's "let's add methods to Kernel and have them pop up everywhere" so it's a trade off. Worth nothing: there are a few proposals around to have some sort of namespaces in Ruby, but it's unclear they'll ever happen. E.g. https://bugs.ruby-lang.org/issues/19744 https://bugs.ruby-lang.org/issues/19744
- thaumasiotes 3y ago> Lexical scoped imports a-la python How is this an alternative to modifying the global interpreter state? Before I do any imports, I can type this: os.getcwd() and the interpreter will crash, telling me that "os" doesn't mean anything. But if I do this: import os then, later, I can type exactly the same thing: os.getcwd() and the interpreter will call the function and return a string. In order for the interpreter's behavior to be different when presented with exactly the same input, it's necessary that the state of the interpreter has changed. What's the alternative? And since this is the entire purpose of a module system -- causing the global interpreter to recognize names that it otherwise wouldn't have -- it follows immediately that all module systems will operate by making changes to a global interpreter state. The reason Python's module system is better than C's has nothing to do with whether or not either system modifies a global interpreter state, since that doesn't differ between the systems. The reason is that Python allows you to specify what you want those modifications to be. If I have two Python libraries named os, I can import them like so: from lib1 import os from lib2 import os as os2 and now they don't conflict. As far as I know, C has no such facility.
- mirekrusin 3y agoIf you qualify imports transitive dependencies won’t pollute your scope. It also naturally deals better with name conflicts, allows cherry picking exports etc.
- 3y ago
- oezi 3y agoI wouldn't call what Ruby has a module system because it is just linear textual require.
- urxvtcd 3y agoAny results from your research? That seems interesting.
- matheusmoreira 3y agoResearch is a bit too strong a word for it but I'll summarize my conclusions. Most people I talked with seemed used to and very happy with the Python model: modules are distinct namespaces for symbols and their referenced objects. Pretty much everyone I talked to had no difficulty understanding modules, it's always the package management that makes things complex no matter which language it is. Something I spent quite a bit of time thinking about was whether it was worth reifying that model into the language as first class objects. In other words, should import be a special keyword that makes the interpreter magically bind symbols or a function which returns a regular everyday normal object? At the end of the day, Python's modules are just dictionaries, just like Javascript modules. Should I make that fact apparent or hide it behind language keywords? Interestingly, both languages went in the opposite directions: Python went from special import syntax to allowing you to access modules as a dictionary, while Javascript went from a require function that returns an object to an import statement. I ultimately chose a somewhat weird mix of both, powered by lisp's flexibility. I also tried to figure out how compiled languages approached modules. So I dug up literature on Modula, Modula-2 and Oberon and tried to figure out how they represented modules. This ended up having a significant influence in my design in the form of isolated modules with export control. Each module contains its own table of symbols and their references. Importing is just setting a local symbol to the value of the other module's symbol, and only symbols in the exports list can be imported. I also liked the qualified and unqualified names: added the option to prefix the module name to the local symbol. I also thought a lot about how to map modules to the file system. The idea of program folders from Windows has been an inspiration for a long time now. The idea is if the module itself is reachable then all of its submodules are also reachable by the loader. I also think it's important that no file escapes the package directory. Python and Ruby frequently have thing.{rb,py} scripts and thing/ directories side-by-side, I sought to eliminate that for the module's root directory only which results in a main file like thing/thing.{rb,py}. To enable module-oriented development, I've found the most important feature of the modules and packaging system is editable libraries. Like pip's editable installs and npm link. When I develop a project, I often end up with several supporting libraries. Languages should support linking these local versions to the main project so they can be developed simultaneously.
- lloeki 3y agoIt mostly comes from the intentional design of open classes, thus the namespace organisation is fundamentally decoupled from the filesystem: any Ruby file has to be able to contain any class/module, otherwise one can't add to an open class/module†! There are a bunch of issues about this upstream about this, e.g: https://bugs.ruby-lang.org/issues/14982 https://bugs.ruby-lang.org/issues/14982 My exploratory take on it: https://github.com/lloeki/package-ruby https://github.com/lloeki/package-ruby I'm currently drafting a proposal and planning to open another one, more focused on a specific aspect and a different angle, and thus with a couple of comparative PoCs that'll be a bit different from my above exploratory attempt. † ... via `class` and `module` keywords, as one could always include/extend/prepend or use `module_eval`/`class_eval`/`instance_eval`.
- matheusmoreira 3y agoAt first glance that makes a lot of sense given Ruby's open classes and extreme dynamism. On closer examination though I conclude open classes are not mutually exclusive with a proper modules system. Ruby's classes are reified as normal objects. A module system manages names, symbols, references to those objects. To modify a class we just need its object, which name it comes from doesn't really matter. We'd be able to modify classes even if Ruby had a real modules system. All we need to do is get a reference to that class object, probably by importing it from somewhere into the current namespace. Ruby does exactly that already. Every class just happens to get stored as a constant of the Object class. So creating a class A just stores the new Class instance in ::Object::A which is totally weird when I think about it. And when programmers write A, Ruby looks up the constant in current scopes and eventually finds ::Object::A which contains the Class instance. A proper modules system would allow programmers to define where that object ends up or gets pulled from.
- lloeki 3y agoI was mostly outlining where it came from, as in, what is the original design. Doesn't mean it can't do other things. And actually it can, which is part of my initial proposal with the linked repo, and goes much more in depth in my pending, non-public draft.
- wodenokoto 3y agoSo who does it right? Or at least good? And what is your hot take on what Python does?
- matheusmoreira 3y agoI think Python does it mostly right as far as dynamic languages go. Like Ruby, modules are programs which run from start to end, altering interpreter state as they go. Unlike Ruby, Python modules execute in a local context which provides isolation. Ruby libraries all share the same global namespace and can clash with each other. Python and Ruby have the same flaw in that they do not have export control. Everything that gets set also gets exported. Ruby excludes only local variables while Python relies on convention. Javascript's system is essentially the same as Python's but with require being a function that returns normal objects instead of an import keyword which magically sets the symbols. At least it was back then, new versions of the language transitioned to a dedicated import keyword. I found the reification of modules into first class objects to be an elegant design, not sure if they discovered problems with it or if they were severe enough to justify that transition. I think a mixture of both plus export control would be best. Syntax is friendly and good for most uses but a function that returns a reified module object is also important since it allows programmatic access to modules, allows creating plugin architectures and much more.