6 ms·

On autoloading

55 points by qudat 5y ago

throwdbaaway 5y ago
While I also quite dislike the autoloading behavior of rails, in practice, I haven't found it to be abused too much in large rails projects. So let's say we take away autoloading, it is likely that mostly models code will need to deal with circular dependencies. Meanwhile, in golang projects, within-package circular dependencies are happening blatantly everywhere, and somehow I don't see much complain about it.
wiradikusuma 5y ago
Similar point to this is wildcard imports in Java. Previously I prefer import bla.* because it's shorter and neater, but as the code grows, more naming conflict and more difficult to find where's where, I prefer explicit import. I can configure my IDE to collapse the import lines anyway.
- EdwardDiego 5y ago
  Yeah, most code styles in JVM code bases I've worked in (Kotlin and Scala included) prohibit wildcard imports, reviewing PRs is a lot harder when you're playing "Where did the function/class come from".
- jbverschoor 5y ago
  The big problem doesn't exist in Java, as it's being compiled, and all classes used are fully qualified in the bytecode. In source-based languages, if a library adds some class, you'll get into some trouble. In Java, this will also happen at source level, but results in not being able to compile.
rgoulter 5y ago
Steve Yegge had a series of essays where he discussed two flavours of programmers. "liberal", which e.g. likes lower-friction dynamic languages, prefers magic to boilerplate; and "conservative", which e.g. would prefer to have bugs known at compile time, prefers boilerplate to magic. https://gist.github.com/mrnugget/49ad3ee4043c746e42187e2820ddc2f6 https://gist.github.com/mrnugget/49ad3ee4043c746e42187e2820d... Autoloading sounds like a feature which reduces the friction while iterating on the program (when the mental model of the program is fresh), but adds a lot of friction if you need to maintain the program and are unfamiliar with the codebase. It feels like the points cited in the article in favour of autoloading are "liberal" ("nicer DX to just have all the classes available everywhere"), and the counterpoints are all "conservative" ("code should be readable/maintainable").
- cturner 5y ago
  Putting it in liberal/conservative terms moves it to a simple matter of opinion. But most code spends most of its life maintenance. Would anyone argue against the importance of maintenance?
  - rgoulter 5y ago
    I think it inevitably comes down to trade-offs. If you can write highly maintainable code without paying the cost of slower speed, surely that's a winning strategy. But I suspect an increase in maintainability means "more friction to change". (e.g. I'd expect adding documentation and unit tests is to take more time in the short term than just writing the code without documentation and unit tests).
    - jatone 5y ago
      this problem is precisely what tooling is for. golang for example solves the explicit import problem quite nicely with almost no friction outside of ambiguous cases, which you'd need to handle explicitly anyways.
      - rgoulter 5y ago
        I think golang makes for an interesting example: Your program won't compile if you have unused imports, or even unused variables. Which is low-friction for maintenance. You're not going to have to worry if deleting something is unsafe. But this strong-arm implementation of a lint means that iterating on code meets high friction. It slows down developer iterations as you have to comment/uncomment in order to compile. -- "But if you use some tool it's not so bad" is advice I've been given, which strikes me as the same as "automatic loading isn't so bad if you're using an IDE".
- that_guy_iain 5y ago
  Having used autoloading in PHP it feels very similar to working with Java/Kotlin. With PHP there are community standards that are follow almost everywhere, you know your dependency code is in the vendor dir and then you know that code will follow a naming convention so the code is really easy to find. As the author of the post said Jetbrains have solved this problem and you can jump to the definition of any class really easily. Their issue is they know there is a tool that does the job but refuse to use it. This seems to me like complaining that using a screwdriver to hammer in nails is a bad idea.
mdoms 5y ago
It's crazy to me that this is even controversial. The fact that I have no idea where code is coming from in a rails codebase is by far the most aggravating thing about the whole ecosystem.
- jaredcwhite 5y ago
  It's crazy to me that people think it's crazy to have reasonable opinions on why autoloading/global class hierarchies is awesome. I'll go into open revolt if Ruby ever starts enforcing explicit imports. The Rails Way is what I love so much about being a Rubyist: convention over configuration.
  - qudat 5y ago
    Author here. I agree with you that it's a matter of personal preference. One of the reasons why I decided to write a blog post about this is because my team in general really likes Ruby and everything it brings to the table (autoloading included) -- so venting to them would not be as satisfying.
noduerme 5y ago
So glad someone wrote this. In PHP - I absolutely hate having to work around autoloaders in third party APIs. I love structuring my imports exactly around the directory structure I design; some project-specific code in /services/ can always call something in ../util/ or a static in ../const/ or a pseudo-global function in ../fwk/ just as I, the designer, intended. And if one of those should break as I structure a new project, any old IDE will notice and highlight it. If something has to be put in some weird directory structure, doesn't matter; I don't consider these imports to be anti-DRY, they're the linkages. You don't want to leave them out of the code. DRY is for logic, not linkage. Linkage is a necessary part of sharding your logic into intelligible and discreet modules, packages or classes. The linkages between those must be obvious for anyone who comes along and reads your code later.
- throw_m239339 5y ago
  I'm not sure what you are talking about here. In PHP, everything can be configured with composer, PHP's defacto package manager, including all your directories and their namespaces. > I absolutely hate having to work around autoloaders in third party APIs. If your "third party API" aren't using composer themselves, a defacto standard in PHP, then they are not professionally written. PHP doesn't have modules or any form of modularity or packages, unlike Ruby. PHP only has direct file references from other files (include,require...) and SPL autoloading which is just a way to automate direct file reference (through namespaces or other techniques). I'm not sure what PHP autoloading has to do with any of this article at first place.
  - matheusmoreira 5y ago
    Ruby doesn't have modules or packages either. The require method just reads and executes a Ruby source file, changing global interpreter state.
    - deleted 5y ago
      [deleted]
- tehbeard 5y ago
smitty1e 5y ago
A feature of importing is namespace management, if two dependencies provide SomeClass.
lmilcin 5y ago
I think larger lesson here is that magic may look enticing, especially to the beginner, but it might cost more over long term than a little more involved but non-magical process. It is irritating to work with a system that from time to time requires you many orders of magnitude more time to do a relatively simple task. I prefer a little bit of upkeep (which I can usually amortize/automate) of a simpler system because it lets me be more predictable with my estimates. I can plan better because there is very little surprise lurking that could suddenly cost me an additional day of debugging. And the cost of that upkeep might be very overestimated. For examples, I never felt that adding imports to a file is costing me anything. My IDE usually handles this for me. Or I might just copy something from somewhere else I had gotten something working in the past.
byroot 5y ago
While I agree that OP's complaint is valid, I think they're wrong in blaming the autoloader for it. If you were to remove the autoloader and to explictly use `require` in your app, when you `require a` if `a` also `require b`, you implicitly get access to `b` at the same time. So the problem isn't the autoloader, but Ruby's and PHP single global namespace. If anything the autoloader makes it easier to locate the source, because it forces you to follow a strict naming convention. If you see `Foo::Bar`, you know it's defined in either `foo/bar.rb` or `foo.rb`, and it's easy to locate with your editor fuzzy finder. Another proof is that Django doesn't have this problem even thought it has an autoloader (IIRC). I'd be very much in favor of a Node or Python like import system for Ruby, but there's nothing wrong with the autoloader.
- shime 5y ago
  There are not a lot of things from JavaScript that I would love to see ported to Ruby, but ES6 imports is one of them.
  - eyelidlessness 5y ago
    I’m not a Ruby dev so I can’t comment on that. But I can say that ESM truly is great, although it’s something many don’t appreciate until it finally clicks. I didn’t see the benefit over say CJS for a long time. It was the static analysis potential that finally made it seem so valuable to me. But being URL based (including data URLs!), idempotent, and extensible (eg type assertions) are also huge benefits.
- qudat 5y ago
  > So the problem isn't the autoloader, but Ruby's and PHP single global namespace. Interesting point and I agree! The thing I like about python and javascript's import system is you import the object that you end of using, creating a namespace as a variable, defined at the top of every source file. This lends itself to traceability which was my primary argument in the post. I can see how I conflated the global namespace in ruby with automatically loading classes. Cheers!
timw4mail 5y ago
I am not familiar with Ruby autoloading, but PHP autoloading is (in the vast majority of cases) not that magical. PHP autoloading (with composer, the de facto standard) is basically just a one to one mapping of namespaces to file names, which have matching class names. Now there can be more magical autoloading, if it is manually defined, but that's not very common compared to directly using `include` or `require`.
kureikain 5y ago
This is where Python shine I belive. Its import system is amazing. We can import the whole file or just particular funcion/method in that file. We can import relative with `.` and `..` and even `...` And we can "import as" to avoid name conflict. Using import together Mock library we can even swap out(Mock) particular imported item during unit test. If one thing I would love to see is having that import system in Python ported to Ruby. Granted that the way Ruby currently works also expose a few functionality for doing DSL easiser.
forgotmypw17 5y ago
>Grep codebase class SomeClass; If nothing was found then This one line kinda says it all. If you reach the point where you grep your whole project for SomeClass and nothing comes up, everything else is just going to be a bandaid.
codesections 5y ago
I'm a bit confused by the terminology here. I thought "autoloading" referred to deferring the loading of a constant until it is first used (as described in this post[0]). But the author seems to be talking about loading code without an "import/use/require" statement. Are those two different meanings for "autoloading"? Or do the two go together in Ruby? (As is probably obvious, I'm not a Ruby dev) [0]: http://www.rubyinside.com/ruby-techniques-revealed-autoload-1652.html http://www.rubyinside.com/ruby-techniques-revealed-autoload-...
- jaredcwhite 5y ago
  Yeah, there's the autoloading vs. eager loading question of when a constant first triggers loading a code file, but I think in a broader sense autoloading is used to refer to the idea that your code files will just automatically get loaded in a project as long as you use the right naming conventions.
jaredcwhite 5y ago
As a long-time Rubyist, I freaking love autoloading and have gone to great lengths to implement it in new frameworks I've developed after using Rails for many years. I find it fatiguing and ofttimes infuriating whenever I have to wade into a TypeScript project and there's 30 lines of import statements before I can get to any actual code. Any syntactical "noise" in a programming language bothers me to no end…I prefer to write expressive syntax which is as close to thoughts in the English language as possible. I'm not saying the OP is "wrong"—that is, they're more than welcome to use whatever language they like and write all the import statements they prefer. I'm just glad I can choose to do away with all those distractions.
legobmw99 5y ago
Having never used Ruby or even heard of this feature before, is this not a massive supply chain-style attack surface? If I define methods/classes with the same names as common other packages, is there any way the user can tell which one they’re using if I get them to install my package?

55 points by qudat 5y ago

While I also quite dislike the autoloading behavior of rails, in practice, I haven't found it to be abused too much in large rails projects. So let's say we take away autoloading, it is likely that mostly models code will need to deal with circular dependencies. Meanwhile, in golang projects, within-package circular dependencies are happening blatantly everywhere, and somehow I don't see much complain about it.

wiradikusuma 5y ago

Similar point to this is wildcard imports in Java. Previously I prefer import bla.* because it's shorter and neater, but as the code grows, more naming conflict and more difficult to find where's where, I prefer explicit import. I can configure my IDE to collapse the import lines anyway.

EdwardDiego 5y ago
Yeah, most code styles in JVM code bases I've worked in (Kotlin and Scala included) prohibit wildcard imports, reviewing PRs is a lot harder when you're playing "Where did the function/class come from".
jbverschoor 5y ago
The big problem doesn't exist in Java, as it's being compiled, and all classes used are fully qualified in the bytecode. In source-based languages, if a library adds some class, you'll get into some trouble. In Java, this will also happen at source level, but results in not being able to compile.

rgoulter 5y ago

Steve Yegge had a series of essays where he discussed two flavours of programmers. "liberal", which e.g. likes lower-friction dynamic languages, prefers magic to boilerplate; and "conservative", which e.g. would prefer to have bugs known at compile time, prefers boilerplate to magic. https://gist.github.com/mrnugget/49ad3ee4043c746e42187e2820ddc2f6 https://gist.github.com/mrnugget/49ad3ee4043c746e42187e2820d... Autoloading sounds like a feature which reduces the friction while iterating on the program (when the mental model of the program is fresh), but adds a lot of friction if you need to maintain the program and are unfamiliar with the codebase. It feels like the points cited in the article in favour of autoloading are "liberal" ("nicer DX to just have all the classes available everywhere"), and the counterpoints are all "conservative" ("code should be readable/maintainable").

cturner 5y ago
Putting it in liberal/conservative terms moves it to a simple matter of opinion. But most code spends most of its life maintenance. Would anyone argue against the importance of maintenance?
- rgoulter 5y ago
  I think it inevitably comes down to trade-offs. If you can write highly maintainable code without paying the cost of slower speed, surely that's a winning strategy. But I suspect an increase in maintainability means "more friction to change". (e.g. I'd expect adding documentation and unit tests is to take more time in the short term than just writing the code without documentation and unit tests).
  - jatone 5y ago
    this problem is precisely what tooling is for. golang for example solves the explicit import problem quite nicely with almost no friction outside of ambiguous cases, which you'd need to handle explicitly anyways.
    - rgoulter 5y ago
      I think golang makes for an interesting example: Your program won't compile if you have unused imports, or even unused variables. Which is low-friction for maintenance. You're not going to have to worry if deleting something is unsafe. But this strong-arm implementation of a lint means that iterating on code meets high friction. It slows down developer iterations as you have to comment/uncomment in order to compile. -- "But if you use some tool it's not so bad" is advice I've been given, which strikes me as the same as "automatic loading isn't so bad if you're using an IDE".
that_guy_iain 5y ago
Having used autoloading in PHP it feels very similar to working with Java/Kotlin. With PHP there are community standards that are follow almost everywhere, you know your dependency code is in the vendor dir and then you know that code will follow a naming convention so the code is really easy to find. As the author of the post said Jetbrains have solved this problem and you can jump to the definition of any class really easily. Their issue is they know there is a tool that does the job but refuse to use it. This seems to me like complaining that using a screwdriver to hammer in nails is a bad idea.

mdoms 5y ago

It's crazy to me that this is even controversial. The fact that I have no idea where code is coming from in a rails codebase is by far the most aggravating thing about the whole ecosystem.

jaredcwhite 5y ago
It's crazy to me that people think it's crazy to have reasonable opinions on why autoloading/global class hierarchies is awesome. I'll go into open revolt if Ruby ever starts enforcing explicit imports. The Rails Way is what I love so much about being a Rubyist: convention over configuration.
- qudat 5y ago
  Author here. I agree with you that it's a matter of personal preference. One of the reasons why I decided to write a blog post about this is because my team in general really likes Ruby and everything it brings to the table (autoloading included) -- so venting to them would not be as satisfying.

noduerme 5y ago

So glad someone wrote this. In PHP - I absolutely hate having to work around autoloaders in third party APIs. I love structuring my imports exactly around the directory structure I design; some project-specific code in /services/ can always call something in ../util/ or a static in ../const/ or a pseudo-global function in ../fwk/ just as I, the designer, intended. And if one of those should break as I structure a new project, any old IDE will notice and highlight it. If something has to be put in some weird directory structure, doesn't matter; I don't consider these imports to be anti-DRY, they're the linkages. You don't want to leave them out of the code. DRY is for logic, not linkage. Linkage is a necessary part of sharding your logic into intelligible and discreet modules, packages or classes. The linkages between those must be obvious for anyone who comes along and reads your code later.

throw_m239339 5y ago
I'm not sure what you are talking about here. In PHP, everything can be configured with composer, PHP's defacto package manager, including all your directories and their namespaces. > I absolutely hate having to work around autoloaders in third party APIs. If your "third party API" aren't using composer themselves, a defacto standard in PHP, then they are not professionally written. PHP doesn't have modules or any form of modularity or packages, unlike Ruby. PHP only has direct file references from other files (include,require...) and SPL autoloading which is just a way to automate direct file reference (through namespaces or other techniques). I'm not sure what PHP autoloading has to do with any of this article at first place.
- matheusmoreira 5y ago
  Ruby doesn't have modules or packages either. The require method just reads and executes a Ruby source file, changing global interpreter state.
  - deleted 5y ago
    [deleted]
tehbeard 5y ago

smitty1e 5y ago

A feature of importing is namespace management, if two dependencies provide SomeClass.

lmilcin 5y ago

I think larger lesson here is that magic may look enticing, especially to the beginner, but it might cost more over long term than a little more involved but non-magical process. It is irritating to work with a system that from time to time requires you many orders of magnitude more time to do a relatively simple task. I prefer a little bit of upkeep (which I can usually amortize/automate) of a simpler system because it lets me be more predictable with my estimates. I can plan better because there is very little surprise lurking that could suddenly cost me an additional day of debugging. And the cost of that upkeep might be very overestimated. For examples, I never felt that adding imports to a file is costing me anything. My IDE usually handles this for me. Or I might just copy something from somewhere else I had gotten something working in the past.

byroot 5y ago

While I agree that OP's complaint is valid, I think they're wrong in blaming the autoloader for it. If you were to remove the autoloader and to explictly use `require` in your app, when you `require a` if `a` also `require b`, you implicitly get access to `b` at the same time. So the problem isn't the autoloader, but Ruby's and PHP single global namespace. If anything the autoloader makes it easier to locate the source, because it forces you to follow a strict naming convention. If you see `Foo::Bar`, you know it's defined in either `foo/bar.rb` or `foo.rb`, and it's easy to locate with your editor fuzzy finder. Another proof is that Django doesn't have this problem even thought it has an autoloader (IIRC). I'd be very much in favor of a Node or Python like import system for Ruby, but there's nothing wrong with the autoloader.

shime 5y ago
There are not a lot of things from JavaScript that I would love to see ported to Ruby, but ES6 imports is one of them.
- eyelidlessness 5y ago
  I’m not a Ruby dev so I can’t comment on that. But I can say that ESM truly is great, although it’s something many don’t appreciate until it finally clicks. I didn’t see the benefit over say CJS for a long time. It was the static analysis potential that finally made it seem so valuable to me. But being URL based (including data URLs!), idempotent, and extensible (eg type assertions) are also huge benefits.
qudat 5y ago
> So the problem isn't the autoloader, but Ruby's and PHP single global namespace. Interesting point and I agree! The thing I like about python and javascript's import system is you import the object that you end of using, creating a namespace as a variable, defined at the top of every source file. This lends itself to traceability which was my primary argument in the post. I can see how I conflated the global namespace in ruby with automatically loading classes. Cheers!

timw4mail 5y ago

I am not familiar with Ruby autoloading, but PHP autoloading is (in the vast majority of cases) not that magical. PHP autoloading (with composer, the de facto standard) is basically just a one to one mapping of namespaces to file names, which have matching class names. Now there can be more magical autoloading, if it is manually defined, but that's not very common compared to directly using `include` or `require`.

kureikain 5y ago

This is where Python shine I belive. Its import system is amazing. We can import the whole file or just particular funcion/method in that file. We can import relative with `.` and `..` and even `...` And we can "import as" to avoid name conflict. Using import together Mock library we can even swap out(Mock) particular imported item during unit test. If one thing I would love to see is having that import system in Python ported to Ruby. Granted that the way Ruby currently works also expose a few functionality for doing DSL easiser.

forgotmypw17 5y ago

>Grep codebase class SomeClass; If nothing was found then This one line kinda says it all. If you reach the point where you grep your whole project for SomeClass and nothing comes up, everything else is just going to be a bandaid.

codesections 5y ago

I'm a bit confused by the terminology here. I thought "autoloading" referred to deferring the loading of a constant until it is first used (as described in this post[0]). But the author seems to be talking about loading code without an "import/use/require" statement. Are those two different meanings for "autoloading"? Or do the two go together in Ruby? (As is probably obvious, I'm not a Ruby dev) [0]: http://www.rubyinside.com/ruby-techniques-revealed-autoload-1652.html http://www.rubyinside.com/ruby-techniques-revealed-autoload-...

jaredcwhite 5y ago
Yeah, there's the autoloading vs. eager loading question of when a constant first triggers loading a code file, but I think in a broader sense autoloading is used to refer to the idea that your code files will just automatically get loaded in a project as long as you use the right naming conventions.

jaredcwhite 5y ago

As a long-time Rubyist, I freaking love autoloading and have gone to great lengths to implement it in new frameworks I've developed after using Rails for many years. I find it fatiguing and ofttimes infuriating whenever I have to wade into a TypeScript project and there's 30 lines of import statements before I can get to any actual code. Any syntactical "noise" in a programming language bothers me to no end…I prefer to write expressive syntax which is as close to thoughts in the English language as possible. I'm not saying the OP is "wrong"—that is, they're more than welcome to use whatever language they like and write all the import statements they prefer. I'm just glad I can choose to do away with all those distractions.

legobmw99 5y ago

Having never used Ruby or even heard of this feature before, is this not a massive supply chain-style attack surface? If I define methods/classes with the same names as common other packages, is there any way the user can tell which one they’re using if I get them to install my package?