5 ms·

I wonder how much things would break if all imports were lazy by default.

• simlevesque 1y ago
  Maybe nothing would break ? edit: ok well "xxx in sys.modules" would indeed be a problem
  • IshKebab 1y ago
    Yeah unfortunately in real world Python code people put side effects in their modules all the time.
    • kstrauser 1y ago
      In Python? I almost never see that. And I’ve certainly never signed off on a PR that did that.
      • oofbey 1y ago
        Maybe your code is awesome, but you're almost certainly using a library which does it.
        • lyu07282 1y ago
          This. The most egregious example I've ever seen is the inflect library, takes almost 2 seconds(!) to import: https://github.com/jaraco/inflect/issues/212 https://github.com/jaraco/inflect/issues/212
      • zahlman 1y ago
        Recursively importing other modules is itself a side effect. In fact, all the code you see in the module is "side effects", in a sense. A `class` body, for example, has to actually run at import time, creating the class object and attaching it as an attribute of the module object. Similarly for functions. Even a simple assignment of a constant actually has to run at module import. And all of these things add up. Further, if there isn't already cached bytecode available for the module, by default it will be written to disk as part of the import process. That's inarguably a side effect.
        • flykespice 1y ago
          Side-effect means you're changing state outside the scope the code is running. Sure thing you can declare globals variable and run anything on a module file global scope (outside funcs and class body), but even that 'global' scope is just an illusion, and everything declared there, as yourself said, is scoped to the module's namespace (and you can't leak the 'globals' when importing the module unless you explicity do so 'from foo import *'. Think of python's import as eval but safer because it doesn't leaks the results from the module execution) So for a module to have side-effect (for me) it would either: - Change/Create attributes from other modules - Call some other function that does side-effect (reflection builtins? IO stuff)
          • IshKebab 1y ago
            Often it's things like registering functions in a global registry, or patching things about how Python works (e.g. adding stuff to the module path).
          • kstrauser 1y ago
            You’re right. “Side effect” here means that your program’s behavior changes simply by importing something, eg by monkey patching something else. If’s not: * Importing other modules. * Taking a long time to import. * Writing .pyc files. If any program can “import foo” and still execute exactly the same bytecode afterward as before, you can say that foo doesn’t have side effects.
• eesmith 1y ago
  All my code which uses import probing would fail, such as fallbacks: try: import module except ImportError: import slow_module as module Conditional support testing would also break, like having tests which only run if module2 is available: try: import module2 except ImportError: def if_has_module2(f): return unittest.skip("module2 not available")(f) else: def if_has_module2(f): return f @if_has_module2 class TestModule2Bindings(.... The proto-PEP also gives an example of using with suppress_warnings(): import module3 where some global configuration changes only during import. In general, "import this" and "import antigravity" - anything with import side-effects - would stop working. Oh, and as the proto-PEP points out, changes to sys.path and others can cause problems because of the delay between time of lazy import and time of resolution. Then there's code where you do a long computation then make use of a package which might not be present. import database # remember to install!! import qcd_simulation universe = qcd_simulation.run(seconds = 30*24*60*60) database.save(universe) All of these would be replaced with "import module; module.__name__" or something to force the import, or by an explicit use to __import__.
  • npage97 1y ago
    You have to opt in with the lazy import keyword no matter what. This pep also prevents lazy import in try catch. I think your concern matters if ‘module’ itself has a lazy import that you want to check exists. With this you now need to be more rigorous in check those sub dependencies. This can already happen with non top level imports so it is not a necessarily a new issue, but could become more prevalent if there is an overall uptake in this feature for optional dependencies.
    • eesmith 1y ago
      My post was a response to the sedatk conjectural "I wonder how much things would break if all imports were lazy by default." I have zero concerns about this PEP and look forward to its implementation.
  • zahlman 1y ago
    With the LazyLoader technique I described at https://news.ycombinator.com/item?id=45467489 https://news.ycombinator.com/item?id=45467489 , there is no problem: >>> import nonexistent_module Traceback (most recent call last): File "<python-input-2>", line 1, in <module> import nonexistent_module File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1262, in _find_spec File "<python-input-0>", line 8, in find_spec base.loader = LazyLoader(base.loader) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'loader' The implementation should probably convert that exception back to ImportError for you, but the point is that the absence of an implementation can still be detected eagerly while the actual loading occurs lazily.
• ashu1461 1y ago
  There will be chaos, all code bases usually have tons of side effects.