4 ms·

All my code which uses import probing would fail, such as fallbacks: try: import module except ImportError: import slow_module as module Condition

All my code which uses import probing would fail, such as fallbacks: try: import module except ImportError: import slow_module as module Conditional support testing would also break, like having tests which only run if module2 is available: try: import module2 except ImportError: def if_has_module2(f): return unittest.skip("module2 not available")(f) else: def if_has_module2(f): return f @if_has_module2 class TestModule2Bindings(.... The proto-PEP also gives an example of using with suppress_warnings(): import module3 where some global configuration changes only during import. In general, "import this" and "import antigravity" - anything with import side-effects - would stop working. Oh, and as the proto-PEP points out, changes to sys.path and others can cause problems because of the delay between time of lazy import and time of resolution. Then there's code where you do a long computation then make use of a package which might not be present. import database # remember to install!! import qcd_simulation universe = qcd_simulation.run(seconds = 30*24*60*60) database.save(universe) All of these would be replaced with "import module; module.__name__" or something to force the import, or by an explicit use to __import__.

npage97 1y ago
You have to opt in with the lazy import keyword no matter what. This pep also prevents lazy import in try catch. I think your concern matters if ‘module’ itself has a lazy import that you want to check exists. With this you now need to be more rigorous in check those sub dependencies. This can already happen with non top level imports so it is not a necessarily a new issue, but could become more prevalent if there is an overall uptake in this feature for optional dependencies.
- eesmith 1y ago
  My post was a response to the sedatk conjectural "I wonder how much things would break if all imports were lazy by default." I have zero concerns about this PEP and look forward to its implementation.
zahlman 1y ago
With the LazyLoader technique I described at https://news.ycombinator.com/item?id=45467489 https://news.ycombinator.com/item?id=45467489 , there is no problem: >>> import nonexistent_module Traceback (most recent call last): File "<python-input-2>", line 1, in <module> import nonexistent_module File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1262, in _find_spec File "<python-input-0>", line 8, in find_spec base.loader = LazyLoader(base.loader) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'loader' The implementation should probably convert that exception back to ImportError for you, but the point is that the absence of an implementation can still be detected eagerly while the actual loading occurs lazily.
- eesmith 1y ago
  Ahh, so you do the find first, and keep that around before loading. I have bad memories of using a network filesystem where my Python app's startup time was 5 or more seconds because of all the small file lookups for the import were really slow. I fixed it by importing modules in functions, only when needed, so the time went down to less than a second. (It was even better using a zipimport, but for other reasons we didn't use that option.) If I understand things correctly, your code would have the same several-second delay as it tries to resolve everything?
  - zahlman 1y ago
    Yes, if checking for a file is slow, then checking for a file is slow. If you need to know up front whether the module exists, then you can't get around using the "figure out whether the module exists" machinery up front. And if the definition of "a module exists" includes cases where the module is represented by a file whose existence you have to check for, then there's no getting around that, either. (Trying to do "fallback" logic with lazily-loaded modules is also susceptible to race conditions, of course. What if someone defines the module before you try to use it?)

npage97 1y ago

You have to opt in with the lazy import keyword no matter what. This pep also prevents lazy import in try catch. I think your concern matters if ‘module’ itself has a lazy import that you want to check exists. With this you now need to be more rigorous in check those sub dependencies. This can already happen with non top level imports so it is not a necessarily a new issue, but could become more prevalent if there is an overall uptake in this feature for optional dependencies.

eesmith 1y ago
My post was a response to the sedatk conjectural "I wonder how much things would break if all imports were lazy by default." I have zero concerns about this PEP and look forward to its implementation.

zahlman 1y ago

With the LazyLoader technique I described at https://news.ycombinator.com/item?id=45467489 https://news.ycombinator.com/item?id=45467489 , there is no problem: >>> import nonexistent_module Traceback (most recent call last): File "<python-input-2>", line 1, in <module> import nonexistent_module File "<frozen importlib._bootstrap>", line 1360, in _find_and_load File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 1262, in _find_spec File "<python-input-0>", line 8, in find_spec base.loader = LazyLoader(base.loader) ^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'loader' The implementation should probably convert that exception back to ImportError for you, but the point is that the absence of an implementation can still be detected eagerly while the actual loading occurs lazily.

eesmith 1y ago
Ahh, so you do the find first, and keep that around before loading. I have bad memories of using a network filesystem where my Python app's startup time was 5 or more seconds because of all the small file lookups for the import were really slow. I fixed it by importing modules in functions, only when needed, so the time went down to less than a second. (It was even better using a zipimport, but for other reasons we didn't use that option.) If I understand things correctly, your code would have the same several-second delay as it tries to resolve everything?
- zahlman 1y ago
  Yes, if checking for a file is slow, then checking for a file is slow. If you need to know up front whether the module exists, then you can't get around using the "figure out whether the module exists" machinery up front. And if the definition of "a module exists" includes cases where the module is represented by a file whose existence you have to check for, then there's no getting around that, either. (Trying to do "fallback" logic with lazily-loaded modules is also susceptible to race conditions, of course. What if someone defines the module before you try to use it?)