5 ms·

I've love to see a revamp of the import system. It is a continuous source of pain points when I write Python. Circular imports all over unless I structure my pr

I've love to see a revamp of the import system. It is a continuous source of pain points when I write Python. Circular imports all over unless I structure my program explicitly with this in mind. Using python path hacks with `sys` etc to go up a directory too.

derr1 2y ago
Pathlib is your friend
baq 2y ago
For me 'from __future__ import no_cyclic_imports' would be good enough, if it's even possible
int_19h 2y ago
The biggest problem with Python imports is that the resolution of non-relative module names always prioritizes local files, even when the import happens in stdlib. This means that, for any `foo` that is a module name in stdlib, having foo.py in your code can break arbitrary modules in stdlib. For example, this breaks: # bisect.py ... # main.py import random with: Traceback (most recent call last): File ".../foo.py", line 1, in <module> import random File "/usr/lib/python3.12/random.py", line 62, in <module> from bisect import bisect as _bisect ImportError: cannot import name 'bisect' from 'bisect' This is very frustrating because Python stdlib is still very large and so many meaningful names are effectively reserved. People are aware of things like "sys" or "json", but e.g. did you know that "wave", "cmd", and "grp" are also standard modules? Worse yet is that these errors are not consistent. You might be inadvertently reusing an stdlib module name without even realizing it just because none of the stdlib (or third-party) modules that you import have it in their import graphs. Then you move on to a new version of Python or some of your dependencies, and suddenly it breaks because they have added an import somewhere. But even if you are careful about checking every single module name against the list of standard modules, a new Python version can still break you by introducing a new stdlib module that happens to clash with one of yours. For example, Python 3.9 added "graphlib", which is a fairly generic name.
- riezebos 2y ago
  I agree, it's unreasonable to expect devs to know the whole standard library. The VSCode extension Pylance does give a warning when this happens. I thought linters might also check this. The one I use doesn't, maybe the issue[0] I just created will lead to it being implemented. [0]: https://github.com/astral-sh/ruff/issues/13676 https://github.com/astral-sh/ruff/issues/13676
  - int_19h 2y ago
    One of the changes in Python 3.13 is that it'll warn you when that is the likely cause of breakage and explain how to fix it: https://docs.python.org/3.13/whatsnew/3.13.html#improved-error-messages https://docs.python.org/3.13/whatsnew/3.13.html#improved-err... But the problem remains, because these warnings - whether they come from linters or Python itself - can only warn you about existing stdlib modules. I'm not aware of any way to guard against conflicts with any future new stdlib modules being added.
- reinhardt 2y ago
  That's not a problem, let alone the biggest one. You should just use relative imports explicitly.
  - int_19h 2y ago
    It is a problem because stdlib does not use relative imports for other stdlib modules, and neither do most third-party packages, which then breaks you regardless of what you do in your code.
notatallshaw 2y ago
There was an attempt to make imports lazy: https://peps.python.org/pep-0690/ https://peps.python.org/pep-0690/ It was ultimately rejected due to issues with how it would need to change the dict object. IMO all the rejection reasons could be overcome with a more focused approach and implementation, but I don't know if there is anyone wishing to give it another go.
reinhardt 2y ago
Getting a cyclic import error is not a bug, it's a feature alerting you that your code structure is like spaghetti and you should refactor it to break the cycles.

derr1 2y ago

Pathlib is your friend

baq 2y ago

For me 'from __future__ import no_cyclic_imports' would be good enough, if it's even possible

int_19h 2y ago

The biggest problem with Python imports is that the resolution of non-relative module names always prioritizes local files, even when the import happens in stdlib. This means that, for any `foo` that is a module name in stdlib, having foo.py in your code can break arbitrary modules in stdlib. For example, this breaks: # bisect.py ... # main.py import random with: Traceback (most recent call last): File ".../foo.py", line 1, in <module> import random File "/usr/lib/python3.12/random.py", line 62, in <module> from bisect import bisect as _bisect ImportError: cannot import name 'bisect' from 'bisect' This is very frustrating because Python stdlib is still very large and so many meaningful names are effectively reserved. People are aware of things like "sys" or "json", but e.g. did you know that "wave", "cmd", and "grp" are also standard modules? Worse yet is that these errors are not consistent. You might be inadvertently reusing an stdlib module name without even realizing it just because none of the stdlib (or third-party) modules that you import have it in their import graphs. Then you move on to a new version of Python or some of your dependencies, and suddenly it breaks because they have added an import somewhere. But even if you are careful about checking every single module name against the list of standard modules, a new Python version can still break you by introducing a new stdlib module that happens to clash with one of yours. For example, Python 3.9 added "graphlib", which is a fairly generic name.

riezebos 2y ago
I agree, it's unreasonable to expect devs to know the whole standard library. The VSCode extension Pylance does give a warning when this happens. I thought linters might also check this. The one I use doesn't, maybe the issue[0] I just created will lead to it being implemented. [0]: https://github.com/astral-sh/ruff/issues/13676 https://github.com/astral-sh/ruff/issues/13676
- int_19h 2y ago
  One of the changes in Python 3.13 is that it'll warn you when that is the likely cause of breakage and explain how to fix it: https://docs.python.org/3.13/whatsnew/3.13.html#improved-error-messages https://docs.python.org/3.13/whatsnew/3.13.html#improved-err... But the problem remains, because these warnings - whether they come from linters or Python itself - can only warn you about existing stdlib modules. I'm not aware of any way to guard against conflicts with any future new stdlib modules being added.
reinhardt 2y ago
That's not a problem, let alone the biggest one. You should just use relative imports explicitly.
- int_19h 2y ago
  It is a problem because stdlib does not use relative imports for other stdlib modules, and neither do most third-party packages, which then breaks you regardless of what you do in your code.

notatallshaw 2y ago

There was an attempt to make imports lazy: https://peps.python.org/pep-0690/ https://peps.python.org/pep-0690/ It was ultimately rejected due to issues with how it would need to change the dict object. IMO all the rejection reasons could be overcome with a more focused approach and implementation, but I don't know if there is anyone wishing to give it another go.

reinhardt 2y ago

Getting a cyclic import error is not a bug, it's a feature alerting you that your code structure is like spaghetti and you should refactor it to break the cycles.