3 ms·
Python has been cleaning up a number of really lethal problems like: (i) wrongly configured character encodings (suppose you incorporated somebody else's libra
by PaulHoule 2y ago
Python has been cleaning up a number of really lethal problems like:
(i) wrongly configured character encodings (suppose you incorporated somebody else's library that does a "print" and the input data contains some invalid characters that wind up getting printed; that "print" could crash a model trainer script that runs for three days if error handling is set wrong and you couldn't change it when the script was running, at most you could make the script start another python with different command line arguments)
(ii) site-packages; all your data scientist has to do is
pip install --user
the wrong package and they'd trashed all of their virtualenvs, all of their condas, etc. Over time the defaults have changed so pythons aren't looking into the site-packages directories but I wasted a long time figuring out why a team of data scientists couldn't get anything to work reliably
(iii) "python" built into Linux by default. People expected Python to "just work" but it doesn't "just work" when people start installing stuff with pip because you might be working on one thing that needs one package and another thing that needs another package and you could trash everything you're doing with python in the process of trying to fix it.
Unfortunately python has attracted a lot of sloppy programmers who think virtualenv is too much work and that it's totally normal for everything to be broken all the time. The average data scientist doesn't get excited when it crumbles and breaks, but you can't just call up some flakes to fix it. [1]
[1] https://www.youtube.com/watch?v=tiQPkfS2Q84 https://www.youtube.com/watch?v=tiQPkfS2Q84
- whatever1 2y agoI don’t exactly remember the situation but a user created a python module named error.py. Then in their main code they imported the said error.py but unfortunately numpy library also has an error.py. So the user was getting very funky behavior.
- PaulHoule 2y ago... it's tricky. In Java there's a cultural expectation that you name a package like package organization.dns.name.this.and.that; but real scalability in a module system requires that somebody else packages things up as package this.and.that; and you can make the system look at a particular wheel/jar/whatever and make it visible with a prefix you specify like package their.this.and.that; Programmers seem to hate rigorous namespace systems though. My first year programming Java (before JDK 1.0) the web site that properly documented how to use Java packages was at NASA and you still had people writing Java classes that were in the default package.
- mdaniel 2y agoBut let's all be real here: the ability of __init__.py to do FUCKING ANYTHING IT WANTS is insanity made manifest I am kind of iffy on golang's import (. "some/packge/for/side-effects") but at least it cannot suddenly mutate GOPATH[0]="/home/jimmy/lol/u/fucked" as one seems to be able to do on the regular with python I am acutely aware that is (programmer|package|organization|culture)-dependent but the very idea that one can do that drives us rigorous people stark-raving
- zahlman 2y agoIt took me a while to realize that you do mean __init__.py running when a package imports. But, you know, top-level code (which can do anything) runs when you import any Python module (the first time), and Python code doesn't have to be in a package to get imported. (The standard library depends on this.)
- vlovich123 2y ago> Programmers seem to hate rigorous namespace systems though Pretty much a nothing burger in Rust, so I disagree that items necessarily hate the concept. Maybe others haven’t done a good job with the UX?
- PaulHoule 2y agoThat. Forcing people to do it right from day one helps a lot. Also Rust attracts a programmer who is willing to accept some pain up front to save pain later, if your motto is "give me convenience or give me death" you might stick with C or Python. If you give people an "easy way out" it is very hard to compel them to do it more rigorously. Look at the history of C++ namespaces as well as the non-acceptance of various "Modula" languages and Ada back in the 1980s.
- hauntsaninja 2y agoYeah, that's not exactly the problematic situation... but the good news is I improved the Python's error message for this in 3.13. See https://docs.python.org/3/whatsnew/3.13.html#improved-error-messages https://docs.python.org/3/whatsnew/3.13.html#improved-error-...
- zahlman 2y agoYep, happens all the time with the standard library. Nowadays, third-party libraries have this issue much less because they can use relative imports except for their dependencies. But the Python standard library, for historical reasons, isn't organized into a package, so it can't do that. Here's a fun one (this was improved in 3.11, but some other names like `traceback.py` can still reproduce a similar problem): /tmp$ touch token.py /tmp$ py3.10 Python 3.10.14 (main, Jun 24 2024, 03:37:47) [GCC 11.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> help() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/opt/python/standard/lib/python3.10/_sitebuiltins.py", line 102, in __call__ import pydoc File "/opt/python/standard/lib/python3.10/pydoc.py", line 62, in <module> import inspect File "/opt/python/standard/lib/python3.10/inspect.py", line 43, in <module> import linecache File "/opt/python/standard/lib/python3.10/linecache.py", line 11, in <module> import tokenize File "/opt/python/standard/lib/python3.10/tokenize.py", line 36, in <module> from token import EXACT_TOKEN_TYPES ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/tmp/token.py) Related Stack Overflow Q&A (featuring an answer from me): https://stackoverflow.com/questions/36250353 https://stackoverflow.com/questions/36250353
- worik 2y ago> Python has been cleaning up a number of really lethal problems like I wish they would stick to semantic versioning tho. I have used two projects that got stuck in incompatible changes in the 3.x Python. That is a fatal problem for Python. If a change in a minor version makes things stop working, it is very hard to recommend the system. A lot of work has gone down the drain, by this Python user, trying to work around that
- meitham 2y agoI assume these breaking changes are in the stdlib and not in the python interpreter (the language), right? There was previous discussions about uncoupling the stdlib (python libraries) from the release and have them being released independently, but I can’t remember why that died off
- zahlman 2y ago> I assume these breaking changes are in the stdlib and not in the python interpreter (the language), right? That's the usual case, but it can definitely also happen because of the language - see https://stackoverflow.com/questions/51337939 https://stackoverflow.com/questions/51337939 . > There was previous discussions about uncoupling the stdlib (python libraries) from the release and have them being released independently, but I can’t remember why that died off This sort of thing is mainly a social problem.
- dagw 2y agoI assume these breaking changes are in the stdlib Well yes, but sometimes the features provided by stdlib can feel pretty 'core'. I remember that how dataclasses work for example changed and broke a lot of our code when we upgraded from 3.8 to 3.10, There have also been breaking changes in the C API. We have one project at work that is stuck at 3.11 since a third party dependency won't build on 3.12 without a rewrite.