The dirty secret about Python packaging problems is that a lot of them... are not relatated to packaging. Python has first and foremost a big __bootstrapping__

The dirty secret about Python packaging problems is that a lot of them... are not relatated to packaging. Python has first and foremost a big __bootstrapping__ problem, and the consequences of that are often seen when you try to install or use dependancies. So people conclude that packaging sucks. Now don't get me wrong, packaging have some problems, but compared to most languages out there, it's actually pretty good. But people cannot enjoy that, because they would have to install, setup and run python in a perfect sequence. So what's wrong with Python bootstrapping, and why does it affect packages? Well, as it is often the case in FOSS, it stems from too much diversity. Diversity is tooted as this one great thing, but its benefits also come with a cost. - there are too many ways to install Python. Official installer, anaconda, Windows Store, pyenv, homebrew, Linux repositories... They all come with different configurations and lead to problems, such as anaconda being incompatible with pip or Linux default setup not providing pip. - it's very common to have numerous and diverse python installed on your machine. Different versions, different distributions, sometimes embeded Python... Now it's your job to know which Python you start and how. But people have no idea you should use "py -3.10 -m pip --user install black" and not "pip install black" on Windows if you want a clean install at the system level. Yes, this weird command is the only clean way to do it outside of a venv. I'm not making it up. - the variation between the different ways to run Python and the commands lead to confusion. "python", "python3.10", "py -3.10", "pip", "pip3", "python -m pip", "sudo pip"... There are many wrong ways to do it, and not many good ones, and it all depends of the distribution, the OS, and wether your are in a venv or not. So how does that affect packaging? Well, people will get a lot of "command not found", and so they will google and paste commands until one works. There is too much to know, and they want a solution, not learning the whole graph of possibilities. Eventually they do something that seems to work, but they probably installed with admin rights, or in anaconda with pip, or in the wrong Python version. On import or on run, something will then break. Maybe now. Or worse, maybe in one month. And they have no way to understand why. What's the short term solution for you? - Always use the official installer on Windows and Mac. Don't use anaconda, don't use homebrew, don't use pyenv. - If you have to use Anaconda (I would advise not to), don't use virtualenv and pip, use their own tool: conda and the anaconda prompt. You will be limited in packages, but it will work. Don't mix conda and pip. - On Linux, you will have to figure out what magic packages you need to install to get a full Python setup, because linux distributions authors don't provide a full Python by default. It's different for each distribution and Python version sometimes. It must include pip, venv, setuptools, tkinter, ssl and headers. Good luck. - Don't use Python outside of a virtualenv. Ever. I know how to do that correctly. You probably don't. So don't. Always create a venv for each project you work on, and activate it before doing anything. From there, you can use pip and Python in peace. E.G: Don't install black or jupyter outside of a venv. You want to use poetry? Don't let it create your venv. Create one, activate it, then install poetry with pip, the use poetry from there. I'm not kidding. - On Windows, the command is "py -3.10 -m venv name_of_your_venv" to create it, with "-3.10" to be replaced with the version of Python you want. On Mac and Linux, the command is "python3.10 -m venv name_of_your_venv". Yes. Not a joke. - To activate it, on Windows it will be name_of_your_venv\Scripts\activate", on Mac and Linux, "source name_of_your_venv/bin/activate.sh". Any variation on this (including using -m or pipx, which I used to recommend but don't anymore) assumes you know what you are doing. What's the long term solution? I give some hints here: https://twitter.com/bitecode_dev/status/1567790837848743937 https://twitter.com/bitecode_dev/status/1567790837848743937 And meanwhile, take the official survey on packaging, this will help pin things down for the team: https://www.surveymonkey.co.uk/r/M5XKQCT https://www.surveymonkey.co.uk/r/M5XKQCT