3 ms·

I recommend pyenv, pyenv-virtualenv, and poetry for Mac development these days. It’s a nice modern experience. If you don’t care about different Python version

I recommend pyenv, pyenv-virtualenv, and poetry for Mac development these days. It’s a nice modern experience. If you don’t care about different Python versions then the built in “python3 -m venv” is fine and you can drop pyenv. You can use Poetry to make and manage virtual envs too. If you don’t need lock files (say you are doing ML or solo scripting) then pip with requirements.txt is fine. Lock files start to become useful when you are collaborating with a team so you know you have exactly the same deps (so theoretically the same behavior).

crabbone 4y ago
Venv isn't built into Python. You are using Python distributions intended for developers, that's why you will usually have it. Python can be configured to create a distribution without it (which is what typically happen on Linux, where Python is part of the distribution). Pip with requirements.txt is never fine. It's always the wrong answer to any problem you may think you are solving with it. The problem is it's a very commonly given wrong answer, so, people are used to it. The reason why it's the wrong answer is this: You intend for your project to be deployed somehow somewhere. So, you do need a process to find the pieces of the project you are working on to put them where you are deploying them, be it a Docker image, a Python package, an RPM etc. But, pip + requirements.txt doesn't produce any of that. At best, you may hope to have it as an intermediary step in your process. Now, a very important piece of your deployment is the code you wrote. That's something that's not in the requirements.txt. You sort-of have "pip install -e ." to do this, but it's awfully bad for this purpose because instead of actually installing anything, it's a wrapper around "setup.py develop", which, in turn doesn't install anything, but modifies Python's path where it searches for installed packages. This has a lot of bad / unintended consequences. For example, if you accidentally introduced a dependency on an artifact on your filesystem, it may or may not translate properly into your deployment environment. This is especially bad when you need to use Python code linked with some native libraries, but on your system and on deployment system they are found in different places. It may result in you confusing the libraries (and versions) you think you are using with those actually being used. In order to avoid all of that, a better way is to create a distributable Python package (i.e. Wheel) and install that, but Wheel has its own way of specifying dependencies, which has nothing to do with requirements.txt. So, you either just don't need that, or try to combine the two somehow... ---- Bottom line: I package couple dozens Python packages for internal use. Those I have full control over don't have requirements.txt and don't use pip at all. They are easy to install and to test, portable between different environments etc. There are teams which produce their own Python packages which I have to package, and in many cases I wasn't able to convince them to abandon the bad (but popular) practices s.a. using pip or / and requirement.txt. This is a constant source of problems when it comes to dependency management. But, authors are convinced they are doing it right...
- Mehdi2277 4y ago
  It’s opposite. Venv is built into python and part of standard library. It is part of cpython repo and developed by python maintainers. Some Linux distributions remove it from standard library but windows/Mac always has it. The Linux distributions that have python without venv are considered incomplete python install and packaging community has considered not supporting that case in future. At moment pip tries to allow that case but in future may just require venv and consider Linux distributions that remove it as shipping a problematic python.
  - crabbone 4y ago
    > It’s opposite. Venv is built into python and part of standard library. Well, you simply don't know your stuff, but are overcome with confidence... where have I seen this before? Let's start with this: standard library (without quotes) means there's a standard that describes a collection of library functions that constitute a standard library. Python doesn't have a standard. Well, technically, there isn't really Python, there are Pythons. Each accepted PEP that deals with the language creates new language, they are just similar enough that they all, collectively, can be referred to as "Python" for many practical purposes. Python is notorious for its vague and incoherent use of nomenclature. It doesn't follow its own definitions, but it also doesn't create good definitions to begin with. Someone in Python's history decided to call a chunk of Python's token implementation "the standard library", and the term stuck. But, in reality, nothing makes code found in CPython repository a standard, nor is it even in any way prescriptive for other implementations. There's a bunch of other code that's optionally included or enabled in the "standard library", and there's no formal document that describes what should be included. In particular, there are a bunch of features that are only enabled by compilation switches (eg. various garbage collection helpers). And you cannot possibly argue that a mere presence of those features in CPython's source code somehow makes them necessarily a part of the "standard library", because then the very definition of "standard library" becomes contingent on the compiler options you provided when compiling Python. Now, whatever pip authors think is "standard library" or "complete" or "incomplete" Python distribution shouldn't bother anyone really. Their opinion is just as valid as mine or that of my cat: none of us defines nor controls what any of those things are. Subsequently: who told you that there aren't any Windows or Mac distributions that don't have venv? I can easily make however many I want. But, most importantly, again, who cares? Why does it even matter? To show that venv isn't a necessary part of something, it's enough to find a single example where it's not included, you don't need to search for places where it's included...

crabbone 4y ago

Venv isn't built into Python. You are using Python distributions intended for developers, that's why you will usually have it. Python can be configured to create a distribution without it (which is what typically happen on Linux, where Python is part of the distribution). Pip with requirements.txt is never fine. It's always the wrong answer to any problem you may think you are solving with it. The problem is it's a very commonly given wrong answer, so, people are used to it. The reason why it's the wrong answer is this: You intend for your project to be deployed somehow somewhere. So, you do need a process to find the pieces of the project you are working on to put them where you are deploying them, be it a Docker image, a Python package, an RPM etc. But, pip + requirements.txt doesn't produce any of that. At best, you may hope to have it as an intermediary step in your process. Now, a very important piece of your deployment is the code you wrote. That's something that's not in the requirements.txt. You sort-of have "pip install -e ." to do this, but it's awfully bad for this purpose because instead of actually installing anything, it's a wrapper around "setup.py develop", which, in turn doesn't install anything, but modifies Python's path where it searches for installed packages. This has a lot of bad / unintended consequences. For example, if you accidentally introduced a dependency on an artifact on your filesystem, it may or may not translate properly into your deployment environment. This is especially bad when you need to use Python code linked with some native libraries, but on your system and on deployment system they are found in different places. It may result in you confusing the libraries (and versions) you think you are using with those actually being used. In order to avoid all of that, a better way is to create a distributable Python package (i.e. Wheel) and install that, but Wheel has its own way of specifying dependencies, which has nothing to do with requirements.txt. So, you either just don't need that, or try to combine the two somehow... ---- Bottom line: I package couple dozens Python packages for internal use. Those I have full control over don't have requirements.txt and don't use pip at all. They are easy to install and to test, portable between different environments etc. There are teams which produce their own Python packages which I have to package, and in many cases I wasn't able to convince them to abandon the bad (but popular) practices s.a. using pip or / and requirement.txt. This is a constant source of problems when it comes to dependency management. But, authors are convinced they are doing it right...

Mehdi2277 4y ago
It’s opposite. Venv is built into python and part of standard library. It is part of cpython repo and developed by python maintainers. Some Linux distributions remove it from standard library but windows/Mac always has it. The Linux distributions that have python without venv are considered incomplete python install and packaging community has considered not supporting that case in future. At moment pip tries to allow that case but in future may just require venv and consider Linux distributions that remove it as shipping a problematic python.
- crabbone 4y ago
  > It’s opposite. Venv is built into python and part of standard library. Well, you simply don't know your stuff, but are overcome with confidence... where have I seen this before? Let's start with this: standard library (without quotes) means there's a standard that describes a collection of library functions that constitute a standard library. Python doesn't have a standard. Well, technically, there isn't really Python, there are Pythons. Each accepted PEP that deals with the language creates new language, they are just similar enough that they all, collectively, can be referred to as "Python" for many practical purposes. Python is notorious for its vague and incoherent use of nomenclature. It doesn't follow its own definitions, but it also doesn't create good definitions to begin with. Someone in Python's history decided to call a chunk of Python's token implementation "the standard library", and the term stuck. But, in reality, nothing makes code found in CPython repository a standard, nor is it even in any way prescriptive for other implementations. There's a bunch of other code that's optionally included or enabled in the "standard library", and there's no formal document that describes what should be included. In particular, there are a bunch of features that are only enabled by compilation switches (eg. various garbage collection helpers). And you cannot possibly argue that a mere presence of those features in CPython's source code somehow makes them necessarily a part of the "standard library", because then the very definition of "standard library" becomes contingent on the compiler options you provided when compiling Python. Now, whatever pip authors think is "standard library" or "complete" or "incomplete" Python distribution shouldn't bother anyone really. Their opinion is just as valid as mine or that of my cat: none of us defines nor controls what any of those things are. Subsequently: who told you that there aren't any Windows or Mac distributions that don't have venv? I can easily make however many I want. But, most importantly, again, who cares? Why does it even matter? To show that venv isn't a necessary part of something, it's enough to find a single example where it's not included, you don't need to search for places where it's included...