4 ms·

I usually make a venv in ~/.venv and then activate it at the top of any python project. Makes it much easier to deal with dependencies when they're all in one p

by eyegor 3y ago

I usually make a venv in ~/.venv and then activate it at the top of any python project. Makes it much easier to deal with dependencies when they're all in one place.

synergy20 3y ago
second this and it's what I do on all Linux distros, just run it inside .venv as the site-installation. if you need extra dependencies that pip can not do well in the .venv case, Conda can help with its own and similar site-based installation. I don't know how it is different in the python installation case between ubuntu and debian, they seem the same to me.
gjvc 3y ago
i am a big fan of .venv/ -- except when it takes ~45 mins to compile the native extension code in question -- then I want it all pre-packaged.
- dgroshev 3y ago
  At this stage [0], uncompiled native extensions are not yet a bug, but a definite oversight of the maintainer. They should come as precompiled wheels [0]: https://pythonwheels.com https://pythonwheels.com
  - doliveira 3y ago
    Honestly I don't think I've ever used a precompiled package in Python. Every single C stuff seems to take ages and requires all that fun stuff of installing native system dependencies. Edit: skimming through this page, precompiling seems like an afterthought, and the linked packages don't even seem to mention how to integrate third-party libraries. So I guess I can see why it doesn't deliver on its promises.
    - toyg 3y ago
      Probably a function of the specific set of packages you use, or the pip options you specify. Pretty much all the major C packages come as wheels these days.
      - doliveira 3y ago
        They all come as wheels, they just aren't precompiled.
        toyg 3y ago
        I honestly can't remember the last time I had to compile anything, and I am on Windows.
        orf 3y ago
        Can you link one that comes as a wheel but is really a source distribution?
    - dgroshev 3y ago
      You can try pip install pillow for a good example of how it works. I suspect there's a strong survivorship bias here, as you'd only notice the packages that don't ship with wheels.
      - doliveira 3y ago
        Yeah, perhaps. One I remember from last year is the cryptography and numpy package, for instance. Now they do seem to ship with binary wheels, at least for my current Python and Linux version. Kerberos and Hadoop stuff obviously still doesn't, though. I guess the joke's on me for being stuck in this stack...
        montecarl 3y ago
        In order for a wheel to be used instead of a source distribution there needs to be one that matches your environment. For numpy you can take a look at the wheels for their latest release[1]. The filename of a wheel specifies where it can be used. Let's take an example: numpy-1.25.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl This specifies cpython 3.9, linux, glibc 2.17 or higher, and x86_64 cpu. Looking through the list you will see that the oldest cpython supported is 3.9. So if you are running with an older version of python you will have to build from source. I just learned a bit more about this recently because I could not figure out why PyQt6 would not install on my computer. It turned out my glibc was too old. Finally upgraded from Ubuntu 18.04. [1] https://pypi.org/project/numpy/1.25.2/#files https://pypi.org/project/numpy/1.25.2/#files
    - martsa1 3y ago
      Try `--only-binary :all:` to force pip to ignore sdist packages, might help avoid those slow compilations.
- j0057 3y ago
  It's a good idea to be caching sdists and wheels — for resilience against PyPI downtime, for left-pad scenarios, and even just good netiquette — and for packages that don't have a wheel for your environment, you can fairly easily build that wheel yourself and stick it into the cache.

by eyegor 3y ago

I usually make a venv in ~/.venv and then activate it at the top of any python project. Makes it much easier to deal with dependencies when they're all in one place.

synergy20 3y ago

second this and it's what I do on all Linux distros, just run it inside .venv as the site-installation. if you need extra dependencies that pip can not do well in the .venv case, Conda can help with its own and similar site-based installation. I don't know how it is different in the python installation case between ubuntu and debian, they seem the same to me.

gjvc 3y ago

i am a big fan of .venv/ -- except when it takes ~45 mins to compile the native extension code in question -- then I want it all pre-packaged.

dgroshev 3y ago
At this stage [0], uncompiled native extensions are not yet a bug, but a definite oversight of the maintainer. They should come as precompiled wheels [0]: https://pythonwheels.com https://pythonwheels.com
- doliveira 3y ago
  Honestly I don't think I've ever used a precompiled package in Python. Every single C stuff seems to take ages and requires all that fun stuff of installing native system dependencies. Edit: skimming through this page, precompiling seems like an afterthought, and the linked packages don't even seem to mention how to integrate third-party libraries. So I guess I can see why it doesn't deliver on its promises.
  - toyg 3y ago
    Probably a function of the specific set of packages you use, or the pip options you specify. Pretty much all the major C packages come as wheels these days.
    - doliveira 3y ago
      They all come as wheels, they just aren't precompiled.
      - toyg 3y ago
        I honestly can't remember the last time I had to compile anything, and I am on Windows.
      - orf 3y ago
        Can you link one that comes as a wheel but is really a source distribution?
  - dgroshev 3y ago
    You can try pip install pillow for a good example of how it works. I suspect there's a strong survivorship bias here, as you'd only notice the packages that don't ship with wheels.
    - doliveira 3y ago
      Yeah, perhaps. One I remember from last year is the cryptography and numpy package, for instance. Now they do seem to ship with binary wheels, at least for my current Python and Linux version. Kerberos and Hadoop stuff obviously still doesn't, though. I guess the joke's on me for being stuck in this stack...
      - montecarl 3y ago
        In order for a wheel to be used instead of a source distribution there needs to be one that matches your environment. For numpy you can take a look at the wheels for their latest release[1]. The filename of a wheel specifies where it can be used. Let's take an example: numpy-1.25.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl This specifies cpython 3.9, linux, glibc 2.17 or higher, and x86_64 cpu. Looking through the list you will see that the oldest cpython supported is 3.9. So if you are running with an older version of python you will have to build from source. I just learned a bit more about this recently because I could not figure out why PyQt6 would not install on my computer. It turned out my glibc was too old. Finally upgraded from Ubuntu 18.04. [1] https://pypi.org/project/numpy/1.25.2/#files https://pypi.org/project/numpy/1.25.2/#files
  - martsa1 3y ago
    Try `--only-binary :all:` to force pip to ignore sdist packages, might help avoid those slow compilations.
j0057 3y ago
It's a good idea to be caching sdists and wheels — for resilience against PyPI downtime, for left-pad scenarios, and even just good netiquette — and for packages that don't have a wheel for your environment, you can fairly easily build that wheel yourself and stick it into the cache.