31 ms·

Announcing Pipenv

664 points by imkevinxu 10y ago

shakna 10y ago
> I wrote a new tool this weekend, called pipenv. > It harnesses Pipfile, pip, and virtualenv into one single toolchain. It features very pretty terminal colors. For a weekend project, this has some very nice things. Which removes the need for me to run my own project that basically does these things... In more or less, a worse way. Everything I've come to expect from Reitz, and hopefully it'll gain some decent ground like other projects of the same author.
command_tab 10y ago
See also: https://github.com/pypa/pipfile https://github.com/pypa/pipfile I'm glad to see Python getting the same attention as other modern package managers. This is all great work!
mikhuang 10y ago
Uninstall by default removing everything seems a little scary. Otherwise looks really neat, looking forward to trying it
- bdukic 10y ago
  I wanted to comment on the same thing, IMHO something like `pip uninstall --all` would be a better choice. Great work otherwise!
choxi 10y ago
Is this like Ruby's Bundler for Python? I've just been getting into Python and am really glad to see this, thanks for creating it!
- uranusjr 10y ago
  Without some shim like bundler exec, but yeah, you can say that.
- igravious 10y ago
  Very similar. I think Pipenv improves on Bundler by leveraging Virtualenv and Ruby doesn't have a per project equivalent to Virtualenv that I'm aware of. You can set the path config variable of Bundler to not place the project Gems in a central location which I think is cleaner and try to remember to always do now. It would be _super_ interesting if the Python and Ruby communities got together to harmonize every last detail of their packaging toolchain. Who is in?
  - sametmax 10y ago
    Actually we could also learn from: - the JS community. npm dependancy graph, webpack resolver and yarn performances; - the rust community like with cargo.
    - K0nserv 10y ago
      Bundler does proper dependency resolution using Molinillo[0] which is also used by CocoaPods[1]. This is definitely something that other package managers can stand to adopt. 0: https://github.com/CocoaPods/Molinillo https://github.com/CocoaPods/Molinillo 1: http://cocopods.org/ http://cocopods.org/
      - sametmax 10y ago
        I have a big doubt about Molinillo. The readme doesn't even link to a documentation. Does it allow you to update one package or the executable version and get a list of all the stuff to update ?
      - igravious 10y ago
        This is exactly what I was trying to get at with my original comment. There needs to be a lot more sharing of package management tools and techniques across language boundaries. Generic platform- and language-agnostic tools and algorithms are a step in the right direction!
  - mhw 10y ago
    > Ruby doesn't have a per project equivalent to Virtualenv that I'm aware of. The nearest equivalent is to place a file called '.ruby-version' in the top level directory, containing the version number of the Ruby you want to use. Version numbers come from https://github.com/rbenv/ruby-build/tree/master/share/ruby-build https://github.com/rbenv/ruby-build/tree/master/share/ruby-b.... rbenv, chruby and rvm all support .ruby-version. One difference from virtualenv is that the Ruby version managers share single installations of each version of Ruby. My understanding from occasional use of Virtualenv is that it copies the python installation into a new per-project subdirectory, which seems a bit wasteful to me. > You can set the path config variable of Bundler to not place the project Gems in a central location which I think is cleaner and try to remember to always do now. Yes, this is what I do. It gives me a shared 'clean' Ruby installation of the right version, plus a project-specific copy of all the gems the project depends on. To me this provides the best trade off between project isolation and not duplicating the whole world. You can set bundler up so this is done automatically by creating '~/.bundle/config' containing --- BUNDLE_PATH: "vendor/bundle" BUNDLE_BIN: ".bundle/bin" (The BUNDLE_PATH one is the important one; see 'bundle config --help' for other options.)
zyxzkz 10y ago
What took Python so long to get a tool like this?
- softinio 10y ago
  Kenneth was busy doing other stuff :-)
- sametmax 10y ago
  Did you work on it ? No. That's why. We can't always wait for one famous guy to do it.
noway421 10y ago
This is very interesting! I had exact same question how to do it in python just a while ago! http://stackoverflow.com/questions/41427500/creating-a-virtualenv-with-preinstalled-packages-as-in-requirements-txt http://stackoverflow.com/questions/41427500/creating-a-virtu... Glad that someone thought about similar thing and made a tool to solve it!
gourneau 10y ago
Hey other Reitz fans. Make sure to check out his newish podcast series: https://www.kennethreitz.org/import-this/ https://www.kennethreitz.org/import-this/
renesd 10y ago
Neat. Now for questions and comments. Often people have a requirements.live.txt, or other packages depending on the environment. Is that handled somehow? Can we use different files or sections? [ED: yes, different sections] Still wondering to myself if this is worth the fragmentation for most people using requirements.txt ? Perhaps the different sections could have a "-r requirements.txt" in there, like how requirements.dev.txt can have "-r requirements.txt". [ED: the pipfile idea seems to have quite some people behind it, and pip will support it eventually. Seems it will be worth it to standardise these things. requirements.txt is a less jargony name compared to Pipfile though, and has a windows/gui friendly extension.] Other tools can set up an environment, download stuff, and run the script. Will pipenv --shell somescript.py do what I want? (run the script with the requirements it needs). ((I guess I could just try it.)) [ED: doesn't seem so] Why Pipfile with Caps? Seems sort of odd for a modern python Thing. It looks like a .ini file? [ED: standard still in development it seems. TOML syntax.] With a setup.py set up, all you need to do is `pip install -e .` to download all the required packages. Or `pip install somepackage`. Lots of people make the setup.py file read the requirements.txt. Do you have some command for handling this integration? Or is this needed to be done manually? [ED: seems no considering about this/out of scope.] Is there a pep? [ED: too early it seems.]
- kenneth_reitz 10y ago
  Check out https://github.com/pypa/pipfile https://github.com/pypa/pipfile for more info. The format is TOML. It is mainly intended for deployments (e.g. web applications)
  - renesd 10y ago
    Thanks.
    - kenneth_reitz 10y ago
      Also `pipenv install -r requirements.txt` is supported, for importing.
      - 10y ago
btashton 10y ago
Interesting. I have been leveraging tox to provide a lot of what this seems to give you, but it certainly has been more of a hack than a solution.
auston 10y ago
I can only express my gratitude, thank you Kenneth!
therealmarv 10y ago
Currently using fish and virtualfish. This seems incompatible to non bash shells. Did somebody tested?
- kenneth_reitz 10y ago
  Fish support was just added!
  - deleted 10y ago
    [deleted]
jaybuff 10y ago
See also, pex: https://www.youtube.com/watch?v=NmpnGhRwsu0 https://www.youtube.com/watch?v=NmpnGhRwsu0
joelbondurant 10y ago
Has anyone in the Python community heard of Docker?
therealmarv 10y ago
For people who want to do it right without using an additional tool read this: setup.py vs. requirements.txt by Donald Stufft https://caremad.io/posts/2013/07/setup-vs-requirement/ https://caremad.io/posts/2013/07/setup-vs-requirement/
- yeukhon 10y ago
  I gave up populating requirements in setup.py. I just use multiple requirements.txt. This article has been debated for years already and there is absolutely no right / wrong.
korijn 10y ago
Windows support?
- kenneth_reitz 10y ago
  coming soon, hopefully via pull request!
cderwin 10y ago
This is great, but sometimes I think that python needs a new package manager from scratch instead of more tools trying to mix and mash a bunch of flawed tools together in a way that's palatable by most of us. Python packaging sucks, the whole lot of it. Maybe I'm just spoiled by rust and elixir, but setuptools, distutils, pip, ez_install, all of it is really subpar. But of course everything uses pypi and pip now, so it's not like any of it can actually be replaced. The state of package management in python makes me sad. I wish there was a good solution, but I just don't see it. Edit: I don't mean to disparage projects like this and pipfile. Both are great efforts to bring the packaging interface in line with what's available in other languages, and might be the only way up and out of the current state of affairs.
- sametmax 10y ago
  I strongly agree but this can work now and is a big improvement over what we currently have. So while I would literally pay to see somebody work on a better package manager (which can generate exe, deb and use a conf file indead of .py), this is a good filler.
  - X-Istence 10y ago
    Why should it generate a deb file? How is this useful on anything but Debian based systems? Why exe? How do you package libraries using this new tool you are envisioning?
    - sametmax 10y ago
      You do package lib the same way as before, although cargo like dependency handling would be a nice thing. Especially for upgrades. But a good package manager should ALSO allow you to produce a: - a stand alone executable for most OS. - a standard package for major OS (msi, deb, snap, rpm, dmg, etc). Doing that right now with Python requires you to setup stuff like nuikta and the like. It works but it's much harder than it should be.
      - snuxoll 10y ago
caconym_ 10y ago
I will definitely be trying this out. Python version and package management is a dumpster fire that wastes gobs of my time on the regular. I'll try anything that promises to end the pain.
sametmax 10y ago
I was really not a fan of the last "made in Reitz" project Maya. But this, I really can get along. The whole things make it way easier to get started for a beginner. Now more activate. No more wondering about virtualenv. Automatic lock files are great since no project I know of use them since they are not well understood. It's like node_packages (easy and obvious), but cleaner (no implicit magic). Like.
zephyrfalcon 10y ago
A bit off-topic, but what font is used in the video/animated GIF?
- kenneth_reitz 10y ago
  Operator Mono! https://www.kennethreitz.org/essays/test-driving-a-200-programming-font-operator-mono https://www.kennethreitz.org/essays/test-driving-a-200-progr...
zoul 10y ago
I always wonder if this could be done once and for all languages, instead of Ruby making bundler, Haskell Cabal sandboxes or stack, Perl Brew, etc. Is this where Nix is going?
- HeyImAlex 10y ago
  I don't know, different languages and environments need different things. I'm not sure that "package management" is as generic as it seems at first glance.
- toyg 10y ago
  You make it one tool, and sysadmins will instantaneously lock it down. These package managers, in most cases, are developer tools built to get around system-wide locks on libraries; the more you centralize them, the more likely it is they will get locked down, and then someone will build tools to get around that, and so on and so forth.
  - olejorgenb 10y ago
    As one of the few package mangers, nix can be used without root access.
- olejorgenb 10y ago
  In some way, absolutely. You can easily get a nice isolated python environment with some packages in nix without using pip, pyenv, etc. `nix-shell -p python pythonPackages.numpy ...` So far I think it works quite well for most languages as long the needed packages are in nixpkgs. Some of the tooling could be better, but the underlying model seems sound. I'm not really convinced language-specific package managers are needed. Nix isn't perfect yet, but it has come a long way.
nejdetckenobi 10y ago
normally I use virtualenvwrapper and that makes a virtualenv directory for all virtualenvs you create with it. before that, I always create my projects' venvs inside my project hierarchy. I had a dilemma about it. But after all, you can not move your venv directory unless you use `--relocatable` option. So, anyone have a strong argument about creating venvs inside your project directory?
- toyg 10y ago
  I just keep venvs in the project folder and add them to .gitignore. --relocatable never worked well for me, I just keep my requirements.txt up to date so it's trivial to blow away a venv and recreate it if necessary (python3 -m venv env && source env/bin/activate && pip install -r requirements.txt). I find tools like virtualenvwrapper and this one from Kenneth tend to solve issues I don't really have. A little bit of repetitive typing here and there is ok to burn knowledge into my mind; and less leaky abstractions I have to deal with, the better.
  - sametmax 10y ago
    It solves more than that: - beginers don't have to understand the whole virtualenv shenanigans. I use pew myself to replace virtualenvwrapper but I will switch to pipenv just to ease the pain of team member joining in. - it enforces good dependency management practice with the toml file and lock file. This is an issue in almost all project I worked on, including ones from Python experts. We all use only requirements.txt file out of convenience, and never lock. - it's one tool to do all the packaging stuff. No need for pip and virtualenv and a wrapper. You got one command.
- icebraining 10y ago
  I've always found the whole virtualenv stuff so superfluous. Do we really need all the machinery with shell scripts and static paths? We just use a directory where we keep our dependencies. It's a matter of: mkdir libs pip install -t libs <package> # then to run PYTHONPATH=libs python app.py From what I can tell, this accomplishes everything a venv does (except bringing the Python interpreter itself along) without requiring any extra tools or conventions to learn.
mulla111 10y ago
https://goo.gl/smCgbj https://goo.gl/smCgbj
deleted 10y ago
[deleted]
istoica 10y ago
Finally someones does it! I was using: pip -t .pip in my code, avoiding virtual-env completely, but that was not enough and incomplete. As this is not cross platform and it would be nice to switch between Linux/Windows while coding to maintain platform compatibility, can the virtualenv envs be created with a os platform & subsystem prefix ? for example, having multiple envs at once: - env/posix/bin/activate - env/nt/Scripts/activate.bat
wyldfire 10y ago
I haven't really used requirements.txt because I found that I could install 'extra' and 'test' specific content based on args to setup() in my setup.py. It seems more like the Right Thing than requirements.txt, from what I can tell. At first glance, this doesn't seem to offer anything beyond what I already see from setup(). What am I missing? It's unfortunate that CPython gave us distutils and took a very long time to converge on a built-in successor (setuptools?) that gives the right composability.
helb 10y ago
> --three / --two Use Python 3/2 when creating virtualenv. I use Python 2.7, 3.4, and 3.5 on various projects. Is there a way to choose between 3.4 and 3.5 using Pipenv? I'm using something like this with virtualenv: $ virtualenv -p `which python3.5` .venv
- iheredia 10y ago
  Just a comment. Your command is equivalent to $ virtualenv -p python3.5 .venv
  - emeraldd 10y ago
    Is that true? His command would evaluate the path at the time which is run, but yours seems to evaluate the path at the time python3.5 is run. Subtle, but very different results.
    - kyrias 10y ago
      virtualenv resolves the interpreter when you run virtualenv, and if you specify a different interpreter with `-p` it will resolve the path to the interpreter and then run virtualenv again using it.
- jjawssd 10y ago
  No, but you could be using pyenv instead of the system package manager because it is far cleaner
throw2016 10y ago
I think an app should not expose end users to its dependencies. That leaves the end user with a lot of pain figuring out versions of dependencies and god forbid you need to compile some dep then you need a build environment and its dependencies any of which can fail in this chain leaving a very unpleasant and even hostile end user experience. Ruby and Node apps are particularly guilty of this pulling in sometimes hundreds of packages some of which need compilation. Compare that to a Go binary which is download and use. These things can get very complicated very fast even for developers or system folks let alone end users who may not be intimately familiar with that specific ecosystem.
JeremyBanks 10y ago
> Otherwise, whatever $ which python will be the default. This is a bit strange because the python binary is always supposed to be Python 2. The Python 3 binary is supposed to be named python3. Some distributons don't follow this, but they're the weird non-conformant ones; it's not a behaviour that should really be relied on.
- jjawssd 10y ago
  Isn't `python` in Ubuntu a Python 3?
  - anuragsoni 10y ago
    Not yet. At least on ubuntu 16.10 python still defaults to `python2`
- jacobmischka 10y ago
  > always supposed to be Python 2 This is not correct. It's a symlink to python2 on systems that rely on calls to python to be python2. On modern systems, python is usually a symlink to python3. This is the case on Arch Linux and I believe other recent distro releases.
  - JeremyBanks 10y ago
    Arch is the oddball. I'm not aware of anybody else who's made the switch. Given that they're not mutually compatible except in rare cases, it's a very silly thing to do. You can upgrade GCC with the same name because you know it will handle most of the same input. If you do that with Python you're breaking tons of existing scripts for very close to zero benefit. Why would you do that?
    - baq 10y ago
      I'm pretty sure Ubuntu 16.04 is Python 3 by default and you have to ask for 2. edit: nope, been imagining things. too much time in venvs i guess. need to get more sleep :)
olejorgenb 10y ago
Seems to be a python specific nix-shell like tool? With nix[OS] you just run `nix-shell -p python[2,3] python[2,3]Pacakges.numpy ...` to get an environment with the required packages. Of course this requires that the python library is packaged in nix, but in my experience the coverage is quite good, and it's not very hard to write packages once you get the hang of it. It also possible (but currently a bit clumsy in some ways) to set up named and persistent environments.
georgeaf99 10y ago
LinkedIn has a similar open source project that is much more mature. It builds on Gradle features to manage complex dependencies and build Python artifacts. If you include this LinkedIn Gradle Plugin [1] you can automatically run tests in a virtual env and source a file to enter the project's virtual env. PyGradle [2]: "The PyGradle build system is a set of Gradle plugins that can be used to build Python artifacts" [1] https://github.com/linkedin/pygradle/blob/01d079e2b53bf9933aa786af1ec7dabcf964c669/docs/python.md https://github.com/linkedin/pygradle/blob/01d079e2b53bf9933a... [2] https://github.com/linkedin/pygradle https://github.com/linkedin/pygradle
kalefranz 10y ago
Hey everyone. I'm Kale, currently the lead developer on the conda project. It's been mentioned a few times in this thread, and I just want to make sure that any questions about it are answered accurately. Feel free to ask me anything about product conda. Thanks!
conradev 10y ago
I'm surprised that no one has mentioned pip-tools: https://github.com/nvie/pip-tools https://github.com/nvie/pip-tools It's a very similar set of tools. I use pip-compile which allows me to put all of my dependencies into a `requirements.in` file, and then "compile" them to a `requirements.txt` file as a lockfile (so that it is compatible with pip as currently exists). This looks great, though, I'm excited to check it out!
- FabioFleitas 10y ago
  We use pip-tools on all our Python projects and it works great. I believe the requirements.in compiled to requirements.txt approach is much more sane and less error-prone.
dvl 10y ago
Why people are trying to complicate things? requirements.txt are much simpler and better.
- sametmax 10y ago
  First, you need to take care of the virtualenv manually. This tool avoid that. You can completly ignore the virtualenv. You don't even have to populate requirements or learn about pip freeze. Secondly, this tool allow you to freeze your requirement list at specific versions. So in your req file, you have the name of the packages you depend on. Bon on the req lock, you get all the pulled dependencies recursively with the version you are using right now. The first one let you dev more easily. The second one deploy more easily. All that, for less complexity. Win win.
lukasm 10y ago
Did anyone had a problem installing in on El Captain? I manage to do it with --user flag but the pipenv command doesn't exist.
natmaster 10y ago
This is cool - it's like yarn for python! :)

664 points by imkevinxu 10y ago

> I wrote a new tool this weekend, called pipenv. > It harnesses Pipfile, pip, and virtualenv into one single toolchain. It features very pretty terminal colors. For a weekend project, this has some very nice things. Which removes the need for me to run my own project that basically does these things... In more or less, a worse way. Everything I've come to expect from Reitz, and hopefully it'll gain some decent ground like other projects of the same author.

command_tab 10y ago

See also: https://github.com/pypa/pipfile https://github.com/pypa/pipfile I'm glad to see Python getting the same attention as other modern package managers. This is all great work!

mikhuang 10y ago

Uninstall by default removing everything seems a little scary. Otherwise looks really neat, looking forward to trying it

bdukic 10y ago
I wanted to comment on the same thing, IMHO something like `pip uninstall --all` would be a better choice. Great work otherwise!

choxi 10y ago

Is this like Ruby's Bundler for Python? I've just been getting into Python and am really glad to see this, thanks for creating it!

uranusjr 10y ago
Without some shim like bundler exec, but yeah, you can say that.
igravious 10y ago
Very similar. I think Pipenv improves on Bundler by leveraging Virtualenv and Ruby doesn't have a per project equivalent to Virtualenv that I'm aware of. You can set the path config variable of Bundler to not place the project Gems in a central location which I think is cleaner and try to remember to always do now. It would be _super_ interesting if the Python and Ruby communities got together to harmonize every last detail of their packaging toolchain. Who is in?
- sametmax 10y ago
  Actually we could also learn from: - the JS community. npm dependancy graph, webpack resolver and yarn performances; - the rust community like with cargo.
  - K0nserv 10y ago
    Bundler does proper dependency resolution using Molinillo[0] which is also used by CocoaPods[1]. This is definitely something that other package managers can stand to adopt. 0: https://github.com/CocoaPods/Molinillo https://github.com/CocoaPods/Molinillo 1: http://cocopods.org/ http://cocopods.org/
    - sametmax 10y ago
      I have a big doubt about Molinillo. The readme doesn't even link to a documentation. Does it allow you to update one package or the executable version and get a list of all the stuff to update ?
    - igravious 10y ago
      This is exactly what I was trying to get at with my original comment. There needs to be a lot more sharing of package management tools and techniques across language boundaries. Generic platform- and language-agnostic tools and algorithms are a step in the right direction!
- mhw 10y ago
  > Ruby doesn't have a per project equivalent to Virtualenv that I'm aware of. The nearest equivalent is to place a file called '.ruby-version' in the top level directory, containing the version number of the Ruby you want to use. Version numbers come from https://github.com/rbenv/ruby-build/tree/master/share/ruby-build https://github.com/rbenv/ruby-build/tree/master/share/ruby-b.... rbenv, chruby and rvm all support .ruby-version. One difference from virtualenv is that the Ruby version managers share single installations of each version of Ruby. My understanding from occasional use of Virtualenv is that it copies the python installation into a new per-project subdirectory, which seems a bit wasteful to me. > You can set the path config variable of Bundler to not place the project Gems in a central location which I think is cleaner and try to remember to always do now. Yes, this is what I do. It gives me a shared 'clean' Ruby installation of the right version, plus a project-specific copy of all the gems the project depends on. To me this provides the best trade off between project isolation and not duplicating the whole world. You can set bundler up so this is done automatically by creating '~/.bundle/config' containing --- BUNDLE_PATH: "vendor/bundle" BUNDLE_BIN: ".bundle/bin" (The BUNDLE_PATH one is the important one; see 'bundle config --help' for other options.)

zyxzkz 10y ago

What took Python so long to get a tool like this?

softinio 10y ago
Kenneth was busy doing other stuff :-)
sametmax 10y ago
Did you work on it ? No. That's why. We can't always wait for one famous guy to do it.

noway421 10y ago

This is very interesting! I had exact same question how to do it in python just a while ago! http://stackoverflow.com/questions/41427500/creating-a-virtualenv-with-preinstalled-packages-as-in-requirements-txt http://stackoverflow.com/questions/41427500/creating-a-virtu... Glad that someone thought about similar thing and made a tool to solve it!

gourneau 10y ago

Hey other Reitz fans. Make sure to check out his newish podcast series: https://www.kennethreitz.org/import-this/ https://www.kennethreitz.org/import-this/

renesd 10y ago

Neat. Now for questions and comments. Often people have a requirements.live.txt, or other packages depending on the environment. Is that handled somehow? Can we use different files or sections? [ED: yes, different sections] Still wondering to myself if this is worth the fragmentation for most people using requirements.txt ? Perhaps the different sections could have a "-r requirements.txt" in there, like how requirements.dev.txt can have "-r requirements.txt". [ED: the pipfile idea seems to have quite some people behind it, and pip will support it eventually. Seems it will be worth it to standardise these things. requirements.txt is a less jargony name compared to Pipfile though, and has a windows/gui friendly extension.] Other tools can set up an environment, download stuff, and run the script. Will pipenv --shell somescript.py do what I want? (run the script with the requirements it needs). ((I guess I could just try it.)) [ED: doesn't seem so] Why Pipfile with Caps? Seems sort of odd for a modern python Thing. It looks like a .ini file? [ED: standard still in development it seems. TOML syntax.] With a setup.py set up, all you need to do is `pip install -e .` to download all the required packages. Or `pip install somepackage`. Lots of people make the setup.py file read the requirements.txt. Do you have some command for handling this integration? Or is this needed to be done manually? [ED: seems no considering about this/out of scope.] Is there a pep? [ED: too early it seems.]

kenneth_reitz 10y ago
Check out https://github.com/pypa/pipfile https://github.com/pypa/pipfile for more info. The format is TOML. It is mainly intended for deployments (e.g. web applications)
- renesd 10y ago
  Thanks.
  - kenneth_reitz 10y ago
    Also `pipenv install -r requirements.txt` is supported, for importing.
    - 10y ago

btashton 10y ago

Interesting. I have been leveraging tox to provide a lot of what this seems to give you, but it certainly has been more of a hack than a solution.

auston 10y ago

I can only express my gratitude, thank you Kenneth!

therealmarv 10y ago

Currently using fish and virtualfish. This seems incompatible to non bash shells. Did somebody tested?

kenneth_reitz 10y ago
Fish support was just added!
- deleted 10y ago
  [deleted]

jaybuff 10y ago

See also, pex: https://www.youtube.com/watch?v=NmpnGhRwsu0 https://www.youtube.com/watch?v=NmpnGhRwsu0

joelbondurant 10y ago

Has anyone in the Python community heard of Docker?

therealmarv 10y ago

For people who want to do it right without using an additional tool read this: setup.py vs. requirements.txt by Donald Stufft https://caremad.io/posts/2013/07/setup-vs-requirement/ https://caremad.io/posts/2013/07/setup-vs-requirement/

yeukhon 10y ago
I gave up populating requirements in setup.py. I just use multiple requirements.txt. This article has been debated for years already and there is absolutely no right / wrong.

korijn 10y ago

Windows support?

kenneth_reitz 10y ago
coming soon, hopefully via pull request!

cderwin 10y ago

This is great, but sometimes I think that python needs a new package manager from scratch instead of more tools trying to mix and mash a bunch of flawed tools together in a way that's palatable by most of us. Python packaging sucks, the whole lot of it. Maybe I'm just spoiled by rust and elixir, but setuptools, distutils, pip, ez_install, all of it is really subpar. But of course everything uses pypi and pip now, so it's not like any of it can actually be replaced. The state of package management in python makes me sad. I wish there was a good solution, but I just don't see it. Edit: I don't mean to disparage projects like this and pipfile. Both are great efforts to bring the packaging interface in line with what's available in other languages, and might be the only way up and out of the current state of affairs.

sametmax 10y ago
I strongly agree but this can work now and is a big improvement over what we currently have. So while I would literally pay to see somebody work on a better package manager (which can generate exe, deb and use a conf file indead of .py), this is a good filler.
- X-Istence 10y ago
  Why should it generate a deb file? How is this useful on anything but Debian based systems? Why exe? How do you package libraries using this new tool you are envisioning?
  - sametmax 10y ago
    You do package lib the same way as before, although cargo like dependency handling would be a nice thing. Especially for upgrades. But a good package manager should ALSO allow you to produce a: - a stand alone executable for most OS. - a standard package for major OS (msi, deb, snap, rpm, dmg, etc). Doing that right now with Python requires you to setup stuff like nuikta and the like. It works but it's much harder than it should be.
    - snuxoll 10y ago

caconym_ 10y ago

I will definitely be trying this out. Python version and package management is a dumpster fire that wastes gobs of my time on the regular. I'll try anything that promises to end the pain.

sametmax 10y ago

I was really not a fan of the last "made in Reitz" project Maya. But this, I really can get along. The whole things make it way easier to get started for a beginner. Now more activate. No more wondering about virtualenv. Automatic lock files are great since no project I know of use them since they are not well understood. It's like node_packages (easy and obvious), but cleaner (no implicit magic). Like.

zephyrfalcon 10y ago

A bit off-topic, but what font is used in the video/animated GIF?

kenneth_reitz 10y ago
Operator Mono! https://www.kennethreitz.org/essays/test-driving-a-200-programming-font-operator-mono https://www.kennethreitz.org/essays/test-driving-a-200-progr...

zoul 10y ago

I always wonder if this could be done once and for all languages, instead of Ruby making bundler, Haskell Cabal sandboxes or stack, Perl Brew, etc. Is this where Nix is going?

HeyImAlex 10y ago
I don't know, different languages and environments need different things. I'm not sure that "package management" is as generic as it seems at first glance.
toyg 10y ago
You make it one tool, and sysadmins will instantaneously lock it down. These package managers, in most cases, are developer tools built to get around system-wide locks on libraries; the more you centralize them, the more likely it is they will get locked down, and then someone will build tools to get around that, and so on and so forth.
- olejorgenb 10y ago
  As one of the few package mangers, nix can be used without root access.
olejorgenb 10y ago
In some way, absolutely. You can easily get a nice isolated python environment with some packages in nix without using pip, pyenv, etc. `nix-shell -p python pythonPackages.numpy ...` So far I think it works quite well for most languages as long the needed packages are in nixpkgs. Some of the tooling could be better, but the underlying model seems sound. I'm not really convinced language-specific package managers are needed. Nix isn't perfect yet, but it has come a long way.

nejdetckenobi 10y ago

normally I use virtualenvwrapper and that makes a virtualenv directory for all virtualenvs you create with it. before that, I always create my projects' venvs inside my project hierarchy. I had a dilemma about it. But after all, you can not move your venv directory unless you use `--relocatable` option. So, anyone have a strong argument about creating venvs inside your project directory?

toyg 10y ago
I just keep venvs in the project folder and add them to .gitignore. --relocatable never worked well for me, I just keep my requirements.txt up to date so it's trivial to blow away a venv and recreate it if necessary (python3 -m venv env && source env/bin/activate && pip install -r requirements.txt). I find tools like virtualenvwrapper and this one from Kenneth tend to solve issues I don't really have. A little bit of repetitive typing here and there is ok to burn knowledge into my mind; and less leaky abstractions I have to deal with, the better.
- sametmax 10y ago
  It solves more than that: - beginers don't have to understand the whole virtualenv shenanigans. I use pew myself to replace virtualenvwrapper but I will switch to pipenv just to ease the pain of team member joining in. - it enforces good dependency management practice with the toml file and lock file. This is an issue in almost all project I worked on, including ones from Python experts. We all use only requirements.txt file out of convenience, and never lock. - it's one tool to do all the packaging stuff. No need for pip and virtualenv and a wrapper. You got one command.
icebraining 10y ago
I've always found the whole virtualenv stuff so superfluous. Do we really need all the machinery with shell scripts and static paths? We just use a directory where we keep our dependencies. It's a matter of: mkdir libs pip install -t libs <package> # then to run PYTHONPATH=libs python app.py From what I can tell, this accomplishes everything a venv does (except bringing the Python interpreter itself along) without requiring any extra tools or conventions to learn.

mulla111 10y ago

https://goo.gl/smCgbj https://goo.gl/smCgbj

deleted 10y ago

[deleted]

istoica 10y ago

Finally someones does it! I was using: pip -t .pip in my code, avoiding virtual-env completely, but that was not enough and incomplete. As this is not cross platform and it would be nice to switch between Linux/Windows while coding to maintain platform compatibility, can the virtualenv envs be created with a os platform & subsystem prefix ? for example, having multiple envs at once: - env/posix/bin/activate - env/nt/Scripts/activate.bat

wyldfire 10y ago

I haven't really used requirements.txt because I found that I could install 'extra' and 'test' specific content based on args to setup() in my setup.py. It seems more like the Right Thing than requirements.txt, from what I can tell. At first glance, this doesn't seem to offer anything beyond what I already see from setup(). What am I missing? It's unfortunate that CPython gave us distutils and took a very long time to converge on a built-in successor (setuptools?) that gives the right composability.

helb 10y ago

> --three / --two Use Python 3/2 when creating virtualenv. I use Python 2.7, 3.4, and 3.5 on various projects. Is there a way to choose between 3.4 and 3.5 using Pipenv? I'm using something like this with virtualenv: $ virtualenv -p `which python3.5` .venv

iheredia 10y ago
Just a comment. Your command is equivalent to $ virtualenv -p python3.5 .venv
- emeraldd 10y ago
  Is that true? His command would evaluate the path at the time which is run, but yours seems to evaluate the path at the time python3.5 is run. Subtle, but very different results.
  - kyrias 10y ago
    virtualenv resolves the interpreter when you run virtualenv, and if you specify a different interpreter with `-p` it will resolve the path to the interpreter and then run virtualenv again using it.
jjawssd 10y ago
No, but you could be using pyenv instead of the system package manager because it is far cleaner

throw2016 10y ago

I think an app should not expose end users to its dependencies. That leaves the end user with a lot of pain figuring out versions of dependencies and god forbid you need to compile some dep then you need a build environment and its dependencies any of which can fail in this chain leaving a very unpleasant and even hostile end user experience. Ruby and Node apps are particularly guilty of this pulling in sometimes hundreds of packages some of which need compilation. Compare that to a Go binary which is download and use. These things can get very complicated very fast even for developers or system folks let alone end users who may not be intimately familiar with that specific ecosystem.

JeremyBanks 10y ago

> Otherwise, whatever $ which python will be the default. This is a bit strange because the python binary is always supposed to be Python 2. The Python 3 binary is supposed to be named python3. Some distributons don't follow this, but they're the weird non-conformant ones; it's not a behaviour that should really be relied on.

jjawssd 10y ago
Isn't `python` in Ubuntu a Python 3?
- anuragsoni 10y ago
  Not yet. At least on ubuntu 16.10 python still defaults to `python2`
jacobmischka 10y ago
> always supposed to be Python 2 This is not correct. It's a symlink to python2 on systems that rely on calls to python to be python2. On modern systems, python is usually a symlink to python3. This is the case on Arch Linux and I believe other recent distro releases.
- JeremyBanks 10y ago
  Arch is the oddball. I'm not aware of anybody else who's made the switch. Given that they're not mutually compatible except in rare cases, it's a very silly thing to do. You can upgrade GCC with the same name because you know it will handle most of the same input. If you do that with Python you're breaking tons of existing scripts for very close to zero benefit. Why would you do that?
  - baq 10y ago
    I'm pretty sure Ubuntu 16.04 is Python 3 by default and you have to ask for 2. edit: nope, been imagining things. too much time in venvs i guess. need to get more sleep :)

olejorgenb 10y ago

Seems to be a python specific nix-shell like tool? With nix[OS] you just run `nix-shell -p python[2,3] python[2,3]Pacakges.numpy ...` to get an environment with the required packages. Of course this requires that the python library is packaged in nix, but in my experience the coverage is quite good, and it's not very hard to write packages once you get the hang of it. It also possible (but currently a bit clumsy in some ways) to set up named and persistent environments.

georgeaf99 10y ago

LinkedIn has a similar open source project that is much more mature. It builds on Gradle features to manage complex dependencies and build Python artifacts. If you include this LinkedIn Gradle Plugin [1] you can automatically run tests in a virtual env and source a file to enter the project's virtual env. PyGradle [2]: "The PyGradle build system is a set of Gradle plugins that can be used to build Python artifacts" [1] https://github.com/linkedin/pygradle/blob/01d079e2b53bf9933aa786af1ec7dabcf964c669/docs/python.md https://github.com/linkedin/pygradle/blob/01d079e2b53bf9933a... [2] https://github.com/linkedin/pygradle https://github.com/linkedin/pygradle

kalefranz 10y ago

Hey everyone. I'm Kale, currently the lead developer on the conda project. It's been mentioned a few times in this thread, and I just want to make sure that any questions about it are answered accurately. Feel free to ask me anything about product conda. Thanks!

conradev 10y ago

I'm surprised that no one has mentioned pip-tools: https://github.com/nvie/pip-tools https://github.com/nvie/pip-tools It's a very similar set of tools. I use pip-compile which allows me to put all of my dependencies into a `requirements.in` file, and then "compile" them to a `requirements.txt` file as a lockfile (so that it is compatible with pip as currently exists). This looks great, though, I'm excited to check it out!

FabioFleitas 10y ago
We use pip-tools on all our Python projects and it works great. I believe the requirements.in compiled to requirements.txt approach is much more sane and less error-prone.

dvl 10y ago

Why people are trying to complicate things? requirements.txt are much simpler and better.

sametmax 10y ago
First, you need to take care of the virtualenv manually. This tool avoid that. You can completly ignore the virtualenv. You don't even have to populate requirements or learn about pip freeze. Secondly, this tool allow you to freeze your requirement list at specific versions. So in your req file, you have the name of the packages you depend on. Bon on the req lock, you get all the pulled dependencies recursively with the version you are using right now. The first one let you dev more easily. The second one deploy more easily. All that, for less complexity. Win win.

lukasm 10y ago

Did anyone had a problem installing in on El Captain? I manage to do it with --user flag but the pipenv command doesn't exist.

natmaster 10y ago

This is cool - it's like yarn for python! :)