4 ms·
We've transitioned our local/dev/prod instances to use conda on Heroku, and couldn't be happier. It was a tiny bit of work to get it set up, but now everything
by wclax04 11y ago
We've transitioned our local/dev/prod instances to use conda on Heroku, and couldn't be happier. It was a tiny bit of work to get it set up, but now everything is consistent, and we can set up new local environments in seconds.
- sandGorgon 11y agoSo I have been considering this. does conda track pypi or does it lag it? I have been concerned about moving over my requirements.text for a webapp with lots of dependencies
- wclax04 11y agoIt slightly lags, but you can include pip requirements in an environment.yml file, and they install normally. I really only use conda for the non-python bits of our stack: numpy/scipy/pandas etc - packages that are a pain to install on Heroku.
- gh02t 11y agoIt's also pretty straightforward to set up your own Conda package tree. Nice for packaging your app for deployment or making sure you have very precise dependencies. http://conda.pydata.org/docs/custom-channels.html http://conda.pydata.org/docs/custom-channels.html
- sandGorgon 11y agoI think deployment is a solved problem with docker. Its libraries like blas,etc that are a huge pain. I'm not sure why static linked bumpy is not possible - even anaconda could not achieve it.
- gh02t 11y agoIf you've ever tried to dive into the NumPy build process you'd see why. It's unbelievably complicated... not that they really could do it better given that they are compiling about a billion scientific libraries and support alternatives and optimizations (like MKL).
- sandGorgon 11y agoYes - unfortunately I have and I failed miserably. These days I'm trying to see if there's a docker build that can build a great numpy (with all optimizations). Interestingly there are even docker images to call cuda APIs from python.
- sandGorgon 11y agoThey are a pain to install on a desktop as well! Especially with all the blas linking. Was there anything special you had to do or was it simply conda install numpy-blas or something like that ?
- wclax04 11y agoFro Heroku we created/modified a custom buildpack (https://github.com/joshowen/conda-buildpack https://github.com/joshowen/conda-buildpack), and use that with the multibuildpack. We use conda-env (https://github.com/conda/conda-env https://github.com/conda/conda-env) which makes it really simple to manage environments locally and in remote environments.
- dwiel 11y agoWe have to use a mix of pypi and conda since quite a few of our dependencies are not in conda. We have a script which checks conda first, then falls back to pypi, all from one requirements.txt
- sandGorgon 11y agoWhy not conda all the way? Anything of concern? Incidentally is the buildscript of anaconda itself opensource? Couldn't find it anywhere.
- wclax04 11y agoNot sure if this is what you're looking for: https://github.com/conda/conda https://github.com/conda/conda
- vamega 11y agoAny chance you can share that script? I'm looking for something similar, since I too am using both conda and pypi for dependencies.
- dwiel 11y agohttps://gist.github.com/dwiel/7a2c0163b802e1cea0f6 https://gist.github.com/dwiel/7a2c0163b802e1cea0f6 Instead of requirements.txt we use py.prereqs, but otherwise this script should get you close.