16 ms·

Man, personally, with Python, programming with the language seems to be the easy part. I have struggled with virtual environments and runtime executables acros

Man, personally, with Python, programming with the language seems to be the easy part. I have struggled with virtual environments and runtime executables across various OSs. What are current best practices?

atoav 6y ago
I settled in using poetry for developement and dependency management and the publishing stuff to pypi and installing it via pip. But this approach might be a bit limiting depending on your target audience.
- bluemania 6y ago
  Agree completely. Poetry has been a watershed moment for my development. For dependency management it's so much better than conda and VE pip.
- makeworld 6y ago
  This. Poetry definitely seems like the best way to go. It's been great when I've used it.
agustif 6y ago
Last time I had problems in Windows I used pyenv/pipenv as per some guide.
dec0dedab0de 6y ago
I use pipenv,and it works well enough for what I'm doing, but in reality the problem is that python just isnt designed to be distributed. There are various techniques each with their own pros and cons, but the only consensus is that it's a pain in the ass.
- mettamage 6y ago
  What are the tradeoffs vs using docker? Just curious.
  - fuhrysteve 6y ago
    > What are the tradeoffs vs using docker? Just curious. Probably some combination of memory usage and complexity, depending on your application. If you're already familiar with using docker as a development environment, definitely go for it. I don't use pipenv, I'm still using plain old virtualenv for development. Mostly it's just a matter of familiarity. If there's not an itch, why scratch?
  - dec0dedab0de 6y ago
    Docker uses more resources, and is trying to isolate your code from the rest of thr computer. It also has a higher learning curve. Virtual environments are just separate copies of python with their own libraries installed. Pipenv is basically just a workflow for organizing virtual environments I can give someone a repo and tell them to type pip install pipenv && pipenv sync and they'll have everything. Assuming they already have the correct version of python installed, which is one nice thing docker handles, but it is easy to install python these days so it hasnt been an issue. My biggest problem with docker was that I ended up using a ton of storage just learning about it. I have a feeling that was mostly my own fault, maybe using too heavy of a base image. I've been trying it out once or twice a year for about 6 years now, every time my conclusion is "wow this is really cool, I wish i could justify spending more time to get it right" Though docker and virtual environments share the same problem, in that they are just a way for a developer to distribute code to other developers, and to production environments. Distributing python applications to end users is a totally different issue. I floated the idea of sending out a local data collection* app to Mac and windows users mostly because I think it would be fun to try. *data collection of troubleshooting information from users within the same company on company hardware, that are actively asking for help. I'm not trying to spy on people.
dcewcrrec 6y ago
1) python3 -m venv venvname 2) source venvname/bin/activate then you do everything in the virtual environment...
- kortilla 6y ago
  And then what’s the packaging/distribution story to run it everywhere else?
  - skrtskrt 6y ago
    Poetry
  - pydry 6y ago
    Do beginners really need to delve into the finer details of all that?
    - kortilla 6y ago
      I mean, yeah. Using the software you wrote on other machines is a big part of the software development experience.
funkydata 6y ago
I've decided to play it simple albeit "old school" (because Docker seemed to add its own set of headaches - also not every project needs to do micro services or work with clusters or scale): - Vagrant (so you don't have to worry about runtime executables) with the stack matching that of the production server (you can even ask ops to provide you with the provisioning script and remove the parts that you don't need - otherwise learning to provision your dev. VMs won't hurt you) - Virtualenv - pip Then simply point your IDE (I only use Vim under duress) to the remote Python interpreter (the one you installed in the Vagrant VM). It does add processing overhead but it worked with my 2010 MacBook Pro until it died and still works (only ten times faster) with the 2016 model. Your only limitation would be the RAM (I would recommend at least 8GB and if you plan to run multiple machines communicating together as much as you can afford - I do believe that Docker has less overhead, but again, for my use, not needed). The best practice is what works for you, not the latest trend.
- lastgeniusua 6y ago
  pipenv is a sort of a spliced-together pip and virtualenv, give it a try!
alexmojaki 6y ago
Haha yeah I find it easy too, which is why we're here, but I'm trying to also make it easy for people who normally think it isn't. And yes, setting up Python is a pain, so I think being able to start learning and running code without any setup is a big deal for learners. Get them excited about coding before they have to deal with that!
ericol 6y ago
One solution is to use Ana / Miniconda. If you are not that tech savvy, if you don't mind downloading several hundred megabytes you can get away with this easily. In my case, and contrary to what one of the other commentators said, I'm using docker. It took a while to get it ironed out, but I use a "base" folder where I have already downloaded all the packages I use by default (Top of my mind are pandas, numpy and youtube-dl ;) ) I have all the related configuration in an env file that tells pip to save the packages in the base folder so they don't disappear when I shut down the container - that I always run with --rm so they get removed when dead - and for creating a new env I just have to copy that folder. As I use a base folder for all of this I don't need to remember the names of the envs, as just need to list folders. Only downside to this - because I'm lazy and it still hasn't bothered me enough to fix it - is that new files & folders are owned by root. I use just a base image with python, and for different versions that work with this - supposedly - I just have to download a different image.
- theelous3 6y ago
  As someone who's been hanging out in freenode python channels for years helping noobs, I recommend strongly against any kind of conda setup for new programmers. It often has strange issues that noobs have difficulty even understanding how to ask for help with, nevermind actually solve. Learning how to use a simple barebones venv is extremely easy, saves a ton of time both in the short and long run, and generalises better. pip install virtualenv cd your/project/location which python virtualenv -p result_of_which_python env source env/bin/activate pip install anything_you_like and do whatever you want from there. Those commands get 100% of the basics out of the way for noobs, and cover like 90% of the stuff you use to do more serious stuff. > but I use a "base" folder where I have already downloaded all the packages I use by default [...etc...] Your setup sounds outrageously complex, and I don't understand why you would do any of that.
  - aldanor 6y ago
    I would strongly recommend to use conda because: - It would work the same way on Windows, Linux and macOS - Most importantly (!), it is not a Python package manager. That is, if a package needs BLAS or MKL or HDF5 or whatever else, you won't be crossing fingers and hoping your system-installed version would work for all your venvs; instead, those binary libraries are properly managed per environment. Pro tip: use mamba instead of conda to get a free 4x speed boost.
sixhobbits 6y ago
I have had this problem every time I try to teach anyone to use Python. I have years of experience with messing around with all of the relevant parts of Windows, Mac, and Linux and it never Just Works. It's super frustrating when others are like 'it's easy just use x'. It's not. If you're a beginner, the setting up of a dev environment is far more of an obstacle to learning go code than the syntax or programming concepts. Repl.it is one way to sidestep these issues but it's not perfect. Making people jump in the deep end and use Linux helps to some extent, but also brings its own set of issues and frustrations. I don't think "add another layer of abstraction to hide the complexity" is often a good solution. Docker brings it's own problems too.
- ericol 6y ago
  > I don't think "add another layer of abstraction to hide the complexity" is often a good solution I wholeheartedly agree here. > Docker brings it's own problems too This is a rather complex statement to reply to. Even thought you might be right, I don't think this applies totally to what is being discussed here. The biggest issue I have with people advising for or against a certain tool, is that they do that from the point of view of the tool, instead of looking at fit from the problem you are trying to solve. that in your case, would be: > I have years of experience with messing around with all of the relevant parts of Windows, Mac, and Linux and it never Just Works As long as you manage to install Docker in all those 3 systems (I have no experience with Mac because I don't use it) both for Windows and Linux installing Docker is a no brainer. There's a slight curve when it comes to fetching the right image and running it, but your problem is not that one; your problem is teaching Python. So you can take care of that yourself, and focus on the teaching part. Supposing that you managed to install Docker, fetch a python image and running it (Something that is a lot more easy to do than it sounds), you have python, whatever version you want, and in an isolated way. For me... it just works.
user5994461 6y ago
1) Create a virtual environment. python3 -m venv myproject (download an interpreter). 2) Run pip install. myproject/bin/activate; pip install requirements.txt; (download all project dependencies). 3) Start the application. myproject/bin/activate; python myapp.py If you can assume that there is an interpreter available on the system, say /usr/bin/python3, you can use that instead of creating a virtual environment. If you want to embed all the dependencies, you can save all the files created by pip install in the /lib directory if I remember the name well. Should I write a full blog post with example? I used to deploy python applications in a bank, can explain all the advanced usage with and without internet access, with and without dependencies.
- aerovistae 6y ago
  lord yes
  - nemosaltat 6y ago
    Seconded
- williamtwild 6y ago
  yes pleasr
- simonebrunozzi 6y ago
  Yes, please. Aaaand, now that you're at it, a single .dmg file for MacOS with binaries and scripts so us lazy people with Macbooks can start coding right away :) (it might be a bit inaccurate but I guess you know what I mean)
- somurzakov 6y ago
  yes please
- cvhashim 6y ago
  That would be awesome
- noitpmeder 6y ago
BerislavLopac 6y ago
In a nutshell, use pyenv [0] to manage your Python versions. It's a bash-shell solution, so it won't work on Windows; but if we're talking about learning Python, I highly recommend using WSL [1] anyway. [0] https://github.com/pyenv/pyenv https://github.com/pyenv/pyenv [1] https://docs.microsoft.com/en-us/windows/wsl/ https://docs.microsoft.com/en-us/windows/wsl/
ttymck 6y ago
It's definitely confusing, because there are so many options. I wrote about dependency management best practices here[1]. Planning to write a follow-up on "cross-platform executables" soon [1] https://havercene.io/blog/no-nonsense-python-dependency-management-in-2020/ https://havercene.io/blog/no-nonsense-python-dependency-mana...
j88439h84 6y ago
Poetry is by far the best way to start a Python project. Way easier than using pip, pipenv, virtualenv, venv, pyenv, conda, miniconda. https://python-poetry.org/ https://python-poetry.org/
ggregoire 6y ago
Recommending Docker in these discussions usually gets me downvotes (?), but I'll do it anyway because it's such an easy way to develop and run Python programs across environments: 1/ add a Dockerfile into your project with the following lines: FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . . CMD [ "python", "./your-daemon-or-script.py" ] 2/ install Docker on your various environments 3/ git clone, docker build, docker run and that's it - Ref: https://hub.docker.com/_/python https://hub.docker.com/_/python
- yonixw 6y ago
  How do you handle debugging? like from vs code?
  - ggregoire 6y ago
    I personally debug with logs, but what you are asking seems possible: - Debug containerized apps: https://code.visualstudio.com/docs/containers/debug-common https://code.visualstudio.com/docs/containers/debug-common - Debug Python within a container: https://code.visualstudio.com/docs/containers/debug-python https://code.visualstudio.com/docs/containers/debug-python You can apparently develop directly inside a container too: - Developing inside a Container: https://code.visualstudio.com/docs/remote/containers https://code.visualstudio.com/docs/remote/containers
  - lovehashbrowns 6y ago
    You can run code-server to run VS Code inside the docker container where your files will be. This is the Dockerfile I use, which I got from someone else's link that was posted on HN: # the base miniconda3 image FROM continuumio/miniconda3:latest # load in the environment.yml file - this file controls what Python packages we install ADD environment.yml / # install the Python packages we specified into the base environment RUN conda update -n base conda -y && conda env update && conda install -y -q moto && conda install -y -q -c conda-forge awscli httmock # download the coder binary, untar it, and allow it to be executed RUN wget https://github.com/cdr/code-server/releases/download/2.1698/code-server2.1698-vsc1.41.1-linux-x86_64.tar.gz https://github.com/cdr/code-server/releases/download/2.1698/... \ && tar -xzvf code-server2.1698-vsc1.41.1-linux-x86_64.tar.gz && chmod +x code-server2.1698-vsc1.41.1-linux-x86_64/code-server COPY docker-entrypoint.sh /usr/local/bin/ ADD ./code /code ENTRYPOINT ["docker-entrypoint.sh"] Building that and running it with: docker run -d -p 127.0.0.1:8443:8080 -p 127.0.0.1:8888:8888 -v $(pwd)/data:/data -v $(pwd)/code:/code --rm -it <image> from the directory where your code is will put those files into the container, and start a VS Code and a Jupyter Notebook server on your localhost. The password for Jypter is the default "local-development" and the password for the VS Code instance is in the Docker logs. You can set these via the Dockerfile but I just keep the defaults. I vastly prefer this to anything else because it means I can install any packages I want without worrying about messing up my environment. You can use virtual envs to make this even better, but I am typically too dumb and lazy for that. Better part still is that my development is the same on my Mac, on my Linux machine, and on my Windows machine. Same VS Code version, same packages, etc. Biggest issue here is with certain VS Code plugins. Some, like the vim plugin, can be finicky and depend heavily on the version of code server that you use. Some plugins break completely. However, I mainly hate plugins so this doesn't present much of an issue for me personally. I have the vim plugin, the python plugin, and a terraform plugin installed. Once they are installed, they work perfectly for me. The way my set up works is I have a repo with that Dockerfile in it as well as the accompanying files such as environment.yml and docker-entrypoint.sh: #!/bin/bash set -e if [ $# -eq 0 ] then jupyter lab --ip=0.0.0.0 --NotebookApp.token='local-development' --allow-root --no-browser &> /dev/null & code-server2.1698-vsc1.41.1-linux-x86_64/code-server --allow-http --no-auth --data-dir /data /code else exec "$@" fi and a .gitignore file with this in it: code/* data/* Oh also I found the repo where I took these things from: https://github.com/caesarnine/data-science-docker-vscode-template https://github.com/caesarnine/data-science-docker-vscode-tem...
Biganon 6y ago
I've been very happy in the last few months with poetry.
nickjj 6y ago
Docker has been mentioned a couple of times but no one has referenced one of the biggest wins of Docker which is it lets you quickly get up and running for things that your Python app might be using in addition to Python itself. If we're talking about web development with Python typically that means PostgreSQL and Redis too, and probably running Celery in addition to a web server such as gunicorn. It's really nice to be able to just run a single Docker Compose command and be up and running in a way that works the same on Windows, MacOS and Linux. This post outlines the differences between creating a Python development environment with and without Docker https://nickjanetakis.com/blog/setting-up-a-python-development-environment-with-and-without-docker https://nickjanetakis.com/blog/setting-up-a-python-developme.... It focuses on the use case of web development.

atoav 6y ago

I settled in using poetry for developement and dependency management and the publishing stuff to pypi and installing it via pip. But this approach might be a bit limiting depending on your target audience.

bluemania 6y ago
Agree completely. Poetry has been a watershed moment for my development. For dependency management it's so much better than conda and VE pip.
makeworld 6y ago
This. Poetry definitely seems like the best way to go. It's been great when I've used it.

agustif 6y ago

Last time I had problems in Windows I used pyenv/pipenv as per some guide.

dec0dedab0de 6y ago

I use pipenv,and it works well enough for what I'm doing, but in reality the problem is that python just isnt designed to be distributed. There are various techniques each with their own pros and cons, but the only consensus is that it's a pain in the ass.

mettamage 6y ago
What are the tradeoffs vs using docker? Just curious.
- fuhrysteve 6y ago
  > What are the tradeoffs vs using docker? Just curious. Probably some combination of memory usage and complexity, depending on your application. If you're already familiar with using docker as a development environment, definitely go for it. I don't use pipenv, I'm still using plain old virtualenv for development. Mostly it's just a matter of familiarity. If there's not an itch, why scratch?
- dec0dedab0de 6y ago
  Docker uses more resources, and is trying to isolate your code from the rest of thr computer. It also has a higher learning curve. Virtual environments are just separate copies of python with their own libraries installed. Pipenv is basically just a workflow for organizing virtual environments I can give someone a repo and tell them to type pip install pipenv && pipenv sync and they'll have everything. Assuming they already have the correct version of python installed, which is one nice thing docker handles, but it is easy to install python these days so it hasnt been an issue. My biggest problem with docker was that I ended up using a ton of storage just learning about it. I have a feeling that was mostly my own fault, maybe using too heavy of a base image. I've been trying it out once or twice a year for about 6 years now, every time my conclusion is "wow this is really cool, I wish i could justify spending more time to get it right" Though docker and virtual environments share the same problem, in that they are just a way for a developer to distribute code to other developers, and to production environments. Distributing python applications to end users is a totally different issue. I floated the idea of sending out a local data collection* app to Mac and windows users mostly because I think it would be fun to try. *data collection of troubleshooting information from users within the same company on company hardware, that are actively asking for help. I'm not trying to spy on people.

dcewcrrec 6y ago

1) python3 -m venv venvname 2) source venvname/bin/activate then you do everything in the virtual environment...

kortilla 6y ago
And then what’s the packaging/distribution story to run it everywhere else?
- skrtskrt 6y ago
  Poetry
- pydry 6y ago
  Do beginners really need to delve into the finer details of all that?
  - kortilla 6y ago
    I mean, yeah. Using the software you wrote on other machines is a big part of the software development experience.

funkydata 6y ago

I've decided to play it simple albeit "old school" (because Docker seemed to add its own set of headaches - also not every project needs to do micro services or work with clusters or scale): - Vagrant (so you don't have to worry about runtime executables) with the stack matching that of the production server (you can even ask ops to provide you with the provisioning script and remove the parts that you don't need - otherwise learning to provision your dev. VMs won't hurt you) - Virtualenv - pip Then simply point your IDE (I only use Vim under duress) to the remote Python interpreter (the one you installed in the Vagrant VM). It does add processing overhead but it worked with my 2010 MacBook Pro until it died and still works (only ten times faster) with the 2016 model. Your only limitation would be the RAM (I would recommend at least 8GB and if you plan to run multiple machines communicating together as much as you can afford - I do believe that Docker has less overhead, but again, for my use, not needed). The best practice is what works for you, not the latest trend.

lastgeniusua 6y ago
pipenv is a sort of a spliced-together pip and virtualenv, give it a try!

alexmojaki 6y ago

Haha yeah I find it easy too, which is why we're here, but I'm trying to also make it easy for people who normally think it isn't. And yes, setting up Python is a pain, so I think being able to start learning and running code without any setup is a big deal for learners. Get them excited about coding before they have to deal with that!

ericol 6y ago

One solution is to use Ana / Miniconda. If you are not that tech savvy, if you don't mind downloading several hundred megabytes you can get away with this easily. In my case, and contrary to what one of the other commentators said, I'm using docker. It took a while to get it ironed out, but I use a "base" folder where I have already downloaded all the packages I use by default (Top of my mind are pandas, numpy and youtube-dl ;) ) I have all the related configuration in an env file that tells pip to save the packages in the base folder so they don't disappear when I shut down the container - that I always run with --rm so they get removed when dead - and for creating a new env I just have to copy that folder. As I use a base folder for all of this I don't need to remember the names of the envs, as just need to list folders. Only downside to this - because I'm lazy and it still hasn't bothered me enough to fix it - is that new files & folders are owned by root. I use just a base image with python, and for different versions that work with this - supposedly - I just have to download a different image.

theelous3 6y ago
As someone who's been hanging out in freenode python channels for years helping noobs, I recommend strongly against any kind of conda setup for new programmers. It often has strange issues that noobs have difficulty even understanding how to ask for help with, nevermind actually solve. Learning how to use a simple barebones venv is extremely easy, saves a ton of time both in the short and long run, and generalises better. pip install virtualenv cd your/project/location which python virtualenv -p result_of_which_python env source env/bin/activate pip install anything_you_like and do whatever you want from there. Those commands get 100% of the basics out of the way for noobs, and cover like 90% of the stuff you use to do more serious stuff. > but I use a "base" folder where I have already downloaded all the packages I use by default [...etc...] Your setup sounds outrageously complex, and I don't understand why you would do any of that.
- aldanor 6y ago
  I would strongly recommend to use conda because: - It would work the same way on Windows, Linux and macOS - Most importantly (!), it is not a Python package manager. That is, if a package needs BLAS or MKL or HDF5 or whatever else, you won't be crossing fingers and hoping your system-installed version would work for all your venvs; instead, those binary libraries are properly managed per environment. Pro tip: use mamba instead of conda to get a free 4x speed boost.

sixhobbits 6y ago

I have had this problem every time I try to teach anyone to use Python. I have years of experience with messing around with all of the relevant parts of Windows, Mac, and Linux and it never Just Works. It's super frustrating when others are like 'it's easy just use x'. It's not. If you're a beginner, the setting up of a dev environment is far more of an obstacle to learning go code than the syntax or programming concepts. Repl.it is one way to sidestep these issues but it's not perfect. Making people jump in the deep end and use Linux helps to some extent, but also brings its own set of issues and frustrations. I don't think "add another layer of abstraction to hide the complexity" is often a good solution. Docker brings it's own problems too.

ericol 6y ago
> I don't think "add another layer of abstraction to hide the complexity" is often a good solution I wholeheartedly agree here. > Docker brings it's own problems too This is a rather complex statement to reply to. Even thought you might be right, I don't think this applies totally to what is being discussed here. The biggest issue I have with people advising for or against a certain tool, is that they do that from the point of view of the tool, instead of looking at fit from the problem you are trying to solve. that in your case, would be: > I have years of experience with messing around with all of the relevant parts of Windows, Mac, and Linux and it never Just Works As long as you manage to install Docker in all those 3 systems (I have no experience with Mac because I don't use it) both for Windows and Linux installing Docker is a no brainer. There's a slight curve when it comes to fetching the right image and running it, but your problem is not that one; your problem is teaching Python. So you can take care of that yourself, and focus on the teaching part. Supposing that you managed to install Docker, fetch a python image and running it (Something that is a lot more easy to do than it sounds), you have python, whatever version you want, and in an isolated way. For me... it just works.

user5994461 6y ago

1) Create a virtual environment. python3 -m venv myproject (download an interpreter). 2) Run pip install. myproject/bin/activate; pip install requirements.txt; (download all project dependencies). 3) Start the application. myproject/bin/activate; python myapp.py If you can assume that there is an interpreter available on the system, say /usr/bin/python3, you can use that instead of creating a virtual environment. If you want to embed all the dependencies, you can save all the files created by pip install in the /lib directory if I remember the name well. Should I write a full blog post with example? I used to deploy python applications in a bank, can explain all the advanced usage with and without internet access, with and without dependencies.

aerovistae 6y ago
lord yes
- nemosaltat 6y ago
  Seconded
williamtwild 6y ago
yes pleasr
simonebrunozzi 6y ago
Yes, please. Aaaand, now that you're at it, a single .dmg file for MacOS with binaries and scripts so us lazy people with Macbooks can start coding right away :) (it might be a bit inaccurate but I guess you know what I mean)
somurzakov 6y ago
yes please
cvhashim 6y ago
That would be awesome
noitpmeder 6y ago

BerislavLopac 6y ago

In a nutshell, use pyenv [0] to manage your Python versions. It's a bash-shell solution, so it won't work on Windows; but if we're talking about learning Python, I highly recommend using WSL [1] anyway. [0] https://github.com/pyenv/pyenv https://github.com/pyenv/pyenv [1] https://docs.microsoft.com/en-us/windows/wsl/ https://docs.microsoft.com/en-us/windows/wsl/

ttymck 6y ago

It's definitely confusing, because there are so many options. I wrote about dependency management best practices here[1]. Planning to write a follow-up on "cross-platform executables" soon [1] https://havercene.io/blog/no-nonsense-python-dependency-management-in-2020/ https://havercene.io/blog/no-nonsense-python-dependency-mana...

j88439h84 6y ago

Poetry is by far the best way to start a Python project. Way easier than using pip, pipenv, virtualenv, venv, pyenv, conda, miniconda. https://python-poetry.org/ https://python-poetry.org/

ggregoire 6y ago

Recommending Docker in these discussions usually gets me downvotes (?), but I'll do it anyway because it's such an easy way to develop and run Python programs across environments: 1/ add a Dockerfile into your project with the following lines: FROM python:3 WORKDIR /usr/src/app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . . CMD [ "python", "./your-daemon-or-script.py" ] 2/ install Docker on your various environments 3/ git clone, docker build, docker run and that's it - Ref: https://hub.docker.com/_/python https://hub.docker.com/_/python

yonixw 6y ago
How do you handle debugging? like from vs code?
- ggregoire 6y ago
  I personally debug with logs, but what you are asking seems possible: - Debug containerized apps: https://code.visualstudio.com/docs/containers/debug-common https://code.visualstudio.com/docs/containers/debug-common - Debug Python within a container: https://code.visualstudio.com/docs/containers/debug-python https://code.visualstudio.com/docs/containers/debug-python You can apparently develop directly inside a container too: - Developing inside a Container: https://code.visualstudio.com/docs/remote/containers https://code.visualstudio.com/docs/remote/containers
- lovehashbrowns 6y ago
  You can run code-server to run VS Code inside the docker container where your files will be. This is the Dockerfile I use, which I got from someone else's link that was posted on HN: # the base miniconda3 image FROM continuumio/miniconda3:latest # load in the environment.yml file - this file controls what Python packages we install ADD environment.yml / # install the Python packages we specified into the base environment RUN conda update -n base conda -y && conda env update && conda install -y -q moto && conda install -y -q -c conda-forge awscli httmock # download the coder binary, untar it, and allow it to be executed RUN wget https://github.com/cdr/code-server/releases/download/2.1698/code-server2.1698-vsc1.41.1-linux-x86_64.tar.gz https://github.com/cdr/code-server/releases/download/2.1698/... \ && tar -xzvf code-server2.1698-vsc1.41.1-linux-x86_64.tar.gz && chmod +x code-server2.1698-vsc1.41.1-linux-x86_64/code-server COPY docker-entrypoint.sh /usr/local/bin/ ADD ./code /code ENTRYPOINT ["docker-entrypoint.sh"] Building that and running it with: docker run -d -p 127.0.0.1:8443:8080 -p 127.0.0.1:8888:8888 -v $(pwd)/data:/data -v $(pwd)/code:/code --rm -it <image> from the directory where your code is will put those files into the container, and start a VS Code and a Jupyter Notebook server on your localhost. The password for Jypter is the default "local-development" and the password for the VS Code instance is in the Docker logs. You can set these via the Dockerfile but I just keep the defaults. I vastly prefer this to anything else because it means I can install any packages I want without worrying about messing up my environment. You can use virtual envs to make this even better, but I am typically too dumb and lazy for that. Better part still is that my development is the same on my Mac, on my Linux machine, and on my Windows machine. Same VS Code version, same packages, etc. Biggest issue here is with certain VS Code plugins. Some, like the vim plugin, can be finicky and depend heavily on the version of code server that you use. Some plugins break completely. However, I mainly hate plugins so this doesn't present much of an issue for me personally. I have the vim plugin, the python plugin, and a terraform plugin installed. Once they are installed, they work perfectly for me. The way my set up works is I have a repo with that Dockerfile in it as well as the accompanying files such as environment.yml and docker-entrypoint.sh: #!/bin/bash set -e if [ $# -eq 0 ] then jupyter lab --ip=0.0.0.0 --NotebookApp.token='local-development' --allow-root --no-browser &> /dev/null & code-server2.1698-vsc1.41.1-linux-x86_64/code-server --allow-http --no-auth --data-dir /data /code else exec "$@" fi and a .gitignore file with this in it: code/* data/* Oh also I found the repo where I took these things from: https://github.com/caesarnine/data-science-docker-vscode-template https://github.com/caesarnine/data-science-docker-vscode-tem...

Biganon 6y ago

I've been very happy in the last few months with poetry.

nickjj 6y ago

Docker has been mentioned a couple of times but no one has referenced one of the biggest wins of Docker which is it lets you quickly get up and running for things that your Python app might be using in addition to Python itself. If we're talking about web development with Python typically that means PostgreSQL and Redis too, and probably running Celery in addition to a web server such as gunicorn. It's really nice to be able to just run a single Docker Compose command and be up and running in a way that works the same on Windows, MacOS and Linux. This post outlines the differences between creating a Python development environment with and without Docker https://nickjanetakis.com/blog/setting-up-a-python-development-environment-with-and-without-docker https://nickjanetakis.com/blog/setting-up-a-python-developme.... It focuses on the use case of web development.