4 ms·

I'm a big fan of using Docker because for real world web app development, your app is often more than just getting Python and a virtualenv set up. Earlier this

by nickjj 9y ago

I'm a big fan of using Docker because for real world web app development, your app is often more than just getting Python and a virtualenv set up. Earlier this week I wrote about the pains of setting up a Python development experience without Docker, and then compared it to Docker as well. If anyone is curious, that's located at https://nickjanetakis.com/blog/setting-up-a-python-development-environment-with-and-without-docker https://nickjanetakis.com/blog/setting-up-a-python-developme.... By the way, I would say Docker is anything but slow. I get near instant development feedback on my Flask applications, even when running things through Docker for Windows / WSL. These are pretty big Flask apps too, which have thousands of lines of code, dozens of packages, tons of assets and require running Celery, Postgres, Redis, etc..

Spiritus 9y ago
My problem with using Docker (only) is that it doesn't translate well to editors. Like, using jedi-vim[1] with a virtualenv constructed by a Docker container doesn't work at all. Unless I actually run vim itself inside said container. So unless your dependences build on macOS (like in my case), everything goes out the door. [1] https://github.com/davidhalter/jedi-vim https://github.com/davidhalter/jedi-vim
- AlexCoventry 9y ago
  Emacs has pretty nice integrations with inferior docker processes. I'm getting them via spacemacs, so I'm not sure what specific package provides it, but I assume it's elpy.
  - cat199 9y ago
    what kinds of interactions? (like: what operations, etc) using elpy here with a native install and can use e.g. tramp for remote editing/python sessions in most cases (even across machines), but determining docker mount points to edit in-machine not so much.. I suppose one can simply map a local code directory into a runtime environment, but this also makes e.g. interacting with an in-container interpreter a bit bothersome (not so bad actually, but have to set a separate interpreter path to something like 'docker exec -it ctid python' would like to track this down I suppose
    - AlexCoventry 9y ago
      C-c C-p will start an inferior python shell in the container, for instance.
- _drFaust 9y ago
  We've got about 30+ backends in python all wrapped in docker containers. Majority of the team was pure vim before I joined and they're slowly converting to pycharm after seeing how nicely you can setup a remote interpreter against a docker container. And it has vim bindings so you don't have to re-learn new hotkeys. I've also been following this VS Code issue on adding remote docker support for python https://github.com/Microsoft/vscode-python/issues/79#issuecomment-366000230 https://github.com/Microsoft/vscode-python/issues/79#issueco... However, if you're a hardcore vim guy then I doubt these IDEs are gonna satiate your current flow.
  - nickjj 9y ago
    Funny you mentioned that issue. I'm eagerly awaiting that feature too. I commented in the issue a few weeks ago: https://github.com/Microsoft/vscode-python/issues/79#issuecomment-362836925 https://github.com/Microsoft/vscode-python/issues/79#issueco.... Once that's implemented, oh man, development nirvana.
  - wirrbel 9y ago
    I remember, on my laptop with pycharm and docker (and the virtual machine docker lived in), the RAM usage was just excessive. I am by no means a minimalist, but data-science stuff was barely impossible with 16 GB RAM. Also, I strongly dislike PyCharm. I am a vim guy by heart, but I am generally not against IDEs. VS Code is okayish. For C++ development, I really loved Visual Studio. But PyCharm just feels wrong, bloated, slow and baroque
  - jsmeaton 9y ago
    What I do is create the virtualenv locally that docker would also create, and point my editor to the local virtualenv. It gives me all of the intellisense locally. I still need to debug within the container/shell for the moment but I'm hoping there's a pathway coming for vscode that pycharm already has.
  - sheraz 9y ago
    yup. happy pycharm and docker user here. even have it working with debugging and breakpoints.
- Walkman 9y ago
  Ever heard about bind mounting? My current docker image for a Flask app doesn't even contains my code.
  - maccam94 9y ago
    Are you mounting your virtualenv into the container somehow? How does your editor see the code/docs for your pip dependencies?
    - yetanother1980 9y ago
      I have not tried it but I imagine this would work fine. Project Project/venv Project/src And then have the whole project as a volume. Then your editor can see the same files as you have in the container.
- mikewhy 9y ago
  Lots of people mentioning you can maybe debug a remote process, but none suggesting what to do about completions. Maybe you can use bind mounts, but I'm not sure how well that works in practice, or with eg a package with a native component.
  - jsmeaton 9y ago
    See my comment a few above. Setup a virtualenv locally that your dev env points to for completions, but actually run the app within the container (with its own virtualenv). Docker is great for dependencies like databases and queues. I find it totally unnecessary for developing python.
axitanull 9y ago
>These are pretty big Flask apps too, which have thousands of lines of code, dozens of packages, tons of assets and require running Celery, Postgres, Redis, etc.. I was trying to create a new project last month trying to put all of these together. One thing that I was stuck in was putting these together: - non-single-thread-Flask - Redis pub-sub, subscribe to listen to a channel - and websocket (such as socket.io) upon receiving redis sub message, pushing the message to client side. I found out that this was a non-trivial thing to do, as redis subscribe is a blocking function, and I was not able to push websocket message from redis subscribe callback. I wonder how did they do that, if any?
- nickjj 9y ago
  In the past, when I needed websockets in a Flask app I used Faye (backed by Redis) for the websocket back-end. It was really easy to set up with Docker as it just becomes another container in your Compose file. Nowadays I would just use Pusher.
- dfee 9y ago
  Fanout/Pushpin works here as does having an aiohttp wrapped WSGI Flask app.

by nickjj 9y ago

Spiritus 9y ago

My problem with using Docker (only) is that it doesn't translate well to editors. Like, using jedi-vim[1] with a virtualenv constructed by a Docker container doesn't work at all. Unless I actually run vim itself inside said container. So unless your dependences build on macOS (like in my case), everything goes out the door. [1] https://github.com/davidhalter/jedi-vim https://github.com/davidhalter/jedi-vim

AlexCoventry 9y ago
Emacs has pretty nice integrations with inferior docker processes. I'm getting them via spacemacs, so I'm not sure what specific package provides it, but I assume it's elpy.
- cat199 9y ago
  what kinds of interactions? (like: what operations, etc) using elpy here with a native install and can use e.g. tramp for remote editing/python sessions in most cases (even across machines), but determining docker mount points to edit in-machine not so much.. I suppose one can simply map a local code directory into a runtime environment, but this also makes e.g. interacting with an in-container interpreter a bit bothersome (not so bad actually, but have to set a separate interpreter path to something like 'docker exec -it ctid python' would like to track this down I suppose
  - AlexCoventry 9y ago
    C-c C-p will start an inferior python shell in the container, for instance.
_drFaust 9y ago
We've got about 30+ backends in python all wrapped in docker containers. Majority of the team was pure vim before I joined and they're slowly converting to pycharm after seeing how nicely you can setup a remote interpreter against a docker container. And it has vim bindings so you don't have to re-learn new hotkeys. I've also been following this VS Code issue on adding remote docker support for python https://github.com/Microsoft/vscode-python/issues/79#issuecomment-366000230 https://github.com/Microsoft/vscode-python/issues/79#issueco... However, if you're a hardcore vim guy then I doubt these IDEs are gonna satiate your current flow.
- nickjj 9y ago
  Funny you mentioned that issue. I'm eagerly awaiting that feature too. I commented in the issue a few weeks ago: https://github.com/Microsoft/vscode-python/issues/79#issuecomment-362836925 https://github.com/Microsoft/vscode-python/issues/79#issueco.... Once that's implemented, oh man, development nirvana.
- wirrbel 9y ago
  I remember, on my laptop with pycharm and docker (and the virtual machine docker lived in), the RAM usage was just excessive. I am by no means a minimalist, but data-science stuff was barely impossible with 16 GB RAM. Also, I strongly dislike PyCharm. I am a vim guy by heart, but I am generally not against IDEs. VS Code is okayish. For C++ development, I really loved Visual Studio. But PyCharm just feels wrong, bloated, slow and baroque
- jsmeaton 9y ago
  What I do is create the virtualenv locally that docker would also create, and point my editor to the local virtualenv. It gives me all of the intellisense locally. I still need to debug within the container/shell for the moment but I'm hoping there's a pathway coming for vscode that pycharm already has.
- sheraz 9y ago
  yup. happy pycharm and docker user here. even have it working with debugging and breakpoints.
Walkman 9y ago
Ever heard about bind mounting? My current docker image for a Flask app doesn't even contains my code.
- maccam94 9y ago
  Are you mounting your virtualenv into the container somehow? How does your editor see the code/docs for your pip dependencies?
  - yetanother1980 9y ago
    I have not tried it but I imagine this would work fine. Project Project/venv Project/src And then have the whole project as a volume. Then your editor can see the same files as you have in the container.
mikewhy 9y ago
Lots of people mentioning you can maybe debug a remote process, but none suggesting what to do about completions. Maybe you can use bind mounts, but I'm not sure how well that works in practice, or with eg a package with a native component.
- jsmeaton 9y ago
  See my comment a few above. Setup a virtualenv locally that your dev env points to for completions, but actually run the app within the container (with its own virtualenv). Docker is great for dependencies like databases and queues. I find it totally unnecessary for developing python.

axitanull 9y ago

>These are pretty big Flask apps too, which have thousands of lines of code, dozens of packages, tons of assets and require running Celery, Postgres, Redis, etc.. I was trying to create a new project last month trying to put all of these together. One thing that I was stuck in was putting these together: - non-single-thread-Flask - Redis pub-sub, subscribe to listen to a channel - and websocket (such as socket.io) upon receiving redis sub message, pushing the message to client side. I found out that this was a non-trivial thing to do, as redis subscribe is a blocking function, and I was not able to push websocket message from redis subscribe callback. I wonder how did they do that, if any?

nickjj 9y ago
In the past, when I needed websockets in a Flask app I used Faye (backed by Redis) for the websocket back-end. It was really easy to set up with Docker as it just becomes another container in your Compose file. Nowadays I would just use Pusher.
dfee 9y ago
Fanout/Pushpin works here as does having an aiohttp wrapped WSGI Flask app.