6 ms·

With Python, the only sane choice is probably through Docker images :-( This is the reason why I consciously attempt to move away from Python and choose Go or

With Python, the only sane choice is probably through Docker images :-( This is the reason why I consciously attempt to move away from Python and choose Go or Rust for new projects if possible. Of course, on existing projects, Python deployment is a pain.

lvncelot 5y ago
Last I've heard, the nix experience for python was kind of lacking, sadly - but I don't have any first-hand experience to go with that statement.
ramtatatam 5y ago
Why is deploying through docker images not OK? I always thought that's what docker is meant to solve.
- willis936 5y ago
  Docker was made for development and testing of code, not production. It has been co-opted for production with good success. It seems fine to do it for hobbyist projects, but still feels dirty in a commercial setting.
  - ramtatatam 5y ago
    I'm using docker in commercial environments for years and cannot complain... But I don't distribute software, which is where the problem is probably more visible.
  - iainmerrick 5y ago
    Why?
    - willis936 5y ago
      Why what?
  - evgen 5y ago
    The largest internet services on the planet were using LXC containers for production ops before you ever heard of the term 'docker'. In its earliest iterations it was easier to push a container to production than to run it on your laptop.
    - KaiserPro 5y ago
      > using LXC containers for production which isn't docker. From what I recall, spotify were the first company with a large footprint to use docker. However for some reason they skipped VMs and went screaming into docker when it was _very_ new. Personally that seemed like a mistake, but you know, each to their own. If you're wanting to get into a bun fight about containers, then IBM 360 and JCL has some time for you.
  - Daishiman 5y ago
    If we stopped using software in prod because it feels "dirty" we would not be using computers at all.
- stult 5y ago
  Desktop/local usage, especially for non technical users or people working in environments where they can’t install docker (eg locked down corporate machine). There isn’t a great way to pass around a single executable or installer for a Python app. There’s pyinstaller, but it’s finicky and doesn’t quite work cross platform (you have to build the executable on the target OS). On the other hand, OP’s setup makes it very easy to publish packages. So you can create a tarball and have a user pip install that, then run your app with a simple CLI. Or publish to pypi if you want it public. The downside is you assume the user has the right version of Python and knows how to switch versions if need be and all the weirdness that comes with that. But for web apps, packaging your app also makes it easy to wrap in a simple dockerfile that basically just installs the package and then runs it.
  - marderfarker2 5y ago
    I’ve deployed Python desktop apps quite successfully by using onedir with pysintaller and then package the contents using NSIS.
  - codethief 5y ago
    > There isn’t a great way to pass around a single executable or installer for a Python app. Have you tried Nuitka or mypyc? (Haven't tried them myself but I've heard very good things about Nuitka.)
    - nerdponx 5y ago
      Also Briefcase: https://beeware.org/project/projects/tools/briefcase/ https://beeware.org/project/projects/tools/briefcase/. "Convert a Python project into a standalone native application."
    - stult 5y ago
      Personally I have not. I have only had one particularly weird use case where I really needed to deliver a Python executable alongside an Electron app, and made it work with Pyinstaller. We’re actually reworking things to use Pyodide to compile the Python to WASM so we don’t have to deliver any separate Python code at all. It’s a real Rube Goldberg machine of an application, but we had a lot of weird constraints we had to meet (eg, we couldn’t use sockets to communicate between Python and Node, so had to package the Python as an executable). It works beautifully but it was a mess to get running.
sitkack 5y ago
tarballs, rpms or debs also work. One should probably run their own package server like https://github.com/pypiserver/pypiserver https://github.com/pypiserver/pypiserver All of that said, containers are nice because you have a log of what is running, easy to transport and coordinate. When you use Go and Rust over Python, does the use of Docker disappear? What replaces it?
- amrox 5y ago
  > One should probably run their own package server like https://github.com/pypiserver/pypiserver https://github.com/pypiserver/pypiserver Never used pypiserver but I’ve had a good experience with https://github.com/devpi/devpi https://github.com/devpi/devpi
asplake 5y ago
I wouldn’t write off platforms such as Elastic Beanstalk and Heroku for web projects, smaller ones of bog-standard shape anyway
- KaiserPro 5y ago
  Heroku, yes I would push for that. Its expensive, but it'll save you an entire devop function right up til you hit the 30 people mark. Elastic beanstalk is just a horrid dev environment. Lots of waiting, lots of non-obvious options, and very little reward. I would personally push for lambda and zappa (https://github.com/zappa/Zappa https://github.com/zappa/Zappa) for python, as it seems to be much easier to deploy and debug.
KaiserPro 5y ago
> the only sane choice is probably through Docker images Its really not. You either tailor your dependencies to match the host (clue, you should be doing this anyway, it makes things much easier in the long run), or use virtual envs. pyenv also gives you some (non obvious) flexibility as well However, having said that, the chances are, your go or rust binary is going to be in a docker env as well(so is your python), so its basically all the same.
blooalien 5y ago
I'm personally a big fan of (especially) the most recent builds of Python 3, but lately I've been interested in learning Go or Rust just because there's a few things I really like about them both vs some other languages I've looked at. So, seein' that you've coded in both Go and Rust apparently, my question to you is: Which do you prefer of the two (and why)? I personally lean a bit toward Go, but I haven't learned enough of either to decide absolutely which of the two I should learn first.
- reacharavindh 5y ago
  I think it is better to not _decide_ which one to learn. They both have their perks and quirks. If you can, I’d recommend to dip your toes on both. For network services, Go feels super nice because of the high quality libraries in its ecosystem. I feel like it is better Java. Rust on the hand, feels like a better C++... with a steep learning curve, and useful when you can/need to get something correct/efficient/safe at the cost of increased developer time and cognitive load. They both have their place IMHO.
  - blooalien 5y ago
    Nice. Thank you for the info. Do they both have pretty good GUI toolkit support available? I currently use PyQt5 in Python, but I'm not totally against GTK if that's what is easier for Go or Rust. Still think I kinda feel like Go's the one of the two I'm prolly gonna learn first, although you've got me leanin' hard toward also learning Rust in addition to Go.
alephu5 5y ago
If you're deploying a REST API with authentication, caching and so on then the KISS solution is python with a framework (such as Django) in a docker container. I've spent many many hours (years) trying nix, rust, Haskell, go, spring framework and all sorts of other things which are a lot of fun but not so good for getting shit done. For other domains this doesn't apply of course; lower-level network stuff, portable CLIs, CPU-intensive workloads and so on are much better in go/rust but you can either integrate them with a network call, spawning an OS process or an FFI in the case of rust.
aitchnyu 5y ago
I'm using Pycharm Pro and it can use Docker for code completion. I use Docker AND venv on host filesystem for friends using Pycharm Community and VSCode. Is this still necessary?

lvncelot 5y ago

Last I've heard, the nix experience for python was kind of lacking, sadly - but I don't have any first-hand experience to go with that statement.

ramtatatam 5y ago

Why is deploying through docker images not OK? I always thought that's what docker is meant to solve.

willis936 5y ago
Docker was made for development and testing of code, not production. It has been co-opted for production with good success. It seems fine to do it for hobbyist projects, but still feels dirty in a commercial setting.
- ramtatatam 5y ago
  I'm using docker in commercial environments for years and cannot complain... But I don't distribute software, which is where the problem is probably more visible.
- iainmerrick 5y ago
  Why?
  - willis936 5y ago
    Why what?
- evgen 5y ago
  The largest internet services on the planet were using LXC containers for production ops before you ever heard of the term 'docker'. In its earliest iterations it was easier to push a container to production than to run it on your laptop.
  - KaiserPro 5y ago
    > using LXC containers for production which isn't docker. From what I recall, spotify were the first company with a large footprint to use docker. However for some reason they skipped VMs and went screaming into docker when it was _very_ new. Personally that seemed like a mistake, but you know, each to their own. If you're wanting to get into a bun fight about containers, then IBM 360 and JCL has some time for you.
- Daishiman 5y ago
  If we stopped using software in prod because it feels "dirty" we would not be using computers at all.
stult 5y ago
Desktop/local usage, especially for non technical users or people working in environments where they can’t install docker (eg locked down corporate machine). There isn’t a great way to pass around a single executable or installer for a Python app. There’s pyinstaller, but it’s finicky and doesn’t quite work cross platform (you have to build the executable on the target OS). On the other hand, OP’s setup makes it very easy to publish packages. So you can create a tarball and have a user pip install that, then run your app with a simple CLI. Or publish to pypi if you want it public. The downside is you assume the user has the right version of Python and knows how to switch versions if need be and all the weirdness that comes with that. But for web apps, packaging your app also makes it easy to wrap in a simple dockerfile that basically just installs the package and then runs it.
- marderfarker2 5y ago
  I’ve deployed Python desktop apps quite successfully by using onedir with pysintaller and then package the contents using NSIS.
- codethief 5y ago
  > There isn’t a great way to pass around a single executable or installer for a Python app. Have you tried Nuitka or mypyc? (Haven't tried them myself but I've heard very good things about Nuitka.)
  - nerdponx 5y ago
    Also Briefcase: https://beeware.org/project/projects/tools/briefcase/ https://beeware.org/project/projects/tools/briefcase/. "Convert a Python project into a standalone native application."
  - stult 5y ago
    Personally I have not. I have only had one particularly weird use case where I really needed to deliver a Python executable alongside an Electron app, and made it work with Pyinstaller. We’re actually reworking things to use Pyodide to compile the Python to WASM so we don’t have to deliver any separate Python code at all. It’s a real Rube Goldberg machine of an application, but we had a lot of weird constraints we had to meet (eg, we couldn’t use sockets to communicate between Python and Node, so had to package the Python as an executable). It works beautifully but it was a mess to get running.

sitkack 5y ago

tarballs, rpms or debs also work. One should probably run their own package server like https://github.com/pypiserver/pypiserver https://github.com/pypiserver/pypiserver All of that said, containers are nice because you have a log of what is running, easy to transport and coordinate. When you use Go and Rust over Python, does the use of Docker disappear? What replaces it?

amrox 5y ago
> One should probably run their own package server like https://github.com/pypiserver/pypiserver https://github.com/pypiserver/pypiserver Never used pypiserver but I’ve had a good experience with https://github.com/devpi/devpi https://github.com/devpi/devpi

asplake 5y ago

I wouldn’t write off platforms such as Elastic Beanstalk and Heroku for web projects, smaller ones of bog-standard shape anyway

KaiserPro 5y ago
Heroku, yes I would push for that. Its expensive, but it'll save you an entire devop function right up til you hit the 30 people mark. Elastic beanstalk is just a horrid dev environment. Lots of waiting, lots of non-obvious options, and very little reward. I would personally push for lambda and zappa (https://github.com/zappa/Zappa https://github.com/zappa/Zappa) for python, as it seems to be much easier to deploy and debug.

KaiserPro 5y ago

> the only sane choice is probably through Docker images Its really not. You either tailor your dependencies to match the host (clue, you should be doing this anyway, it makes things much easier in the long run), or use virtual envs. pyenv also gives you some (non obvious) flexibility as well However, having said that, the chances are, your go or rust binary is going to be in a docker env as well(so is your python), so its basically all the same.

blooalien 5y ago

I'm personally a big fan of (especially) the most recent builds of Python 3, but lately I've been interested in learning Go or Rust just because there's a few things I really like about them both vs some other languages I've looked at. So, seein' that you've coded in both Go and Rust apparently, my question to you is: Which do you prefer of the two (and why)? I personally lean a bit toward Go, but I haven't learned enough of either to decide absolutely which of the two I should learn first.

reacharavindh 5y ago
I think it is better to not _decide_ which one to learn. They both have their perks and quirks. If you can, I’d recommend to dip your toes on both. For network services, Go feels super nice because of the high quality libraries in its ecosystem. I feel like it is better Java. Rust on the hand, feels like a better C++... with a steep learning curve, and useful when you can/need to get something correct/efficient/safe at the cost of increased developer time and cognitive load. They both have their place IMHO.
- blooalien 5y ago
  Nice. Thank you for the info. Do they both have pretty good GUI toolkit support available? I currently use PyQt5 in Python, but I'm not totally against GTK if that's what is easier for Go or Rust. Still think I kinda feel like Go's the one of the two I'm prolly gonna learn first, although you've got me leanin' hard toward also learning Rust in addition to Go.

alephu5 5y ago

If you're deploying a REST API with authentication, caching and so on then the KISS solution is python with a framework (such as Django) in a docker container. I've spent many many hours (years) trying nix, rust, Haskell, go, spring framework and all sorts of other things which are a lot of fun but not so good for getting shit done. For other domains this doesn't apply of course; lower-level network stuff, portable CLIs, CPU-intensive workloads and so on are much better in go/rust but you can either integrate them with a network call, spawning an OS process or an FFI in the case of rust.

aitchnyu 5y ago

I'm using Pycharm Pro and it can use Docker for code completion. I use Docker AND venv on host filesystem for friends using Pycharm Community and VSCode. Is this still necessary?