5 ms·

2012: Python is Awesome! 2014: Python is a pain in the butt to manage packages and dependencies, how the hell am I gonna deploy this? It's still a mess 10 yea

by sph 2y ago

2012: Python is Awesome! 2014: Python is a pain in the butt to manage packages and dependencies, how the hell am I gonna deploy this? It's still a mess 10 years later unless you live deep in the ecosystem and know what third-party solutions du jour to manage that complexity. At least we have Docker now. Also let's not forget the Python 3 migration fiasco that lasted ~2008-2018 and I still find myself porting libraries to this day. I haven't used Python in a personal project in 10 years because of these painful paper cuts .

andreif 2y ago
Dependencies is a sign of weakness.
- tmountain 2y ago
  As opposed to writing everything from scratch? Python has a huge standard library. Making it bigger to avoid dependencies would add more bloat to the distribution.
- pjc50 2y ago
  Ah yes, all applications should be a single very large source file that issues OS syscalls directly.
- markhahn 2y ago
  and automatic internet-supplied dependencies are a sign of supplychain vulnerability
aduwah 2y ago
chore(omfg): fix print to print()
- dotancohen 2y ago
  I have a feeling that you've never dealt with sequences of bytes (encoded strings) in Python 2.
- bb88 2y ago
  Fix print to logger.info() why not?
derriz 2y ago
At least for Python dependency management, I'd hardly call it a mess these days and yes it was a horrible 10 years ago. The de-facto standard of venv and pip (using a requirements.txt file) is generally painless these days. To the extent that moving between MacOS, Linux and Windows is feasible for most (of my) Python work. And the Python 2/3 pain stopped being an issue a few years back. I haven't been forced because of library requirement to use Python 2 in years. Your criticisms WERE valid 5 or 10 years ago. In my experience, the ecosystem has improved greatly since then. I wouldn't use Python for large and shared developer code base but for one-man experiments and explorations, Python is pretty compelling these days in my opinion.
- theshrike79 2y ago
  It still won't beat the deployment speed of scp executable user@host:direc/tory/ that you get with Go. I still use python for stuff that never leaves my computer, but in most cases if I know I need to run it on Someone Else's Machine (or even a server of mine) I'll reach for Go instead.
  - sph 2y ago
    I agree with you, though if you live in a Linux+podman world, you can literally scp whole containers, so at least we made some progress.
  - derriz 2y ago
    Sure but that's not going to work across platforms, right? So not really an apples to apples comparison.
    - thrwaway1985882 2y ago
      Even the most ardent of Go-haters have to be fair and admit it makes cross-compilation really nice. GOOS=linux GOARCH=amd64 go build && scp executable user@host:direc/tory/ Out of the box, no toolchains to manage.
      - 2y ago
imtringued 2y ago
The only answer to is to use rye and I mean it. pyenv, pip, venv, pipenv, poetry, it's all crap. (yes rye uses venv, etc under the hood, I know)
underdeserver 2y ago
Hah, even at Google, the Python 3 migration lasted well into 2020.
- bb88 2y ago
  I remember a large bank building a major new project in 2017 with python 2.7.
sunrunner 2y ago
> At least we have Docker now I'm not so sure this is a desirable state to be in long-term. While Docker (and other container solutions) does provide solutions to some of the challenges around environment consistency, build artefact management, development environments with linked services, etc., it's also one of numerous de-facto tools that in some ways ends up putting more effort on the end-user to adopt. Less effort than installing all dependencies yourself? Absolutely. More effort (and for no real reward) than a single binary built natively for the target platform? Yep. This feels like a sort of 'shift right' approach to releasing an application in terms of developer effort. As a developer you can just bundle everything into a container image and now the end user needs Docker and the install is going to leave a mess of image layers on their system. You don't need all the dependencies installed system-wide, that's true, but why can't they just be bundled with the application? It's also not a great cross-platform tool, as Docker on anything except Linux just hides the requisite virtualisation. It's mostly transparent, true, but is still plagued by platform-specific issues such as write performance on mounted volumes on macOS being ~10x as slow (there's a fairly old thread on the Docker forums that's still open with this as a reported issue, and it does't look like it's going away). Additionally, while disk storage _is_ (comparably) cheap nowadays, the buildup of image layers that contain endless copies of binary dependencies doesn't feel great. I don't want or need hundreds of copies of libc et al. all with slightly different versions sitting around. Even the 'slim' Debian images are min. 120MB, let alone the 1GB full ones. As Python is on topic for this thread the Alpine images are smaller, but then Alpine image builds for Python don't work with the standard PyPI wheels (I'm not actually sure if this is still true, need to check). As languages like Python are just source releases, some tooling is definitely needed to make that more palatable in terms of handling releases of Python tools (just big collections of scripts), but being able to build and copy a single binary for a Go application feels great having spent a lot of time practically requiring Docker for release Python tools. And all of this is really just getting us back to where we were years ago, where it was actually feasible to release a native binary for a target platform. The difference is that Go's tooling around related areas such as dependency management and cross-platform compilation is more standardised, and as a result the developer experience for this is much nicer than with other comparable languages that also compile to native code. Arguably a lot of the development decisions around how to manage and release tools comes down to how good the experience is for developers. Docker ends up being favourable for Python because there's no 'standard' Python way to bundle applications with their dependencies and Docker basically solves this problem, even if in a messy way. Does Docker have its place? Definitely. Do I hope it sticks around in favour of improved cross-platform developer tooling? Absolutely not.
mtalantikite 2y ago
I've always wanted to like Python, but I got out of undergrad in 2006 and pretty much the entire first part of my career was the ridiculous Python 3 migration nonsense. Every time I wanted or needed to use Python there was some dependency management nightmare to deal with. For like a decade or so. I think I'm like you, I have some sort of Python ptsd from those years and don't really care to venture into it again. It sounds like it's gotten a lot better recently, but also after years of using Go and TypeScript I can't imagine working on a codebase without static types.
bb88 2y ago
We use poetry for dependency management in production, and it's fine.

by sph 2y ago

andreif 2y ago

Dependencies is a sign of weakness.

tmountain 2y ago
As opposed to writing everything from scratch? Python has a huge standard library. Making it bigger to avoid dependencies would add more bloat to the distribution.
pjc50 2y ago
Ah yes, all applications should be a single very large source file that issues OS syscalls directly.
markhahn 2y ago
and automatic internet-supplied dependencies are a sign of supplychain vulnerability

aduwah 2y ago

chore(omfg): fix print to print()

dotancohen 2y ago
I have a feeling that you've never dealt with sequences of bytes (encoded strings) in Python 2.
bb88 2y ago
Fix print to logger.info() why not?

derriz 2y ago

At least for Python dependency management, I'd hardly call it a mess these days and yes it was a horrible 10 years ago. The de-facto standard of venv and pip (using a requirements.txt file) is generally painless these days. To the extent that moving between MacOS, Linux and Windows is feasible for most (of my) Python work. And the Python 2/3 pain stopped being an issue a few years back. I haven't been forced because of library requirement to use Python 2 in years. Your criticisms WERE valid 5 or 10 years ago. In my experience, the ecosystem has improved greatly since then. I wouldn't use Python for large and shared developer code base but for one-man experiments and explorations, Python is pretty compelling these days in my opinion.

theshrike79 2y ago
It still won't beat the deployment speed of scp executable user@host:direc/tory/ that you get with Go. I still use python for stuff that never leaves my computer, but in most cases if I know I need to run it on Someone Else's Machine (or even a server of mine) I'll reach for Go instead.
- sph 2y ago
  I agree with you, though if you live in a Linux+podman world, you can literally scp whole containers, so at least we made some progress.
- derriz 2y ago
  Sure but that's not going to work across platforms, right? So not really an apples to apples comparison.
  - thrwaway1985882 2y ago
    Even the most ardent of Go-haters have to be fair and admit it makes cross-compilation really nice. GOOS=linux GOARCH=amd64 go build && scp executable user@host:direc/tory/ Out of the box, no toolchains to manage.
    - 2y ago

imtringued 2y ago

The only answer to is to use rye and I mean it. pyenv, pip, venv, pipenv, poetry, it's all crap. (yes rye uses venv, etc under the hood, I know)

underdeserver 2y ago

Hah, even at Google, the Python 3 migration lasted well into 2020.

bb88 2y ago
I remember a large bank building a major new project in 2017 with python 2.7.

sunrunner 2y ago

> At least we have Docker now I'm not so sure this is a desirable state to be in long-term. While Docker (and other container solutions) does provide solutions to some of the challenges around environment consistency, build artefact management, development environments with linked services, etc., it's also one of numerous de-facto tools that in some ways ends up putting more effort on the end-user to adopt. Less effort than installing all dependencies yourself? Absolutely. More effort (and for no real reward) than a single binary built natively for the target platform? Yep. This feels like a sort of 'shift right' approach to releasing an application in terms of developer effort. As a developer you can just bundle everything into a container image and now the end user needs Docker and the install is going to leave a mess of image layers on their system. You don't need all the dependencies installed system-wide, that's true, but why can't they just be bundled with the application? It's also not a great cross-platform tool, as Docker on anything except Linux just hides the requisite virtualisation. It's mostly transparent, true, but is still plagued by platform-specific issues such as write performance on mounted volumes on macOS being ~10x as slow (there's a fairly old thread on the Docker forums that's still open with this as a reported issue, and it does't look like it's going away). Additionally, while disk storage _is_ (comparably) cheap nowadays, the buildup of image layers that contain endless copies of binary dependencies doesn't feel great. I don't want or need hundreds of copies of libc et al. all with slightly different versions sitting around. Even the 'slim' Debian images are min. 120MB, let alone the 1GB full ones. As Python is on topic for this thread the Alpine images are smaller, but then Alpine image builds for Python don't work with the standard PyPI wheels (I'm not actually sure if this is still true, need to check). As languages like Python are just source releases, some tooling is definitely needed to make that more palatable in terms of handling releases of Python tools (just big collections of scripts), but being able to build and copy a single binary for a Go application feels great having spent a lot of time practically requiring Docker for release Python tools. And all of this is really just getting us back to where we were years ago, where it was actually feasible to release a native binary for a target platform. The difference is that Go's tooling around related areas such as dependency management and cross-platform compilation is more standardised, and as a result the developer experience for this is much nicer than with other comparable languages that also compile to native code. Arguably a lot of the development decisions around how to manage and release tools comes down to how good the experience is for developers. Docker ends up being favourable for Python because there's no 'standard' Python way to bundle applications with their dependencies and Docker basically solves this problem, even if in a messy way. Does Docker have its place? Definitely. Do I hope it sticks around in favour of improved cross-platform developer tooling? Absolutely not.

mtalantikite 2y ago

I've always wanted to like Python, but I got out of undergrad in 2006 and pretty much the entire first part of my career was the ridiculous Python 3 migration nonsense. Every time I wanted or needed to use Python there was some dependency management nightmare to deal with. For like a decade or so. I think I'm like you, I have some sort of Python ptsd from those years and don't really care to venture into it again. It sounds like it's gotten a lot better recently, but also after years of using Go and TypeScript I can't imagine working on a codebase without static types.

bb88 2y ago

We use poetry for dependency management in production, and it's fine.