7 ms·
Python Modern Practices
- joshuamcginnis 2y agoI like miniconda for managing personal python projects.
- jw887c 2y ago>Avoid using Poetry for new projects. Poetry predates many standards for Python tooling. This means that it uses non-standard implementations of key features, such as the dependency resolver and configuration formats in pyproject.toml files. What? This is the first I've heard of this.
- SCUSKU 2y agoWas going to comment the same thing. Would love to hear the author expand further on why not use Poetry. I've found it to be pretty solid and continue to use Poetry + Pyenv for all my projects, but open to hearing the case for PDM or Hatch.
- hobs 2y agoOne thing I saw which was weird as hell was poetry has serious issues with changing windows requirements (specifically removing them) and has a tendency to throw all sorts of fun errors (its method of removing files is not file system friendly.) As far as I can tell the issue is still open and has been for awhile.
- alisonatwork 2y agoI've never worked on a team that uses Poetry, but in my current company another team uses it, and I haven't found it really as slick as I would have imagined, primarily because you need to create a venv and install poetry into that before you even get started, which by that point why not just pip install the rest anyway? For standalone applications it just seems like an unnecessary extra step. It doesn't even mandate a build lifecycle like Maven so what are you getting? But that's not what soured me on Poetry. What soured me was recently I needed to create a release of one of their libraries with a Git commit in the local version identifier and... Poetry doesn't do that. There's an issue that was open on GitHub for years before they finally agreed to implement it, and since February the change is now merged to master, but despite several point releases since then, that change has not landed in any of them. When will we be able to get a local version part? Who knows! This experience has really made me skeptical of Poetry being the One True Packaging Tool that fixes everything. As usual, it just fixes the things the devs want fixed and everything else is still janky or half implemented. From my perspective, if you're gonna deal with jank anyway, might as well just deal with the standard jank that comes as part of Python itself.
- skeledrew 2y agoActually poetry comes with an optional self-installer, though I prefer to manage it with pipx. And it's recommended not to install it into your project env, as there's the potential for conflicting dependency versions.
- odie5533 2y agoPoetry should not be installed into the local venv. That is a mistake. You should install it with homebrew or using its standalone installer.
- rat87 2y agoAfter struggling with the complexity of pyenv and slowness of poetry I'm really happy with rye. It manages both python versions (which it downloads instead of compiling) and package versions. It's written in rust so it's faster and can replace pipx as well for installing global tools. (Some people will recommend uv which rye is slowly merging with buy uv is still missing some rye features, probably some time in the future you might want to switch).
- odie5533 2y agomise is also a great alternative to pyenv and works with for Node projects too.
- spacephysics 2y agoYeah to me Poetry was a game changer in dependency tooling. Not that better tooling can come about, but it’s recently worked great on multiple production grade projects. No complaints, and has shoot-in-the-foot safety features
- tinix 2y agopyenv? great! then just use pip with requirements.txt... what's wrong with pip freeze? why are there so many competing tools? it's very anti-python IMO.
- deleted 2y ago[deleted]
- tmnvix 2y ago> what's wrong with pip freeze? I prefer my requirements.txt to include only the packages I install with pip myself (and not their dependencies).
- tinix 2y agoso then maintain a curated requirements.txt...
- jampekka 2y agoPython packaging has always been a mess, and will probably always be. The reasons are mostly sociological.
- jcparkyn 2y ago> what's wrong with pip freeze? Aside from all the obvious issues of having no distinction between transitive and direct dependencies, it completely breaks cross-platform support if any of your dependencies have platform-specific sub-dependencies (which is not uncommon in python).
- tinix 2y agopip freeze was an overly simplified hyperbolic jest... fair point, so when this packaging mess happens, can one not strike a balance, and define the dependencies that you need, then, and let the resolver handle the rest?
- agumonkey 2y agoI recall some discussion on the extended tool configuration, the section names have seen different conventions over time.
- cmcconomy 2y agopyenv and poetry always my goto
- odie5533 2y agoReplace pyenv with mise. It's faster and supports Node projects.
- mikepurvis 2y agoAgree, I'm a poetry person too. I feel like TFA might be getting ahead of itself on this point. Just because something was ratified as a "standard" doesn't immediately make everything prior irrelevant.
- chao- 2y agoAgreed. Most of these are fairly uncontroversial. A few will probably be new to some, particularly because users of Python are heterogenous and people find themselves with years of experience on projects that never upgraded their language version, e.g. from 3.6, and don't know that they're missing. A final few, even with the explanation given, feels like preference, such as the Poetry opinion. I suppose if I wrote a similar article, it would end up with a similar mix of obvious defaults and a few of my preferences. It could also be because of different use cases? For example, writing a library for packaging, vs a deployed application, vs a personal projects' workbench. Or perhaps if you are collaborating with people who use Windows without WSL. I have heard tell that Poetry can sometimes trip over its own shoelaces on Windows. I have never experienced it myself, and don't personally care to support any projects involving Windows, but if I did I might have different preferences.
- helsinki 2y agoI've not had any issues using mamba environments and vanilla pip. Not sure why I would need to change that? I have packages that thousands of people use, and others that help manage billions of dollars. I've managed to keep things simple without upgrading tooling. This article feels like it's suggesting flavor of the month technologies without explaining why the status quo is bad. Also, you can't really use virtualenvs easily when you have packages that depend on C libraries.
- skeledrew 2y agoI use conda exactly so I can manage non-Python stuff in envs.
- martinky24 2y agoOverall, generally reasonable. Nothing is a hard and fast rule, if you know what you're doing and have a good reason for going down an alternative path, that's great. But if on a new project you generally follow these you'll be setting yourself up for success.
- behnamoh 2y agoHas anyone tried orchestrating Python code using Elixir/Erlang? I feel like the true concurrency can only be achieved in the actor model of BEAM, and async still just does concurrent code, not parallel.
- ctoth 2y agoSad to not see Attrs mentioned. Dataclasses are cool, I guess, but Attrs is seriously great.
- justanotherbody 2y agoIt's interesting to me that dataclasses seemed to be a slimmed down attrs but in practice I find it replaces namedtuple not attrs For my use the key is how easily you can add simple conversion of values to attrs. IIRC this was intentionally omitted from dataclasses. For a 1-off using a factory with a dataclass is easy but repeated uses send me back to attrs
- eesmith 2y agoYeah, I think namedtuple's popularity (recommended in this link) was a mistake. It's sole modern use case should be to turn tuples used as an unnamed struct into a named struct while preserving tuple indexing for backward compatibility. Most people don't need, say, a[-3] as an alias for a.field_name. Otherwise, for those who want the standard library, use a dataclass with frozen=True for immutability.
- saboot 2y agoattrs looks interesting, can it be used in workflows for processing GBs of data? I don't see any Cython compatibility or other Python to compiled code options. I typically just get by with numpy recarrays, but I'm old fashioned and looking for something more modern.
- justanotherbody 2y agoAttrs would not replace your record array. IIRC you can supply slots so it might be useful for converting raw data into your numpy types, but I suspect you'd find the overhead in function calls problematic If you want easier than recarray and almost as fast I think you're in pandas-ville
- frankjr 2y agoI'm surprised the post doesn't mention uv, which did quite well on HN a few months ago. Any experience? https://github.com/astral-sh/uv https://github.com/astral-sh/uv https://news.ycombinator.com/item?id=39387641 https://news.ycombinator.com/item?id=39387641
- screye 2y agoI like using UV, it is painless, a proper in-place replacements and blazingly fast. I have seen occasional hiccups, so we don't use it in prod just yet. But I use it in local all the time. They've been responsive to issues, so give it a shot.
- rat87 2y agoI'm surprised it doesn't mention rye. I now that technically the projects are combining and uv is eventually meant to eat rye but for now I think it offers higher level functionality that I'm unsure if uv provides. Like pinning /downloading different Python interpreters. Rye and uv both allow installing tools in custom venvs removing the need for pipx.
- tgz 2y agoCan uv replace a `pipx --global` install option?
- kebman 2y agoFor many sections I'd ask myself "Why?" Sure, I know you can click on the links and do deeper research, but I'd love a quick "why" blurb for each section. Just a short "Why this is nice? Well, because..."
- thomasahle 2y ago> Use requirements.txt Files to Install Packages Into Environments Isn't this obsoleted by the previous section on using pyproject.toml?
- igorguerrero 2y agoAs far as I know you're right, also using pyproject.toml generates platform independent dependency trees (I don't have experience with the tools he mentions but that's what poetry does), requirements.txt does not and must be generated on a server or carefully picking the crossplatform version of the packages. Also kinda weird he says to not use pip, but requirements.txt are pip commands... And should be running with pip... Hmm I don't know about this article anymore.
- rat87 2y agouv and rye (which depends on uv) use pyproject.toml and spit out requirements.txt like files as lock files. They aren't platform independent but are easier to install in a docket container (just use pip inside the container). Not sure about hatch/pdm which they recommended. Some people use pip-compile which also converts toml files in requirements.txt files
- skeledrew 2y agoNot until it gets thrown out kicking and screaming like python 2 was, I guess.
- routerl 2y agoAFAIK several cloud services rely on requirements.txt for deployment.
- throwadobe 2y agoFor what it's worth, I still prefer to use pip, python -m venv and virtualenvwrapper. Solves 99% of the issues with 1% of the complexity
- mikepurvis 2y agoI prefer that for environment-oriented usage (literally I just need this stuff on my pythonpath pls) vs poetry I use for project oriented use.
- routerl 2y agoIs there something wrong with pipenv[0] that I'm not aware of? It really feels like a full solution, I've been using it for years, and I can't understand why it isn't more widely used. [0] pipenv.pypa.io
- lolpanda 2y agowhat are the pros and cons with pydantic models vs data classes? i like the runtime validation in pydantic models. the pattern i landed on is that 1) only use primitive data types such as str, int, pydantic models and pandas dataframes to present data. 2) no classes with methods, no polymorphism, only use module functions.
- martinky24 2y agoPydantic models are significantly more heavyweight than dataclasses -- They serve as validators as opposed to dataclasses, which are primarily just tools for managing structured data. Both are important problems, but they are different. The power of Pydantic models comes with benefits in terms of what they're able to do out of the box. But this comes at a significant cost of speed, which does matter for some applications. There are plenty of applications where it simply doesn't make sense to validate types every time you stand up a class. Sure you can turn some of that off with Pydantic, etc, etc, but the fact that dataclasses don't require a third party install and are efficient and easy out of the box makes them useful and non-duplicative.
- wanderingmind 2y agoCurious why there is no mention of conda and its fast alternative mamba (micromamba). In my experience, conda is much more robust in managing. dependencies conflict (thanks to AWS and especially boto3) compared to pip.
- skeledrew 2y agoI think nowadays conda uses mamba under the hood.
- deleted 2y ago[deleted]
- waterslidelovr 2y ago[flagged]
- skeledrew 2y agoI've been using conda+poetry as my goto combo for years now and it's served me very well. Sure it's said poetry doesn't follow certain standards, but it just abstracts away so much that I don't see myself needing anything else really. If I want to share a non-poetry artifact, it's a simple `poetry build` to create a wheel. I did come across a weird dependency confusion issue recently when I tried using it on a server setup with piku though, but haven't gotten around to find what exactly is happening. Also don't think I'll ever give up on YAML for it's easy read+write, and I love the simplicity of fire for CLI things. And pathlib.Path().glob() for directory listing.
- Kevcmk 2y agoI agree with most of this article but the "don't use poetry" comment is ridiculous. Pdm could be great with time but suggesting it is premature.
- diimdeep 2y agoSlapped together gossip turned cargo cult use this practices.