17 ms·

Hypermodern Python

• cs702 6y ago
  It's evident from reading the OP and previous similar posts on HN that many developers find it difficult to specify and replicate deterministic Python environments for their applications. Personally, I have found it best to use (a) a virtualenv or conda environment, with (b) a requirements file that specifies fixed version numbers for packages (e.g., `pandas==1.0.3`). Only very rarely have I run into issues doing this; it works quite well for me. -- That said, from a security standpoint, I'm not sure it's a good idea to run a script downloaded from the web, without verification, on your local command line: curl https://pyenv.run | bash If that URL ever gets hijacked, you would be running malicious code. At a minimum, you may want to take a look at the script before running it, or otherwise verify that you're downloading what you actually want to run.
  • hojijoji 6y ago
    Don't you get a bunch of incompatible packages when you restrict to specific fixed version numbers without indication? I guess this is only helpful if you don't plan to reuse your code in another project.
    • cs702 6y ago
      Good question. The approach I proposed works for production applications that require an easy-to-replicate, deterministic environment, but I wouldn't recommend it if you're trying to build, say, Python packages or frameworks meant to be used in diverse environments.
  • tyrion 6y ago
    I completely agree. I have no idea why many people seem to be fan of this `curl x | bash` approach. For pyenv all is needed is to clone their git repository and add a couple of lines to your .bashrc.
    • BiteCode_dev 6y ago
      Honestly, the best approach should be "sudo apt get pyenv", but it's such a pain to package a program as a deb I undertand many people don't bother. Beside, curl works on all linux AND on mac. Saves time.
    • dahfizz 6y ago
      I think `curl | bash` is treated unfairly. Whether you `git clone` or `curl` a script, you are fundamentally doing the same thing: downloading and executing code from the internet. `git clone` just feels safer because it is hiding that fact under layers of abstraction. If I want to run pip, I need to trust PYPA. It's their code I want to run, and I need to download it one way or another. If I don't trust them to keep their domain secure, I don't see why I would trust them to keep their github repo secure. And the whole point of pip is to download code from PYPI and run it. pip, git, curl|bash, all do the same exact thing in this case. curl|bash just smells funny because it makes it more plainly obvious what is going on.
      • dtech 6y ago
        git clone will not execute arbitrary code from the internet without inspection.
        • hagy 6y ago
          I assume the cloned git repo includes arbitrary code that will be executed with the same security profile as a downloaded shell script.
        • dahfizz 6y ago
          If you read the source of all the programs you run, more power to you. Most people don't.
  • jzzskijj 6y ago
    Exactly. The WHOIS information of pyenv.run doesn't reveal anything who is controlling the domain. And the "more information" link https://raw.githubusercontent.com/pyenv/pyenv-installer https://raw.githubusercontent.com/pyenv/pyenv-installer in that script points to "400: Invalid request" result.
  • thefunnyman 6y ago
    Perhaps this is just my own inexperience showing but I haven’t ever had an issue using venv (which is included in python3 now) and a requirements.txt file.
• euler_angles 6y ago
  One thing that I haven't been able to figure out with the 20 minutes or so of reading about poetry that I just did -- does poetry support editable installs akin to pip install -e .?
  • npage97 6y ago
    So you shouldn't need to use pip install -e . as poetry already configures your package to be installed as editable. For adding a separate package in your filesystem, I believe using https://python-poetry.org/docs/dependency-specification/#path-dependencies https://python-poetry.org/docs/dependency-specification/#pat... is what you are looking for ``` poetry add ../path/to/package ```
  • arthmer 6y ago
    yes, thanks to PEP-517 and PEP-518, it just needs to have a build section, https://python-poetry.org/docs/pyproject/#poetry-and-pep-517 https://python-poetry.org/docs/pyproject/#poetry-and-pep-517
• i3winner 6y ago
  I think poetry is great and I hope it becomes more widespread.
  • matanrubin 6y ago
    Totally agree. Poetry is a joy to work with
• seemslegit 6y ago
  > This article series is a guide to modern Python tooling with a focus on simplicity and minimalism No it isn't, it introduces at least two external tools of questionable necessity - pyenv and poetry when python comes with venv and pip natively.
  • jacobush 6y ago
    Yeah, but pip and venv don’t leave me impressed, so please elaborate why those are better.
    • baq 6y ago
      they're bundled. unless you're on some flavor of ubuntu where they aren't bundled with the system python. then you're in python packaging hell.
    • maest 6y ago
      I have yet to find venv or pip lacking in anything I do - I think the onus is on external tools to prove their worth.
    • seemslegit 6y ago
      Whether they are better or not is up to you to decide, but "focus on simplicity and minimalism" is not what's going on here.
    • smcl 6y ago
      I think that given pip and venv are bundled with Python, then anyone who wants to introduce new dependencies to replace them needs to justify themselves. I haven't used either pyenv or poetry so I have no opinion on them.
      • dhagz 6y ago
        The benefit of pyenv seems clear to me - it's a simpler way to have multiple Python installs and manage which one is your "default" Python at a project, directory, user, system, and global level (they can even all be different). Poetry...not so much. requirements.txt is good enough for me most of the time.
• russellallen 6y ago
  Hypermodern Python seems to have a lot more steps to it than Ye Olde Python, where you put a hello.py file somewhere and did 'python hello.py'. Yes, yes, I know. But...
  • lifeisstillgood 6y ago
    Its not the language python, its the ... how to build a project that others can use. Its the scaffolding that 'just' knowing how to program does not teach you but working in a real environment forces you to learn. Just skim-read but it seems to cover the sensible parts - all of this applies if you are writing 100,000 lines of code or 3 lines of hello.py I am trying to write thedevmanual.com which is basically all of that - what it takes to run in real life. it is of course opinionated, and in lockdown :-(
    • deleted 6y ago
      [deleted]
  • luch 6y ago
    I genuinely thought that was a satire post about over-engineering Python projects
    • ra 6y ago
      I think it's a very enthusiastic post. Anything can be over engineered - and generally young enthusiastic programmers are keen to learn about the options.
    • abledon 6y ago
      id be tempted to clone the structure of the posts, and replace the images with some stock photos of supermodels lying down on modernist furniture with cobras and other snakes wrapped around them or lying on the floor nearby.
• manojlds 6y ago
  Ok talks about some alternatives like Poetry and pyenv, which have their own issues. pyenv is a poor cousin of rbenv, has such a poor cli api and a leaky abstraction over virtualenv (with its plugin for virtualenv)
  • hprotagonist 6y ago
    it’s not an abstraction over virtualenv at all; pyenv installs python interpreters, and those interpreters should use -m venv instead of virtualenv anyway. did you mean pipenv instead? we suck at naming things (and pipenv has its share of issues, no arguments there).
    • manojlds 6y ago
      No I meant the pyenv virtualenv plugin. Have been using it for a long time that forgot that there's a plugin involved.
      • hprotagonist 6y ago
        oh! Yeah, I don't use it at all. pyenv-installer drags that plugin in, and i cleverly fail to source it since it's totally useless for my purposes.
• languagehacker 6y ago
  Great read! I see a few approaches I'd be interested in adopting. Nox sounds particularly useful for tooling consistency across development environments. It's worth noting that this a rather opinionated toolset, and that we shouldn't mistake opinionated for hypermodern. You could replace poetry for pipenv (particularly now that it's being maintained again...), and I tend to prefer unittest to pytest. The further into it you get, the more opinionated it is -- for instance, code coverage support and CI/CD is not one-size-fits-all, but the choices of CodeCov and GitHub Actions are nicely illustrative.
  • kingosticks 6y ago
    I often end up frustrated when jumping back from pytest to old projects that are still using unitest I really like pytests's fixtures and it's caplog stuff. Out of interest, what do you prefer in unittest?
  • fastball 6y ago
    Please no pipenv. A horribly managed project with worse design choices than Poetry. And a confusing name to boot (it makes it sound like a clean bridge between pip and pyenv, but it's definitely not). On top of everything you can throw in Ken Reitz's ego. Literally the only reason pipenv gained traction is because there are still an unnerving number of people in the Python community that think Ken Reitz shits gold, when he is clearly more of a one-hit-wonder (love requests btw). Even the testimonials are self-important. > Pipenv is finally an abstraction meant to engage the mind instead of merely the filesystem. Like, really? I want to interpret that as a joke, but I don't think it is. Poetry should be the future of Python dependency management. I'd like to forget pipenv ever happened. Unfortunately since PyPA took pipenv under its wing, that might not happen.
• tgb 6y ago
  I have to say, at first blush I would not choose click over argparse. I do have to look up the docs of argparse every time I use it, but I like that it's just gives me the args and lets me structure the program flow how I want to, which I think is more natural. And then I can do things like import a big package (pandas) only after parsing the args, which is highly convenient to users that want to check the argument options without a five second lag.
  • empthought 6y ago
    I agree; click's approach is like some kind of old-school 4GL that tries to automatically create GUI elements from your database tables, except mapping CLI to functions in your modules. People should be putting enough thought into their CLI that Click doesn't really help them much.
  • i3winner 6y ago
    I concur. I've found argparse to be more straightforward and also has the benefit of being part of python itself.
  • kortex 6y ago
    Click has one feature which is huge for me: parsing env variables following some pattern eg MYAPP_FOO. For lazy import - You can do it with click, too, just put your imports in your entrypoint functions. Click isn't minimalistic though. Nor is poetry or pyenv. Seems to be the main complaints about the article.
    • jabwork 6y ago
      Configargparse is a great drop in replacement for argparse that I use to handle bridging env vars with cli flags https://pypi.org/project/ConfigArgParse/ https://pypi.org/project/ConfigArgParse/
      • kortex 6y ago
        YES I need this in my life! Tried searching for such a thing but I guess had the wrong keywords.
• darkerside 6y ago
  Wouldn't hyper modern Python rely on containerization for isolation and portability? Poetry and pyenv seem like incremental improvements rather than a qualitative leap forward.
  • jacobush 6y ago
    Heh, but that's one modern thing I don't like. Someone complained about click adding to startup time, running a new container is certainly "modern" but tiring.
    • darkerside 6y ago
      It's certainly overkill for a one man side project. But if you have a team and deploy to clustered servers, virtual environments sure start looking like a horse and buggy.
• woile 6y ago
  Great resource, I'd add for the typing section, the usage of `py.typed` file. It'd be worth sharing the alternative minimalistic approach ``` python3 -m venv .venv . .venv/bin/activate pip install -e . ``` But is good for people who are willing to install those tools to simplify their work. I use pyenv and poetry when I can.
• jalons 6y ago
  This helps highlights the main issue I have with python today, and that's running python apps. Having to ship a compiler to a host|container to build python with pyenv, needing all kinds of development headers like libffi for poetry. The hoopla around poetry init (or pipenv's equivalent) to get a deterministic result of the environment and all packages. Or you use requirements files, and don't get deterministic results. Or you use effectively an entire operating system on top of your OS to use a conda derivative. And we still haven't executed a lick of python. Then there's the rigmarole around getting one of these environments to play nice with cron, having to manipulate your PATH so you can manipulate your PATH further to make the call. It's really gotten me started questioning assumptions on what language "quick wins" should be written in.
  • squaresmile 6y ago
    I probably haven't bought into all of poetry yet but for deployment, I have been using "poetry export" to get the pinned requirements.txt, commit it to the repo and install to a virtualenv. A bit of work to keep it in sync with the poetry dependency file but that's ok. For PATH with cron or others, I use the full path to the virtualenv such as /path/to/project/.venv/bin/python. The path can be extracted by "which" or "Get-Command" when the venv is active. Using a python version different from the system python version is probably the messiest part but well, targeting 3.6 is alright. I do agree it could be better and it's not quite as streamlined as other ecosystems.
    • BiteCode_dev 6y ago
      Honestly, pip freeze includes the whole content of a venv site-packages, and the exact versions. For most projects, that's equivalent to all the dependancies recursively pins with peotry, although you don't have the clean pyproject-dev-prod/lock file separation. So a huge number of cases can be handled with just that. It will be "reproducible" enought for a lot of people.
      • squaresmile 6y ago
        > although you don't have the clean pyproject-dev-prod/lock file separation That's why I use "poetry export -f requirements.txt > requirements.txt" instead of pip freeze. It only exports prod requirements from the poettry lock file.
• dilandau 6y ago
  This post is a snapshot of one webdev's toolbox. It certainly is indicative of webdev's predilection for increasingly complex and opaque tooling, mixed-markup formats, and over-engineering (if it can even be called engineering). You don't really need any of that stuff. That whole tooling ecosystem undergoes a rewrite every couple years, anyways, so I predict this post will not age well. Just stick to setup.py, pip, and virtualenv. It's sufficient into all thy needs.
  • raverbashing 6y ago
    This The post wants to sell the point that "poetry is like npm" as if it was an advantage. It isn't. (Not that npm is bad per se) Can't wait for left-pad python the way things are going...
  • carapace 6y ago
    I was with you up until your last sentence. pip and virtualenv are hacky kludges that grew legs and lungs, that's how old python's issues around deployment are, there's an era before pip and virtualenv. There should be an abstraction for it (deployment, versions, import hacking) like what pathlib did for file path manipulation. We have a classic lava-flow pattern. http://mikehadlow.blogspot.com/2014/12/the-lava-layer-anti-pattern.html http://mikehadlow.blogspot.com/2014/12/the-lava-layer-anti-p...
    • raverbashing 6y ago
      What makes you think the newer tools are better than pip and virtualenv? (They most likely use some of those in the background) Edit: the link you quoted is very relevant > We developers should recognise that we suffer from a number of quite harmful pathologies when dealing with legacy code: > We are highly (and often overly) critical of older patterns and technologies. > We think that the current shiny best way is the end of history; that it will never be superseded or seen to be suspect with hindsight
      • carapace 6y ago
        > What makes you think the newer tools are better than pip and virtualenv? I don't. Heck, maybe they are. I don't use them. My point is that rather than being the beneficiary of Python's excellent abstraction powers, distribution et. al. is and has been a mess for decades. Here's a history of the sordid mess starting from 1998: "The distutils-sig dicussion list was created to discuss the development of distutils." https://www.pypa.io/en/latest/history/ https://www.pypa.io/en/latest/history/ FWIW, Since pip and virtualenv became relatively stable and "blessed" and PyPI has matured the only new thing I've tried is Anaconda. Unfortunately my main Python project right now uses Tkinter and the folks who make Anaconda have a circular dependency in their build system such that you need python to build Freetype so their Python/Tkinter/TCL/Tk has gruesomely-bad support for fonts, so my project looks like a potato. https://github.com/ContinuumIO/anaconda-issues/issues/6833 https://github.com/ContinuumIO/anaconda-issues/issues/6833 Someone has put in a PR to hopefully fix it, and-- ah! --it seems to have gotten some attention earlier today: https://github.com/conda-forge/tk-feedstock/pull/40 https://github.com/conda-forge/tk-feedstock/pull/40 Fingers crossed for great good! Anyhow, things are rough all over, eh?
• BiteCode_dev 6y ago
  I don't see what's modern about it. Those tools have been existing for years, and as always with Python, they serve only a subset of the Python users. Not that it's a bad article. If you don't know about them, give it a read, it's good to know it exists and the post is well written. But don't think it's the ultimate stack or whatever. And if you learn Python, you should always strive to master the basics first: - installing python.org for windows and mac, or apt/yum on Linux (those ones are tricky, it's more than just the "python" package to get pip and venv) - don't try to get the latest and greatest version (which is 3.8, or 3.9 alpha today). 3.6 is great already and is wildly available. I personnally strive for 3.7 now, but I'm happy with 3.6 if I don't have to use asyncio. So get the most modern version that is easy to install for you. - be comfortable with "-m", the "py" command on windows, "pip" (-m, --user, install, install -r, freeze), and "-m venv". - make sure you understand what the PYTHON PATH is and how it works regarding the import system - know how to configure your favorite editor to work with those Once you have those basics, you can go and try whatever have you: peotry, pyenv, pew, virtualenvwrapper, pipenv and so on. It will be easy because you will have a solid understanding of where it comes from and how it plugs in into the ecosystem. You will be able to chose if they are worth it or not for you. And more importantly, when you will have to move away from them (because of work, because they don't suit you, because they are abandonned...), you will always be able to fall back to basics. But franckly, there are so many things that you should be learning before those extras: pdb, black, pylint, zipapps... Or even concepts like generators, decorators, unpacking, etc. They all have a better value for the time you spend on them than learning a new "modern" stack. Not that it's not useful to have a modern stack. I update my stack all the time. I'm currently using pip + pew + doit + nox + pytest, and I'm experimenting with dephell. But it's my full time job. I'm an expert. Most Python devs have a much limited amount of resource to spent in learning stuff. They have deadlines, and other stuff to care about than Python. I know because I go from company to administrations to students to train them, and it's always the same: they chose Python because it's an efficient use of their time.
  • greenshackle2 6y ago
    Context is important. Developing applications vs libraries vs scripts is different, developing internal company software vs external software is different. If you're shipping a package on pypi, you will want to develop and automatically test using multiple Python versions and maybe a range of versions of your dependencies. Having good tools to manage multiple sets of Python versions and dependencies and parallel automated testing is a godsend. If you're writing an internal or web application that will run only in a single well-known environment, pip + venv + pytest might well be good enough.
• Kovah 6y ago
  I am not into Python, but the article lost me before it even started: you are required to install a bunch of compiler tools on your device to be able to proceed. Excuse me? Is this really what hypermodern Python looks like?
  • tyrion 6y ago
    Those are needed to compile different versions of Python with pyenv. Usually distributions bundle one or two specific versions of Python. Pyenv makes it super easy to install and use all the versions of Python that you want. Even though for most people it might be enough to just use whatever version of Python comes installed with your system, for a team it might be important that everyone has the exact same version. Moreover, pyenv-virtualenv makes it painless to use virtualenvs and so I recommend you give pyenv a try even if you do not need additional Python versions.
    • simiones 6y ago
      Again, do these tools really require a C compiler toolchain? If so, they are completely out of the question - they add much more complexity then they could possibly fix. Why not just install the required version of Python, maybe from a 3rd party repo if not available in the main repos? Why would I ever want to get a compiler toolchain to get my Python interpreter?
      • Chiron1991 6y ago
        > Why not just install the required version of Python, maybe from a 3rd party repo That's the thing. pyenv just downloads the source code of the specified Python version, and then compiles it on your machine. They did that to be agnostic of the OS you're running on.
        • simiones 6y ago
          But they are not agnostic of the OS. In fact, they have a huge dependency on the C toolchain, which I have to fix apparently.
• nojito 6y ago
  I’ve moved away from requests to httpx.
  • darkteflon 6y ago
    I’ve had my eye on httpx for a while. Could you elaborate on your reasons for changing and your experience with it?
    • nojito 6y ago
      Primarily async and it works extremely well alongside FastAPI.
    • andrewshadura 6y ago
      HTTP/2.
  • tomrod 6y ago
    Why?
    • ies7 6y ago
      I guess because of async.
• fabioz 6y ago
  I guess that there are many tools for environments now... my personal choice is using conda (I actually start with miniconda and download what I need from conda-forge). It provides the needed isolation, you can get lots of packages without having to compile from conda-forge and it still works with pip for the odd cases where a given package is not directly available (as a bonus, it works for other native toolchains, not only Python, which is a huge plus for me). As a note, I personally use it with `conda devenv` to be able to replicate installs for a given project -- i.e.: https://conda-devenv.readthedocs.io/en/latest/ https://conda-devenv.readthedocs.io/en/latest/.
  • tyrion 6y ago
    The only thing I never understood about conda, is why it does not support packages from pypi. I mean, the Python community has a standard repository for packages (pypi). You cannot even name a library $something if $something is not available on pypi. Why would I use something hosted by a private company that does not even interface with the rest of the community? This is not an attack on conda. I just cannot understand its rationale.
    • fabioz 6y ago
      The real problem (which is solved by conda) is that pypi doesn't solve dealing with non-python packages well (say, compile scipy and make all related native packages communicate well). This is a huge issue for people dealing with scientific packages (and the main pain point which pypi being just python-focused on installing on site-packages doesn't solve well enough IMHO). Also, while it was initially done by a private company, I'd say it's definitely a community effort right now (it's also the reason I tend to use https://conda-forge.org/ https://conda-forge.org/, which is community driven and not anaconda). As for dealing with pypi, it does integrate well enough for me (given that you can just pip install packages on the python for which conda is managing the env), but yes, the other way around isn't true (conda solves a bigger problem than pypi up to the point that it's possible to even have non-python tools available -- one real use case example I have here is having innosetup binaries as a tool in the PATH in some conda env for doing builds). Note: I don't have any affiliation with any of that, these are just my preferences for managing python envs (when I'm developing pydevd, which is the debugger engine used in pydev/pycharm/vscode, many times I need to reproduce some weird env and before using conda that was pretty annoying).
• king_magic 6y ago
  This feels like what will happen when the Node.js world gets bored with JavaScript and sets their sights on Python.
  • jennasys 6y ago
    Yes, this! I'm not against the language evolving, but why do so many people seem to want to turn it into JavaScript? I've been using Python for the last 10 years, and when learning it went out of my way to make sure I was using the idioms of the language and not just use it as a different language with a Python syntax.
• ra 6y ago
  Hypermodern? is that like the bleeding edge?
• totalperspectiv 6y ago
  I think that at this point all the ceremony around setting up and deploying a python project outweighs it's 'easy to read and use' aspects. Unless there is a library you can't live without or rewrite, it seems like a language with better tooling and real benefits of a type system is a better choice. If you're mainstream: Go or Java. If you're edgy: Nim, Scala, or Crystal All of those have much more sane type, build, and packaging systems. @perl-people, was this a solved problem when Perl was big? Or is python walking the same roads?
  • tinco 6y ago
    It most definitely was not a solved problem when Perl was big, as far as I know Ruby is the language that finally solved it with bundler, which was released after Rails was, so that's like 2007 or something. The recipe for the 'solved' deployment: - a compiler that ships with a build tool (so building is homogenous in the community) - a centralized or at least uniformly accessible package repository (so dependency acquisition is homogenous in the community) - a common file format for describing dependencies (so dependency resolution is homogenous in the community) - a common file format for locking dependency versions (so deployment can be done reliably without vendoring dependencies) - optional but very nice: a tool for managing compiler versions so it's easy to switch/upgrade projects Any programming language that has all of these boxes ticked is a modern programming language in my book. As far as I know Ruby is the first that ticked all of them, but other ones I've used that have this: Node.js, Go, Rust, Haskell, Python (though it's a bit messy). I'm pretty sure C# checks them as well nowadays, but I haven't used it in over 5 years so I'm not sure. Same for Java.
    • sansnomme 6y ago
      Somebody needs to tell the Racket people about this. Dependency management in Racket is still C-style no-version-pinning-anything-goes. The third party dependency managers (see below) are all primitive and do not support multiple versions of the same libraries which is a standard feature in Go/Node.js/Rust/Java class loaders https://github.com/Bogdanp/racksnaps https://github.com/Bogdanp/racksnaps
    • 6y ago
• mark_l_watson 6y ago
  Poetry looks good, I haven't used it. It reminds me of the way Quicklisp really revolutionized how to maintain and use packages in Common Lisp. I don't do too much Python now, but I bookmarked this article to possibly use as a future reference.
• albi_lander 6y ago
  > This article series is a guide to modern Python tooling with a focus on simplicity and minimalism I'm not sure using pyenv + poetry + click qualifies as a minimalist setup. I have never had to use pyenv as I figured out it was more robust to install the python versions directly from python.org and create the appropriate symlinks. Then using the venv module which has been included in python since 3.3. The argparser module of python is not that bad once you are used to it. I'm not saying the tools and libs mentioned in the article shouldn't be used, it's just that they're not mandatory for whoever wants to stay close to the bare minimum.
  • wiremine 6y ago
    Once you get in the habit, using pyenv/pipenv (in my case) is incredibly quick an easy. I use it for almost everything Python related I touch, except for quick scripts of the REPL.
  • skrtskrt 6y ago
    I would use pyenv locally so I can easily switch versions but dev and prod is a Docker container with one python version, then poetry install the virtual environment. You don’t even really need to install into a venv, since you’re already in an isolated container, but going straight for a venv is kind of a reflex for most Python devs
• meddlepal 6y ago
  This is why I gave up on Python.
  • panarky 6y ago
    What do you use now, and how is it better?
    • meddlepal 6y ago
      I work mostly in Go but personal stuff is a mix of Go, Java, or Kotlin. Go's only flaw is the module system sucks. Java is rock solid and rolls up nicely into a fat jar.
• kissgyorgy 6y ago
  If it would be really hypermodern, it would use httpx instead of requests: https://github.com/encode/httpx https://github.com/encode/httpx
• nemetroid 6y ago
  I have a really high usefulness threshold for adding external dependencies to Python projects. If you can get away with never getting into the virtual environment mess, distribution/installation/development becomes _so_ much simpler. Of course, sometimes you can't avoid external dependencies (personally, this often involves pandas). But the standard library gets you really far. And even though urllib.request is clunky, I will only use Requests if something else already is forcing me to add external dependencies.
  • dahfizz 6y ago
    I definitely agree for production-grade applications. Most of the time I'm using python, though, its to script a task I need to do or throw together a small app on a raspberry pi at home. In those cases, I have my own "standard library" of packages I always have installed system-wide, like requests. Unless I am making a "real" application, I do my best to avoid virtualenvs altogether.
    • BiteCode_dev 6y ago
      You don't have to. Use zipapps, this allows you to create a project with as many deps as you want, but use it as if it was a single python module. It's an official feature (https://docs.python.org/fr/3/library/zipapp.html https://docs.python.org/fr/3/library/zipapp.html) that lets Python execute an entire project bundled into a zip file. If you don't want to create it by hand, use shiv: https://pypi.org/project/shiv/ https://pypi.org/project/shiv/ It's the best of both worlds.
      • dahfizz 6y ago
        That's really cool! I can't believe I've never come across this, thanks for sharing!
        • BiteCode_dev 6y ago
          You are not alone. __main__.py in a zip has been supported litterally since Python 2.6 but very few people knew about it. Eventually in 3.5, the zipapp module has been released to make the feature more discoverable, but again it has been mostly ignored. I think the reason is that something like shiv was missing: it streamlines the process, shows a finish product instead of just telling people "all the thing they can do", and the automatic unzipping solved plenty of problems that you had with alternative like Pex, espacially on windows or with static resources. So now you have it. Spread the word.
• cpburns2009 6y ago
  All of these virtual environment managers over complicate dependencies with Python. Using a simple virtual environment is quite easy. All you need is Python, its built-in venv module, setuptools and pip. I've found pip-tools helps with generating/merging comprehensive requirements.txt files.
• neurostimulant 6y ago
  What's the benefit of using pyenv? I've been using `python -m venv .venv` to create virtualenv within the project directory for a while now, and VS Code recognize the .venv directory as python virtualenv and apply it to the workspace automatically. So far it's been quite painless for me.
  • jhrmnn 6y ago
    They serve a different purpose. Pyenv enables you to maintain several Python versions. You can that use the venv module of each respective Python to create a virtual environment with a Python at a given version.
    • neurostimulant 6y ago
      I see, I think I confuse it with pyenv-virtualenv. I never need to use multiple versions of python so I never try it so far. I always install the latest version globally and use python 3 docker image as the base deployment image.
      • thijsvandien 6y ago
        If you have multiple projects, it may be undesirable that you're forced to have them all on the same Python version. Of course it helps that Python has fairly good backward compatibility, and even using a newer version to develop and test than the one you'll deploy with is not the end of the world, but pyenv is easy enough that I see no reason not to do it properly.
  • shadycuz 6y ago
    Pyenv is for installing and managing python versions on a machine. I don't think it makes sense to compare to virtualenv as that doesn't install python only copies it from the system into the current directory. You could use both tools.
  • richardkmichael 6y ago
    If you want to use different versions of Python itself, without installing them as different named binaries or using shell aliasing, etc. Helpful to install Python 2.7, Python 3.x, 3.y, etc. and invoke each as simply `python ...`.
• thmzlt 6y ago
  I wrote a tutorial the other day on using Nix to handle the Python package situation. No need to install Python or even touch pip: https://thomazleite.com/posts/development-with-nix-python/ https://thomazleite.com/posts/development-with-nix-python/
  • zcoyle 6y ago
    There is also the wonderful poetry2nix project: https://github.com/nix-community/poetry2nix https://github.com/nix-community/poetry2nix
  • silviogutierrez 6y ago
    +1 for nix. Nothing comes close. Pain to first learn it though. But now you have truly deterministic builds. Well, almost. Don't forget to pin channels.
• tempotemporary 6y ago
  Want a real hyper modern Python? Try Haskell. (As someone who spent 10y with Python)
  • ca_parody 6y ago
    Nim feels more like hyper modern python to me. Haskell is an entirely different paradigm.
• wingi 6y ago
  This is hypermodern, but a security nightmare. curl someting | bash
  • kevsim 6y ago
    This pops up occasionally on HN but I don't get it. Are you reading every line of every installation script you run normally? What about the things they download and execute? Are you never running a binary installer where you don't have access to the code? Genuinely curious what the nightmare is.
• deegles 6y ago
  This is great! Is there something like this for Typescript?
• crad 6y ago
  This is so painful to me. So much of a reinventing the wheel for things that are not broken. :(
• gbasin 6y ago
  Excited for 2030 when this is reduced to one click
• triztian 6y ago
  This is exactly what I needed as a somewhat on/off longtime user of Python. It basically boils down to using pyenv, poetry and setting up the pyproject.toml and project src layout. I read the comments before the article and was expecting a MUCH more complicated content and intricate bash, tool or config setups but it seems straightforward. I feel everything else is part of getting started into professional software development (git, modern OS, env vars setup, bash or shell scripts, etc) which is it's own right complicated for newcomers. I think deployment and packaging vary a lot depending on the target deploy env which is why I'm fine with it being left out of the article.
• andrewshadura 6y ago
  With setup.cfg supporting nearly everything setup.py needed to include, I fail to see a reason to use poetry's own configuration file.
• ggregoire 6y ago
  Reading this article (which asks me to install 18 apt packages on my system) and the top comments here (complaints about developing, running and deploying apps, replicating deterministic Python environments and so on) makes me wonder why Python developers don't use Docker.
• maliker 6y ago
  My team has struggled a lot with how to do python deployment. We know we can isolate the interpreter and modules via venv/docker/etc. and get repeatable and reliable deployments. However, because we're a utility module, we like to allow users to import our code freely, and that's a lot harder when that code is isolated and bundled with very specific requirements. Seems like the only perfect solution is to just support every conceivable version of python and our required modules. Which is of course very hard. It would probably require greatly reducing our usage of tensorflow and some other packages which change a lot and are trickier to use on Windows. Good old dependency hell.
• orsenthil 6y ago
  The problem with this article is, it will take me a day or probably more to get started with my application. A newcomer will give up before the "hello world" program.
  • wegs 6y ago
    I don't think this is for a newcomer. Newcomer, here you go: print("Hello World!") This is for someone who has done Python for many years, but might have fallen behind on the latest and greatest trends. It collects a series of promising, bleeding-edge tools and gives an overview of what makes them clever and how to use them.
    • airstrike 6y ago
      > It collects a series of promising, bleeding-edge tools and gives an overview of what makes them clever and how to use them. "Promising" being the operative word here
• mrlonglong 6y ago
  I loved the wonderful Victorian drawings of air-fishes with people in them. Who did those and where can I find out more about these?
  • imadethis 6y ago
    Google Reverse Image search gave me https://publicdomainreview.org/collection/leaving-the-opera-in-the-year-2000 https://publicdomainreview.org/collection/leaving-the-opera-...
• sabujp 6y ago
  s/hypermodern/annotations
• icedchai 6y ago
  Seems overly complicated. I'll just stick with pip and virtualenv for now.
• Uninen 6y ago
  Sad thing is `poetry init --no-interaction` fails if your name happens to have non-ascii characters :(
• davv 6y ago
  hm