5 ms·

Super true. One of the best tests of this is setting up a new laptop. Some of the best experiences are when you get a new laptop, and just clone the codebase an

by cglan 2y ago

Super true. One of the best tests of this is setting up a new laptop. Some of the best experiences are when you get a new laptop, and just clone the codebase and everything works as it did before, no special magic. Golang with vendored dependencies seems to be wonderful for this but I've had relatively decent experiences with newer java projects. My worst experiences universally have always been python projects. I don't think I've had a single time where I cloned a python project and had it just work. Beyond just the code, I've had lots of mixed experiences with CI/CD being smooth. I unfortunately don't think I've been in a single shop where deployments or ci have been a good experience. They often feel very fragile and undocumented and hard for newcomers.

sameoldtune 2y ago
Relevant xkcd https://xkcd.com/1987/ https://xkcd.com/1987/
sethammons 2y ago
I have a couple decades of experience and have ridden a small start up through public and I have worked intimately with 6 companies. I know about taking a product and ultra scaling it in both technical and organizational scale. I will never recommend Python outside of a small team. It is organizational molasses. My current company has multiple teams striving to keep our Python tech stack serving our growing technical and organizational scale. I have fixed this in two companies in no small part with migrating to Go. I am on my third.
- aaronblohowiak 2y ago
  The hoops that people go through to solve this sometimes creates something even more complex and not great, like forcing all development into a docker container.. Ever try conda though? I’ve had moderate success with pipenv, but tbh I don’t love it as it hides too many things when installing a package fails.
  - imp0cat 2y ago
    I quite like docker for local development. Docker and docker compose do make it incredibly easy to start everything that's required for local development and testing. Your service A needs B and C? Grab those images of B and C and run them all on your machine. The only limitation is the amount of RAM you have available locally. And if you think a bit about your Dockerfiles (ie. have the layers set-up to take advantage of caching, have icecc+ccache mounts for c++ projects to distribute compilation and cache results, have mounts for apt or other package manager cache downloaded packages that you use) the local image rebuilds can be quite fast. Those are the little tricks to make your life with docker less miserable.
    - aaronblohowiak 2y ago
      Yea all of that is stuff I do not want to bother with. At all. “It’s nice if you take on maintenance burden of a bunch of additional moving parts” is the opposite of what I want. If you have to support a diverse set of languages / runtimes / environments and you deploy using containers, maybe it makes sense, but that seems like a use of the complexity budget I’d rather spend on… something else
      - adamc 2y ago
        In big organizations, it solves a lot of problems. Docker isn't perfect (hence interest and growth of Nix), but in day-to-day use it's fairly replicable.
  - bunderbunder 2y ago
    Doing a Python development environment inside of Docker can get particularly obnoxious in the long run because there are approximately a zillion ways that a base image upgrade can break things by changing something about the system Python packages. (And by the time it happens you might have a real mess on your hands because by that point the dev container's dockerfile has quite possibly grown into an undocumented spaghetti tangle of band-aids as a result of every dev on the team tweaking things in whatever way seemed to make the most sense at the time without a whole lot of regard for the end-to-end cohesiveness of the situation.) The standard advice in the Python community is "never trust the system Python", but tools like pyenv that we have for protecting ourselves from the operating system aren't always straightforward to get working sensibly inside of a container. It seems like it should be easy, but I've seen people get it wrong far more often than I've seen them get it right. A big part of the problem is that the Python community has developed an extremely severe case of TMTOWTDI when it comes to dependency management, packaging and deployment. It's led to a situation where, if you're just googling around for problem solutions in an ad-hoc manner, you're likely to end up with a horrible chimera of different philosophies of how to do Python devops, and they won't necessarily mesh well together.
humanfromearth9 2y ago
You may want to consider using Nix, with nix flakes.
- bryanlarsen 2y ago
  How much of that just hides complexity? I remember back in the day hiding a large amount of complexity behind vagrant. A new dev could get up and running quickly with "install vagrant; vagrant up", but that was hiding a lot of complexity behind a very leaky abstraction.
deleted 2y ago
[deleted]
senko 2y ago
> My worst experiences universally have always been python projects. I don't think I've had a single time where I cloned a python project and had it just work. I'm curios if you can spot a pattern in the platform (win/osx/linux), type of project, or is it all over the place? My own experience with Python boils down to creating a virtualenv, installing the deps, setting up configuration (or just copying it from somewhere) and creating a database, and I'm off to the races. The only exception in recent memory was when a project had two dozen microservices, half of the codebase was on private package repository, and we used Poetry. The combo required somewhat more involved setup. That said, IIRC all the projects had fully pinned package versions (package==x.y.z). In contrast, every time I touch something in JS land I get the same experience you described for Python. On one project we literally copied node_modules across machines (including servers), because it was unbounded amount of time trying to do a full reinstall. Anecdotally, amount of churn in JS is much higher, and the maintenance load increases proportionally. Usually it's something like: - have a project in JS with some dependency X that's no longer on the bleeding edge, but works nice - want to depend on a new package Y for some new feature - the new package Y depends on a library Z that's higher than what the other dependency (X) can work with - try to update the original dependency (X) - wailing, gnashing of teeth, and considering the switch to agriculture instead In my experience, if you're not closely tracking the bleeding edge, upgrading packages and updating your code accordingly, your JS developer experience will be abysmal. Agree on the CD part, especially the fragility and more manual work than if the deploy is some manually driven (semi-)automated process.
- imp0cat 2y ago
  You can get the same JS/node_modules experience with Python, just use pdm. ;)
pphysch 2y ago
> My worst experiences universally have always been python projects. I don't think I've had a single time where I cloned a python project and had it just work. I got a new Chromebook from work, and had VSCode+Docker running an existing Postgres+Django+etc dev environment in literally 15 minutes. I was shocked. Devcontainers are magic, and poor Python DX is a skill issue.
- porridgeraisin 2y ago
  > Poor Python DX is a skill issue Oh yes, the language whose ecosystem only hears about backwards compatibility in their own death marches? Not their problem. It's the developers, it's _their_ problem. Not the standard library which _removes_ packages, breaking code which I recently cloned. See "imp". And not the next python version, which throws a syntax error on bare excepts, breaking old code for absolutely zero benefit beyond pretending to be a linter.
AlienRobot 2y ago
I love Python but it always amazes me how hard it is for it to just... work. So there is virtualenv, built in, but... if there is a venv directory, Python doesn't just use it. Like you have app.py, and you python app.py, that doesn't run it with the venv python. This leads to all sorts of problems with scripts that assume they're running under venv. Which means you probably want to write a script that sources venv just so you don't forget, but if you place it in the same directory you may forget you need to call the script, so you probably want to add an extra directory to hide all the python code so you only see the shell script that you need to run to properly setup the environment to run the python code. Or just use an IDE. Just "pip install." But pip isn't installed and ensure pip doesn't work? What do I even do then? I recall downloading a project that required a library that wasn't available for the newest version of python, so when you tried to install the requirements pip wouldn't find it. I discovered this, naturally, because I updated my operating system so the python version changed which means the project that used to work stopped working! What is the solution for installing multiple python versions side by side? Hint: it's not an official project by the Python organization but something you can find on github.
- adammarples 2y ago
  My recent workflow is to use a great program called mise. You have a config file in your directory and hey presto, python venvs work, they install themselves if they don't already exist, and it will install the exact version of python you specify in your config. On top of that is will set environment variables for you and unload them when you change directory. If you combine this with uv (just tell mise you want uv installed in the config) you can run uv pip sync and instantly reflect any changes in your requirements file directly into your venv very quickly.
- tmnvix 2y ago
  For the past 4-5 years this is what has worked exceptionally well for me: - pyenv for installing multiple versions of python on my machine - direnv for managing environments (env variables, python version, and virtual environment) - pip for installing dependencies (pinning versions and only referencing primary packages in requirements.txt - none of their dependencies) This makes everything extremely easy to work with. When I cd into a project directory direnv loads everything necessary for that environment. Each project directory has a .env and a .envrc file. The .envrc looks something like this: layout python ~/.pyenv/versions/3.11.0/bin/python3 dotenv .env Absolutely no headaches working on dozens of local python projects.
nzach 2y ago
> My worst experiences universally have always been python projects. Do you mind sharing why do you think this happens ? Although I never worked professionally with python, this sentiment matches with my experiences as a user. So I don't have a lot of context why this is the case. Some siblings in this thread provided some explanations that mostly boils down to 'bad tooling' in one form or another. But this doesn't feel right. In my opinion if it was just bad tooling this problem would be solved by now.
ronakjain90 2y ago
Every time I setup a JS project which is older than a few years, it's 1. Extremely difficult to setup the code base, because of dependency spaghetti 2. Lot of breaking changes across different libraries, making maintenance not so easy. Easiest projects to maintain were written on Go, Java, Ruby,

by cglan 2y ago

sameoldtune 2y ago

Relevant xkcd https://xkcd.com/1987/ https://xkcd.com/1987/

sethammons 2y ago

I have a couple decades of experience and have ridden a small start up through public and I have worked intimately with 6 companies. I know about taking a product and ultra scaling it in both technical and organizational scale. I will never recommend Python outside of a small team. It is organizational molasses. My current company has multiple teams striving to keep our Python tech stack serving our growing technical and organizational scale. I have fixed this in two companies in no small part with migrating to Go. I am on my third.

aaronblohowiak 2y ago
The hoops that people go through to solve this sometimes creates something even more complex and not great, like forcing all development into a docker container.. Ever try conda though? I’ve had moderate success with pipenv, but tbh I don’t love it as it hides too many things when installing a package fails.
- imp0cat 2y ago
  I quite like docker for local development. Docker and docker compose do make it incredibly easy to start everything that's required for local development and testing. Your service A needs B and C? Grab those images of B and C and run them all on your machine. The only limitation is the amount of RAM you have available locally. And if you think a bit about your Dockerfiles (ie. have the layers set-up to take advantage of caching, have icecc+ccache mounts for c++ projects to distribute compilation and cache results, have mounts for apt or other package manager cache downloaded packages that you use) the local image rebuilds can be quite fast. Those are the little tricks to make your life with docker less miserable.
  - aaronblohowiak 2y ago
    Yea all of that is stuff I do not want to bother with. At all. “It’s nice if you take on maintenance burden of a bunch of additional moving parts” is the opposite of what I want. If you have to support a diverse set of languages / runtimes / environments and you deploy using containers, maybe it makes sense, but that seems like a use of the complexity budget I’d rather spend on… something else
    - adamc 2y ago
      In big organizations, it solves a lot of problems. Docker isn't perfect (hence interest and growth of Nix), but in day-to-day use it's fairly replicable.
- bunderbunder 2y ago
  Doing a Python development environment inside of Docker can get particularly obnoxious in the long run because there are approximately a zillion ways that a base image upgrade can break things by changing something about the system Python packages. (And by the time it happens you might have a real mess on your hands because by that point the dev container's dockerfile has quite possibly grown into an undocumented spaghetti tangle of band-aids as a result of every dev on the team tweaking things in whatever way seemed to make the most sense at the time without a whole lot of regard for the end-to-end cohesiveness of the situation.) The standard advice in the Python community is "never trust the system Python", but tools like pyenv that we have for protecting ourselves from the operating system aren't always straightforward to get working sensibly inside of a container. It seems like it should be easy, but I've seen people get it wrong far more often than I've seen them get it right. A big part of the problem is that the Python community has developed an extremely severe case of TMTOWTDI when it comes to dependency management, packaging and deployment. It's led to a situation where, if you're just googling around for problem solutions in an ad-hoc manner, you're likely to end up with a horrible chimera of different philosophies of how to do Python devops, and they won't necessarily mesh well together.

humanfromearth9 2y ago

You may want to consider using Nix, with nix flakes.

bryanlarsen 2y ago
How much of that just hides complexity? I remember back in the day hiding a large amount of complexity behind vagrant. A new dev could get up and running quickly with "install vagrant; vagrant up", but that was hiding a lot of complexity behind a very leaky abstraction.

deleted 2y ago

[deleted]

senko 2y ago

> My worst experiences universally have always been python projects. I don't think I've had a single time where I cloned a python project and had it just work. I'm curios if you can spot a pattern in the platform (win/osx/linux), type of project, or is it all over the place? My own experience with Python boils down to creating a virtualenv, installing the deps, setting up configuration (or just copying it from somewhere) and creating a database, and I'm off to the races. The only exception in recent memory was when a project had two dozen microservices, half of the codebase was on private package repository, and we used Poetry. The combo required somewhat more involved setup. That said, IIRC all the projects had fully pinned package versions (package==x.y.z). In contrast, every time I touch something in JS land I get the same experience you described for Python. On one project we literally copied node_modules across machines (including servers), because it was unbounded amount of time trying to do a full reinstall. Anecdotally, amount of churn in JS is much higher, and the maintenance load increases proportionally. Usually it's something like: - have a project in JS with some dependency X that's no longer on the bleeding edge, but works nice - want to depend on a new package Y for some new feature - the new package Y depends on a library Z that's higher than what the other dependency (X) can work with - try to update the original dependency (X) - wailing, gnashing of teeth, and considering the switch to agriculture instead In my experience, if you're not closely tracking the bleeding edge, upgrading packages and updating your code accordingly, your JS developer experience will be abysmal. Agree on the CD part, especially the fragility and more manual work than if the deploy is some manually driven (semi-)automated process.

imp0cat 2y ago
You can get the same JS/node_modules experience with Python, just use pdm. ;)

pphysch 2y ago

> My worst experiences universally have always been python projects. I don't think I've had a single time where I cloned a python project and had it just work. I got a new Chromebook from work, and had VSCode+Docker running an existing Postgres+Django+etc dev environment in literally 15 minutes. I was shocked. Devcontainers are magic, and poor Python DX is a skill issue.

porridgeraisin 2y ago
> Poor Python DX is a skill issue Oh yes, the language whose ecosystem only hears about backwards compatibility in their own death marches? Not their problem. It's the developers, it's _their_ problem. Not the standard library which _removes_ packages, breaking code which I recently cloned. See "imp". And not the next python version, which throws a syntax error on bare excepts, breaking old code for absolutely zero benefit beyond pretending to be a linter.

AlienRobot 2y ago

I love Python but it always amazes me how hard it is for it to just... work. So there is virtualenv, built in, but... if there is a venv directory, Python doesn't just use it. Like you have app.py, and you python app.py, that doesn't run it with the venv python. This leads to all sorts of problems with scripts that assume they're running under venv. Which means you probably want to write a script that sources venv just so you don't forget, but if you place it in the same directory you may forget you need to call the script, so you probably want to add an extra directory to hide all the python code so you only see the shell script that you need to run to properly setup the environment to run the python code. Or just use an IDE. Just "pip install." But pip isn't installed and ensure pip doesn't work? What do I even do then? I recall downloading a project that required a library that wasn't available for the newest version of python, so when you tried to install the requirements pip wouldn't find it. I discovered this, naturally, because I updated my operating system so the python version changed which means the project that used to work stopped working! What is the solution for installing multiple python versions side by side? Hint: it's not an official project by the Python organization but something you can find on github.

adammarples 2y ago
My recent workflow is to use a great program called mise. You have a config file in your directory and hey presto, python venvs work, they install themselves if they don't already exist, and it will install the exact version of python you specify in your config. On top of that is will set environment variables for you and unload them when you change directory. If you combine this with uv (just tell mise you want uv installed in the config) you can run uv pip sync and instantly reflect any changes in your requirements file directly into your venv very quickly.
tmnvix 2y ago
For the past 4-5 years this is what has worked exceptionally well for me: - pyenv for installing multiple versions of python on my machine - direnv for managing environments (env variables, python version, and virtual environment) - pip for installing dependencies (pinning versions and only referencing primary packages in requirements.txt - none of their dependencies) This makes everything extremely easy to work with. When I cd into a project directory direnv loads everything necessary for that environment. Each project directory has a .env and a .envrc file. The .envrc looks something like this: layout python ~/.pyenv/versions/3.11.0/bin/python3 dotenv .env Absolutely no headaches working on dozens of local python projects.

nzach 2y ago

> My worst experiences universally have always been python projects. Do you mind sharing why do you think this happens ? Although I never worked professionally with python, this sentiment matches with my experiences as a user. So I don't have a lot of context why this is the case. Some siblings in this thread provided some explanations that mostly boils down to 'bad tooling' in one form or another. But this doesn't feel right. In my opinion if it was just bad tooling this problem would be solved by now.

ronakjain90 2y ago

Every time I setup a JS project which is older than a few years, it's 1. Extremely difficult to setup the code base, because of dependency spaghetti 2. Lot of breaking changes across different libraries, making maintenance not so easy. Easiest projects to maintain were written on Go, Java, Ruby,