7 ms·

Python seems like a really poor choice for infrastructure. - Python is not easy to build into portable binaries - The package ecosystem is very hard to use in

Python seems like a really poor choice for infrastructure. - Python is not easy to build into portable binaries - The package ecosystem is very hard to use in a reproducible way - The language is not truly typed - types add massive value for infrastructure and scripts because they are less likely to be unit-tested - The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested) Maybe I'm missing something? I don't know why I would want to introduce Python in this domain.

aprdm 2y ago
Maybe because Python is already in use by pretty much every company that makes money in this (and others) domain ? Some of what you mention looks like pebkac problems as well.
- mhh__ 2y ago
  Well so is pretty much any configuration language under the sun, and all the other options that aren't python.
Fizzadar 2y ago
Extremely aware of this (see pyinstaller attempt): https://github.com/pyinfra-dev/pyinfra/pull/768 https://github.com/pyinfra-dev/pyinfra/pull/768) I chose Python because it’s what I was writing all day back in 2015. Which makes me realise pyinfra is almost 10! Edit: I mostly write Go or YAML (k8s) these days but Python still makes an appearance from time to time (outside of pyinfra dev).
bborud 2y ago
Python is a nightmare when used for tooling. I’ve wasted so much time wrangling Python tooling for embedded development. Go would be a much better choice.
JoBrad 2y ago
What would you have used? All of your issues aside, Python is very approachable to people who are used to managing infrastructure but may not have a strong programming background.
benrutter 2y ago
I think Go would be a logically choice if you're being completely language agnostic, but most teams aren't. If teams are working exclusively in python already for web or data projects, there's a benefit to not introducing a new language just for architecture deployment if that's a small part of a teams function.
DandyDev 2y ago
Why is it important to be able to build into portable binaries? Pyinfra doesn't require running Python on the machines you manage. Pyinfra basically turns your Python code into shell commands which it runs over SSH. So only your development machine has to run Python. I think there is not a lot of overlap between people who need to automate infrastructure and people who don't know how to install Python on their development machine. As for your other comments regarding Python as a language: I mostly agree. I have stepped away from Python as a language to develop production software. In Python I miss the confidence I get from static typing. Having said that, for automating infrastructure, you're effectively comparing Pyinfra and Python to bash scripts and YAML (for things like Ansible), which are both orders of magnitude worse if you like static typing or any form of being able to verify what you wrote.
- yjftsjthsd-h 2y ago
  N=1 I'm capable of handling python on my dev/deploy box(es), but that doesn't mean it's not a pain. In my perfect world, ansible/puppet/chef/whatever would ship as a single static binary even when they mostly ran against remote SSH targets.
- BirAdam 2y ago
  Right, but this makes me wonder why I can’t just do: ssh user@host “echo ‘Hello World’” All these kinds of tools essentially just executing commands over SSH… I could just SSH.
- fire_lake 2y ago
  > So only your development machine has to run Python. If you have a team of developers and a CI process, then portability is important. There isn’t one development machine.
mixmastamyk 2y ago
Sounds like you'ved misjudged the use case. Tools like this do the deployment, they aren't generally deployed themselves. So a portable binary is not a requirement. Other points like let or types are not an impediment either, there are many quality tools available if you need them (ruff, pyflakes, mypy), and python has been doing this kind of work productively for thirty years now.
- posix_monad 2y ago
  > Tools like this do the deployment, they aren't generally deployed themselves. It will have to be executed on many different developer machines (or even your own machine several years in the future) so a simple, reproducible build process, including fetching pip dependencies, is critical.
  - mixmastamyk 2y ago
    Presumably the tool keeps backward compatibility, as did ansible or salt, so this doesn’t seem to be a real-world concern. Very few folks are doing nix-level stuff, yet the world marches on. There will likely be a security fix in it or a dep at a later point, so you wouldn’t want to use the exact same version anyway.
exceptione 2y ago
I 100% agree with your points. No type checking = no serious job. I have learned enough from Ansible to not ever touch that kind of stuff again. There has been a time that Python was a fringe language, only known by some hardcore nerds. I thought Joel Spolsky had once mentioned that having Python on your resume was a signal of a quality developer, someone who went off the beaten path. Times have changed. Python is now the MS Excel for developers. It shines for quick and dirty data mangling. Unfortunately, that is how a seizable portion of people approach software engineering. My theory is that for some having to do abstract thinking and perform a dry analysis beforehand is an impediment. They can only discover what they want while banging out something. They fix the runtime errors they could catch, and slap some more features on top. Types imply a kind of foresight, and that is what some people really have difficulty with. EDIT: Might sound negative, so I admit that the quick feedback cycle you can get from an interpreter language like php/python is a feature in itself.
dboreham 2y ago
Shiv is a decent solution for making a portable package. Single file that only depends on a recent system Python being installed.
IshKebab 2y ago
I agree, Python is a pretty bad choice for all the reasons you mentioned. That said I think there are precious few good alternatives. I've been using Deno a fair bit for "scripting" and it works pretty well, but I wish there were more options. Also I have to say if you are using a tool like this to manage thousands of machines you're absolutely doing it wrong. I don't even work in ops/infra but even I know that manually running commands on multiple machines via SSH is asking for trouble.
nijave 2y ago
A decent Python development tool chain handles most of that. Docker, pylint, black, type hints integrated with IDE/editor Admittedly some languages like Go do a better job integrating all this into the core of the language. However, Go doesn't tend to have as powerful of a stdlib so it tends to be a lot more verbose to achieve the same thing.
- fire_lake 2y ago
  Docker should not be required for day-to-day development.
  - nijave 2y ago
    Why?
    - fire_lake 2y ago
      It’s hard to install on some setups. It’s much slower than native. It’s admitting defeat.
mountainriver 2y ago
It’s better than Yaml or HCL though
goodpoint 2y ago
Python is an excellent choice. > - Python is not easy to build into portable binaries > - The package ecosystem is very hard to use in a reproducible way People use OS packages since 4 decades. > - The language is not truly typed The language IS strictly typed. > - types add massive value for infrastructure and scripts because they are less likely to be unit-tested 99% of errors in deployment are not solved by typing. > - The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested) If your logic is so complex that let/var makes a difference you should be not touching infra.
kortex 2y ago
This might have been true a few years ago, but these are all solved problems in 2024. > Python is not easy to build into portable binaries https://pex.readthedocs.io/en/v2.1.40/buildingpex.html https://pex.readthedocs.io/en/v2.1.40/buildingpex.html - The package ecosystem is very hard to use in a reproducible way pip, virtualenv, and requirements.in/txt is extremely reproducible. I will offer that it's not exactly idiot-proof yet and there are tons of stale tutorials out there > The language is not truly typed - types add massive value for infrastructure and scripts because they are less likely to be unit-tested Yes it is, if you want it to be. There's nothing stopping someone from using mypy, pyright, or other type tool on the strictest setting, and not passing builds unless you have 100% type coverage. > The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested) No, but you get ~95% of the safety guarantees by using immutable-esque objects like @dataclass(frozen=True), pydantic models with the same, or attrs/cattrs with similar setting.

aprdm 2y ago

Maybe because Python is already in use by pretty much every company that makes money in this (and others) domain ? Some of what you mention looks like pebkac problems as well.

mhh__ 2y ago
Well so is pretty much any configuration language under the sun, and all the other options that aren't python.

Fizzadar 2y ago

Extremely aware of this (see pyinstaller attempt): https://github.com/pyinfra-dev/pyinfra/pull/768 https://github.com/pyinfra-dev/pyinfra/pull/768) I chose Python because it’s what I was writing all day back in 2015. Which makes me realise pyinfra is almost 10! Edit: I mostly write Go or YAML (k8s) these days but Python still makes an appearance from time to time (outside of pyinfra dev).

bborud 2y ago

Python is a nightmare when used for tooling. I’ve wasted so much time wrangling Python tooling for embedded development. Go would be a much better choice.

JoBrad 2y ago

What would you have used? All of your issues aside, Python is very approachable to people who are used to managing infrastructure but may not have a strong programming background.

benrutter 2y ago

I think Go would be a logically choice if you're being completely language agnostic, but most teams aren't. If teams are working exclusively in python already for web or data projects, there's a benefit to not introducing a new language just for architecture deployment if that's a small part of a teams function.

DandyDev 2y ago

Why is it important to be able to build into portable binaries? Pyinfra doesn't require running Python on the machines you manage. Pyinfra basically turns your Python code into shell commands which it runs over SSH. So only your development machine has to run Python. I think there is not a lot of overlap between people who need to automate infrastructure and people who don't know how to install Python on their development machine. As for your other comments regarding Python as a language: I mostly agree. I have stepped away from Python as a language to develop production software. In Python I miss the confidence I get from static typing. Having said that, for automating infrastructure, you're effectively comparing Pyinfra and Python to bash scripts and YAML (for things like Ansible), which are both orders of magnitude worse if you like static typing or any form of being able to verify what you wrote.

yjftsjthsd-h 2y ago
N=1 I'm capable of handling python on my dev/deploy box(es), but that doesn't mean it's not a pain. In my perfect world, ansible/puppet/chef/whatever would ship as a single static binary even when they mostly ran against remote SSH targets.
BirAdam 2y ago
Right, but this makes me wonder why I can’t just do: ssh user@host “echo ‘Hello World’” All these kinds of tools essentially just executing commands over SSH… I could just SSH.
fire_lake 2y ago
> So only your development machine has to run Python. If you have a team of developers and a CI process, then portability is important. There isn’t one development machine.

mixmastamyk 2y ago

Sounds like you'ved misjudged the use case. Tools like this do the deployment, they aren't generally deployed themselves. So a portable binary is not a requirement. Other points like let or types are not an impediment either, there are many quality tools available if you need them (ruff, pyflakes, mypy), and python has been doing this kind of work productively for thirty years now.

posix_monad 2y ago
> Tools like this do the deployment, they aren't generally deployed themselves. It will have to be executed on many different developer machines (or even your own machine several years in the future) so a simple, reproducible build process, including fetching pip dependencies, is critical.
- mixmastamyk 2y ago
  Presumably the tool keeps backward compatibility, as did ansible or salt, so this doesn’t seem to be a real-world concern. Very few folks are doing nix-level stuff, yet the world marches on. There will likely be a security fix in it or a dep at a later point, so you wouldn’t want to use the exact same version anyway.

exceptione 2y ago

I 100% agree with your points. No type checking = no serious job. I have learned enough from Ansible to not ever touch that kind of stuff again. There has been a time that Python was a fringe language, only known by some hardcore nerds. I thought Joel Spolsky had once mentioned that having Python on your resume was a signal of a quality developer, someone who went off the beaten path. Times have changed. Python is now the MS Excel for developers. It shines for quick and dirty data mangling. Unfortunately, that is how a seizable portion of people approach software engineering. My theory is that for some having to do abstract thinking and perform a dry analysis beforehand is an impediment. They can only discover what they want while banging out something. They fix the runtime errors they could catch, and slap some more features on top. Types imply a kind of foresight, and that is what some people really have difficulty with. EDIT: Might sound negative, so I admit that the quick feedback cycle you can get from an interpreter language like php/python is a feature in itself.

dboreham 2y ago

Shiv is a decent solution for making a portable package. Single file that only depends on a recent system Python being installed.

IshKebab 2y ago

I agree, Python is a pretty bad choice for all the reasons you mentioned. That said I think there are precious few good alternatives. I've been using Deno a fair bit for "scripting" and it works pretty well, but I wish there were more options. Also I have to say if you are using a tool like this to manage thousands of machines you're absolutely doing it wrong. I don't even work in ops/infra but even I know that manually running commands on multiple machines via SSH is asking for trouble.

nijave 2y ago

A decent Python development tool chain handles most of that. Docker, pylint, black, type hints integrated with IDE/editor Admittedly some languages like Go do a better job integrating all this into the core of the language. However, Go doesn't tend to have as powerful of a stdlib so it tends to be a lot more verbose to achieve the same thing.

fire_lake 2y ago
Docker should not be required for day-to-day development.
- nijave 2y ago
  Why?
  - fire_lake 2y ago
    It’s hard to install on some setups. It’s much slower than native. It’s admitting defeat.

mountainriver 2y ago

It’s better than Yaml or HCL though

goodpoint 2y ago

Python is an excellent choice. > - Python is not easy to build into portable binaries > - The package ecosystem is very hard to use in a reproducible way People use OS packages since 4 decades. > - The language is not truly typed The language IS strictly typed. > - types add massive value for infrastructure and scripts because they are less likely to be unit-tested 99% of errors in deployment are not solved by typing. > - The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested) If your logic is so complex that let/var makes a difference you should be not touching infra.

kortex 2y ago

This might have been true a few years ago, but these are all solved problems in 2024. > Python is not easy to build into portable binaries https://pex.readthedocs.io/en/v2.1.40/buildingpex.html https://pex.readthedocs.io/en/v2.1.40/buildingpex.html - The package ecosystem is very hard to use in a reproducible way pip, virtualenv, and requirements.in/txt is extremely reproducible. I will offer that it's not exactly idiot-proof yet and there are tons of stale tutorials out there > The language is not truly typed - types add massive value for infrastructure and scripts because they are less likely to be unit-tested Yes it is, if you want it to be. There's nothing stopping someone from using mypy, pyright, or other type tool on the strictest setting, and not passing builds unless you have 100% type coverage. > The lack of a "let" or "var" keyword makes simple programming errors more likely (again, this code is less likely to be unit-tested) No, but you get ~95% of the safety guarantees by using immutable-esque objects like @dataclass(frozen=True), pydantic models with the same, or attrs/cattrs with similar setting.