5 ms·

Maybe this is the best thing ever, but the documentation doesn't answer one simple question: What problem is it solving? It is a configuration management tool,

Maybe this is the best thing ever, but the documentation doesn't answer one simple question: What problem is it solving? It is a configuration management tool, like Ansible? Is it meant for running one-off commands across the infrastructure, like Salt? It says it integrates with Terraform, so it's not a provisioning tool... What does it do different (and presumably better) than other tools? The Getting Started guide doesn't cover this. The FAQ doesn't cover this, and the Docs doesn't have an Introductory section to cover this. It's disheartening to find a potentially interesting project, but not really know what it does and how it might fit in your workflow.

paulddraper 2y ago
It's like Ansible. That's what I discovered by reading the homepage.
- emacsen 2y ago
  Digging in the docs, it uses words like "inventories" and "operations" which indeed look like a configuration management system, much like Ansible, it's agent-less. And that's cool- Ansible is a bit of an oddball system, but then I'm still left wondering, why is this better, or why it is better for the author at least? I've used cfengine, Puppet, Chef, bcfg2 (briefly) and ansible. I want to know what makes this tool different and better. :)
- mdaniel 2y ago
  I would disagree with that since ansible actually does two things simultaneously: cloud provisioning and local provisioning (and that "local" is actually hiding a 3rd axis, actual local, not just local to the managed instance, say for example if you needed the azure libraries or such, you can use a pre_tasks: block to create a virtualenv and install the deps locally before firing up the main workload) Reasonable people can 100% disagree about whether yaml is the correct packaging for those operations, and ansible is a bit too imperative for my liking, but as far as "I have one hammer..." it does all the things
  - activatedgeek 2y ago
    Any kind of provisioning doesn't seem too far a step though. It is just another "operation" with its own state management logic.
    - mdaniel 2y ago
      I mean, I hear you in that python is Turing complete so all things are possible through another level of indirection, but I didn't see one shred of amazon.aws.autoscaling_group anywhere in their docs so .. what, I write my own? If I was going to go through the trouble of writing custom shit for Yet Another Awesome Cloud Thingy I'd fire me
      - activatedgeek 2y ago
        The fact that Pyinfra does not currently support a feature which can be implemented using Pyinfra philosophy does not make it different than Ansible. I believe that was what the parent comment was about.
  - paulddraper 2y ago
    Ansible can technically do cloud and local provisioning, just as Terraform can technically do cloud and local. But practically, these tools have their areas of speciality.
    - mdaniel 2y ago
      > just as Terraform can technically do cloud and local. I feel as though we're splitting hairs here, given there is, to the best of my knowledge, no `resource remote_file make_sshd_config { inventory_host = "whatever" dest = "/etc/sshd_config" src = "./sshd_config.tmpl" vars = {...} }` in TF. There is template, and there is local_exec and the rest is a Simple Matter Of Programming :-/ I'm waiting patiently for someone to chime in "well, just spawn ansible in local_exec" as if they're missing the point
fermigier 2y ago
It's similar to Ansible, but uses Python as a declation language rather than YAML. It can also run one-off commands across the infrastructure (like Tentakel: https://pypi.org/project/tentakel/ https://pypi.org/project/tentakel/ ). I've been using Pyinfra for some time. It's good enough for me.
- js2 2y ago
  I've been using it as well with great success. A couple years ago I inherited about 100 Mac Pros that are part of $dayjob's CI infrastructure. They had been managed over the years using a combination of shell scripts, Chef, and manually via VNC. No two machines were alike. The Chef recipes had all bit-rotted and weren't usable and due to $reasons were based on an old version of Chef that $company was stuck on. So I looked around for alternatives, and being most comfortable in Python, I explored Ansible, Salt, and Pyinfra. Ansible seemed like the obvious choice, but it has very few playbooks/actions for macOS systems. I was going to have to write my own. As I dug into its documentation, I found it was taking me a long time to wrap my head around all that I needed to do and started to sour on its complexity. This is a matter of taste, but I just didn't find Ansible very welcoming. I wanted something simpler. I had previously used Fabric, so considered using it again. But Fabric offers too little (it's really not much more than parallel ssh-if you want idempotent operations you have to write that yourself), and I don't agree with the direction it took with version 2.x. Then I found Pyinfra. It took me less than 30 minutes to understand it in its entirety. It's conceptually simple: you have an inventory of machines that it connects to in parallel over ssh. You provide it with a deploy script that combines facts and operations. Pyinfra uses the deploy script to gather facts about each machine, then you use those facts to decide whether you need to perform any operations. It then performs those operations on each machine as needed. The inventory file, deploy script, facts, and operations are trivial to write for someone comfortable with Python. It's all Python with the facts and operations being decorated functions. There is no DSL to learn. (It comes with a bunch of pre-written facts and operations, but they are mostly for Linux systems. I had to mostly wrote my own for macOS, but I found them really easy to write.) I had it operational the same day I found it. I used it to successfully get all of the Mac Pros into consistent state: things like system settings, installing Xcode, automating installs of brew packages all at the same version, installing JVMs, updating and upgrading macOS, installing Sentinel One, etc. I've been very happy with it, even contributing a few PRs to fix small bugs and contribute minor functionality.
koko-blat 2y ago
it's bc it solves all problem ever exist.
deleted 2y ago
[deleted]
dangoodmanUT 2y ago
People will know whether it solves their problem when they see it, no need to akchually the OP or maintainer
Fizzadar 2y ago
> It is a configuration management tool, like Ansible? Yes > Is it meant for running one-off commands across the infrastructure, like Salt? Also yes. > It says it integrates with Terraform, so it's not a provisioning tool... The TF integration is specifically to use TF as an inventory source - ie TF to create resources and pyinfra to then configure them. > What does it do different (and presumably better) than other tools? The homepage covers the highlights, I originally created pyinfra because debugging Ansible was complicated (no plain stderr as not "just" commands on the remote side) and slow, but things have evolved significantly since then. > The Getting Started guide doesn't cover this. The FAQ doesn't cover this, and the Docs doesn't have an Introductory section to cover this. Hugely appreciate this feedback, this is super helpful and something I will attempt to make clearer. --- Quick attempt at a better explanation: You write Python code that defines operations (either state "this apt package should be installed" or stateless "run this command"), provide an inventory of targets (SSH, local machine) and pyinfra executes it. Roughly sits where Ansible does for configuring servers, but also solves the case of "how do I run this command across my server fleet" (which I believe Ansible can also do).
- the_duke 2y ago
  > and slow, but things have evolved significantly since then Well, Ansible is still dog-slow, so that part has not evolved...
  - Fizzadar 2y ago
    Heh yeah this is very true, I updated the perf test repo earlier this year to confirm https://docs.pyinfra.com/en/next/performance.html https://docs.pyinfra.com/en/next/performance.html
- emacsen 2y ago
  I hope I wasn't coming across as too negative. I genuinely think that an introduction with a few user stories would go a long way!
remram 2y ago
> Great for ad-hoc command execution, service deployment, configuration management and more I found that pretty clear to be honest.
tryauuum 2y ago
Salt is not meant for running one-off commands. You can easily make sure state.apply is run for all of your infra several times an hour

paulddraper 2y ago

It's like Ansible. That's what I discovered by reading the homepage.

emacsen 2y ago
Digging in the docs, it uses words like "inventories" and "operations" which indeed look like a configuration management system, much like Ansible, it's agent-less. And that's cool- Ansible is a bit of an oddball system, but then I'm still left wondering, why is this better, or why it is better for the author at least? I've used cfengine, Puppet, Chef, bcfg2 (briefly) and ansible. I want to know what makes this tool different and better. :)
mdaniel 2y ago
I would disagree with that since ansible actually does two things simultaneously: cloud provisioning and local provisioning (and that "local" is actually hiding a 3rd axis, actual local, not just local to the managed instance, say for example if you needed the azure libraries or such, you can use a pre_tasks: block to create a virtualenv and install the deps locally before firing up the main workload) Reasonable people can 100% disagree about whether yaml is the correct packaging for those operations, and ansible is a bit too imperative for my liking, but as far as "I have one hammer..." it does all the things
- activatedgeek 2y ago
  Any kind of provisioning doesn't seem too far a step though. It is just another "operation" with its own state management logic.
  - mdaniel 2y ago
    I mean, I hear you in that python is Turing complete so all things are possible through another level of indirection, but I didn't see one shred of amazon.aws.autoscaling_group anywhere in their docs so .. what, I write my own? If I was going to go through the trouble of writing custom shit for Yet Another Awesome Cloud Thingy I'd fire me
    - activatedgeek 2y ago
      The fact that Pyinfra does not currently support a feature which can be implemented using Pyinfra philosophy does not make it different than Ansible. I believe that was what the parent comment was about.
- paulddraper 2y ago
  Ansible can technically do cloud and local provisioning, just as Terraform can technically do cloud and local. But practically, these tools have their areas of speciality.
  - mdaniel 2y ago
    > just as Terraform can technically do cloud and local. I feel as though we're splitting hairs here, given there is, to the best of my knowledge, no `resource remote_file make_sshd_config { inventory_host = "whatever" dest = "/etc/sshd_config" src = "./sshd_config.tmpl" vars = {...} }` in TF. There is template, and there is local_exec and the rest is a Simple Matter Of Programming :-/ I'm waiting patiently for someone to chime in "well, just spawn ansible in local_exec" as if they're missing the point

fermigier 2y ago

It's similar to Ansible, but uses Python as a declation language rather than YAML. It can also run one-off commands across the infrastructure (like Tentakel: https://pypi.org/project/tentakel/ https://pypi.org/project/tentakel/ ). I've been using Pyinfra for some time. It's good enough for me.

js2 2y ago
I've been using it as well with great success. A couple years ago I inherited about 100 Mac Pros that are part of $dayjob's CI infrastructure. They had been managed over the years using a combination of shell scripts, Chef, and manually via VNC. No two machines were alike. The Chef recipes had all bit-rotted and weren't usable and due to $reasons were based on an old version of Chef that $company was stuck on. So I looked around for alternatives, and being most comfortable in Python, I explored Ansible, Salt, and Pyinfra. Ansible seemed like the obvious choice, but it has very few playbooks/actions for macOS systems. I was going to have to write my own. As I dug into its documentation, I found it was taking me a long time to wrap my head around all that I needed to do and started to sour on its complexity. This is a matter of taste, but I just didn't find Ansible very welcoming. I wanted something simpler. I had previously used Fabric, so considered using it again. But Fabric offers too little (it's really not much more than parallel ssh-if you want idempotent operations you have to write that yourself), and I don't agree with the direction it took with version 2.x. Then I found Pyinfra. It took me less than 30 minutes to understand it in its entirety. It's conceptually simple: you have an inventory of machines that it connects to in parallel over ssh. You provide it with a deploy script that combines facts and operations. Pyinfra uses the deploy script to gather facts about each machine, then you use those facts to decide whether you need to perform any operations. It then performs those operations on each machine as needed. The inventory file, deploy script, facts, and operations are trivial to write for someone comfortable with Python. It's all Python with the facts and operations being decorated functions. There is no DSL to learn. (It comes with a bunch of pre-written facts and operations, but they are mostly for Linux systems. I had to mostly wrote my own for macOS, but I found them really easy to write.) I had it operational the same day I found it. I used it to successfully get all of the Mac Pros into consistent state: things like system settings, installing Xcode, automating installs of brew packages all at the same version, installing JVMs, updating and upgrading macOS, installing Sentinel One, etc. I've been very happy with it, even contributing a few PRs to fix small bugs and contribute minor functionality.

koko-blat 2y ago

it's bc it solves all problem ever exist.

deleted 2y ago

[deleted]

dangoodmanUT 2y ago

People will know whether it solves their problem when they see it, no need to akchually the OP or maintainer

Fizzadar 2y ago

> It is a configuration management tool, like Ansible? Yes > Is it meant for running one-off commands across the infrastructure, like Salt? Also yes. > It says it integrates with Terraform, so it's not a provisioning tool... The TF integration is specifically to use TF as an inventory source - ie TF to create resources and pyinfra to then configure them. > What does it do different (and presumably better) than other tools? The homepage covers the highlights, I originally created pyinfra because debugging Ansible was complicated (no plain stderr as not "just" commands on the remote side) and slow, but things have evolved significantly since then. > The Getting Started guide doesn't cover this. The FAQ doesn't cover this, and the Docs doesn't have an Introductory section to cover this. Hugely appreciate this feedback, this is super helpful and something I will attempt to make clearer. --- Quick attempt at a better explanation: You write Python code that defines operations (either state "this apt package should be installed" or stateless "run this command"), provide an inventory of targets (SSH, local machine) and pyinfra executes it. Roughly sits where Ansible does for configuring servers, but also solves the case of "how do I run this command across my server fleet" (which I believe Ansible can also do).

the_duke 2y ago
> and slow, but things have evolved significantly since then Well, Ansible is still dog-slow, so that part has not evolved...
- Fizzadar 2y ago
  Heh yeah this is very true, I updated the perf test repo earlier this year to confirm https://docs.pyinfra.com/en/next/performance.html https://docs.pyinfra.com/en/next/performance.html
emacsen 2y ago
I hope I wasn't coming across as too negative. I genuinely think that an introduction with a few user stories would go a long way!

remram 2y ago

> Great for ad-hoc command execution, service deployment, configuration management and more I found that pretty clear to be honest.

tryauuum 2y ago

Salt is not meant for running one-off commands. You can easily make sure state.apply is run for all of your infra several times an hour