8 ms·

Hey all, I'm the creator/primary maintainer of pyinfra! Super excited (a little terrified) to see this on the frontpage, happy to answer any questions :) I als

• negus 2y ago
  As you can see here, the main question is what are the advantages over Ansible, a mature and the most popular agentless configuration management tool written in Python. So I propose putting this answer right to the landing page
  • Fizzadar 2y ago
    I think I tried to shy away from specifically being "Ansible does this bad so pyinfra does this" and instead focus on the features that differentiate like "Instant debugging with realtime stdin/stdout/stderr output (-vvv).". But it seems like that isn't enough and the landing page needs to be more explicit in comparison. Ty for the feedback!
    • erikbye 2y ago
      Is performance better than Ansible? I have used Ansible extensively and find it excruciatingly slow.
      • helsinki 2y ago
        Excruciatingly slow is an understatement :)
        • surfingdino 2y ago
          Why being slow is a bad thing? Ansible gives me a legitimate excuse to have proper lunch. ;-)
          • verdverm 2y ago
            You're supposed to be writing compilers during that time
      • Fizzadar 2y ago
        Yes! https://docs.pyinfra.com/en/next/performance.html https://docs.pyinfra.com/en/next/performance.html
        • mxuribe 2y ago
          Hi @Fizzadar and congrats on making this and getting it out the door; kudos! As you craft your "Why this and not Ansible" content, you might actually state clearly what you already noted on the Performance page, namely: "One of the reasons pyinfra was started was performance of agent-less tools at the time." If I read that, it'd instantly make me want to stick around and read some more, play with pyinfra, etc. BTW, i will be playing with it anyway, but just wanted to point out that you likely won;t need to start from scratch for copy (on a comparison or answering "Why this and not Ansible" content). Cheers!
    • gh02t 2y ago
      I applaud trying to be positive and focus on "this is what we do well," but yeah at least some explicit comparison would help. The copy right now is kind of assuming the reader already knows Ansible to compare against as a baseline. Which is probably fair for most people who find your project, but people who find your project are also probably not happy with Ansible and want to know if this addresses their pain points immediately. Is very interesting though, I think I'm gonna try it myself.
    • Anarch157a 2y ago
      I like Ansible, but that doesn't mean thera are no pain points. One of them is handling "if-this-then-that-else-that". Being purely declarative, Ansible is horrible at that. Pyinfra can be used in imperative mode, am I right? This would make the use of if-else a breeze, which would be a really good reason for me to to switch.
      • tryauuum 2y ago
        Ansible is declarative? In puppet and saltstack you can declare that a folder is empty and declare a specific file in this folder. The system's smart enough to delete all the files except the one. To achieve such feat in ansible is hard. Easiest way is to have two tasks, one deletes everything and second recreates your file. Doesn't feel very declarative Unrelated thing, they don't even try to be declarative in ansible E.g you can have a file with state "touch". It not a state if it updates each playbook run!
        • emmelaich 2y ago
          rsync covers this and should ne used instead of a lot of Ansible tasks
        • Anarch157a 2y ago
          You're confusing declarative with idempotent. Ansible is both, it won't change anything if the state is already what you declared. The nitpicked case you chose, you want the file to have the latest timestamp, this is a valid state to declare.
          • verdverm 2y ago
            The shell task breaks the declarative nature a bit, along with registering task results and then writing conditional whens based on them. Interpolating values based on the registered results does too imho
            • aflukasz 2y ago
              Plus, what your are declaring is often times not the state you desire but... action you want to take. Say you use `apt.name: [pkg1, pkg2]`, run it, then remove `pkg2` from the list. Running this again won't remove `pkg2` from your system. So it's declarative, but not necessarily on the optimal level all the times.
    • ShakataGaNai 2y ago
      That would be appreciated. I saw the homepage and my first thought was "Ansible is python. How are these things different?" Obviously pure python vs yaml is one thing. But beyond that it's not clear. Perhaps are specific use cases in your mind where one or the other is a better fit, and that would be helpful as well.
  • ransom1538 2y ago
    Stupid question. Ansible is mentioned TWO times in HN hiring thread, it will soon be zero. Isn't this dated tech? Ansible is good at patching servers (have cute names, make sure they are patched). Patching??? But, why not use containers? Aren't we moving away from Ansible/Puppet? I would much rather have AWS/CDK or pythoninfra like this if I was into pet machines.
    • jethro_tell 2y ago
      Someone somewhere has to set up machines so you can do containers. Doesn't name an they are pets.
      • ransom1538 2y ago
        You mean terraform?
        • jethro_tell 2y ago
          No, before terraform, someone has to boot a machine and slap an image on it or do an OS install, Register the host in some way and have it checking for the terraform. I use ansible for creating machine images or initial provisioning. (I don't run the ansible, someone racks the host, sets it's build state to install, and boots the host and it joins the appropriate cluster and people do container things. I don't necessarily know when my ansible runs against a host. I also have a pretty good stack of ansible playbooks that I use manually day to day for hardware validation for new server models and one off type stuff. But again, I never really know what I'm running against or have pet servers. A good chunk of hardware validation runs automatically if the boot target is set to hw-validate, but the whole point is that you are gonna find stuff that doesn't work with your standard process and either pass on it or adjust. I do run tf to provision cloud infra so its transparent to the devs, and, honestly, not sure how ansible is dated and tf is not, they are pretty much the same thing in a different coat. And honestly, generating thousands of lines of conflicting generic yaml isn't really much of an improvement over writing it once and running it automatically on 1000s of boxes.
• OJFord 2y ago
  Is it declarative? Obviously python isn't, but since it's not executed as a script but rather the module passed to pyinfra, it could be, and looks like maybe it is just registering work to (potentially) do on module load? If so, nice, shout about it more - it's my number one requirement of such a tool, why I think Terraform (or OpenTofu) is great and mostly everything else sucks, and I think it should be everyone's. It's just obviously (at least, once someone makes it available!) the correct paradigm for managing stateful resources and coping with drift.
  • Fizzadar 2y ago
    Yes... and no. It depends on the operation (the docs explicitly state if an operation is _not_ idempotent "stateless operation"). Operations are either: - state definitions, "ensure this apt package is installed" (apt.packages: https://docs.pyinfra.com/en/next/operations/apt.html#operations-apt-packages https://docs.pyinfra.com/en/next/operations/apt.html#operati...) - stateless, "run these shell commands" (server.shell: https://docs.pyinfra.com/en/next/operations/server.html#operations-server-shell https://docs.pyinfra.com/en/next/operations/server.html#oper...) Most operations are state definitions and much preferred, the stateless ones exist to satisfy edge cases where either the state-ful version isn't implemented or simply isn't possible.
    • tetha 2y ago
      Ah, so this is similar to the Terraform CDK approach? In Terraform CDK, you use a language like python to compute the set of resources and such you want to have, and then hand that over to the terraform core, which does the usual terraform song and dance to make it happen. This is actually interesting to me, because we struggle with even the simplest data transformations in ansible so much. Like, as soon as you start thinking about doing a simple list comprehension in python in jinja templating, lots and lots of pain starts. From there, we're really starting to think about templating the inventories in some way because it would be less painful.
      • Fizzadar 2y ago
        Interesting, not heard of CDK before! Kind of similar? As long as the language is Python I suppose! Would be possible to integrate with other languages too I guess, not something I’ve ever looked into though. Totally agree on templating which is why inventories have always been python code just as operations, giving maximum flexibility (with some complexity/type confusion drawbacks).
• InitEnabler 2y ago
  dang this exploded. I came across the project this morning when I was looking at a blog on how to implement a generic programming language to become a configuration language and it mentioned pyinfra. Glad this project is getting some exposure. :)
• esafak 2y ago
  How does it compare with pulumi?
  • mdaniel 2y ago
    Almost exactly as it compares to terraform, since both TF and Pulumi only get down into the shell of any provsioned virtual machine via "connect and run some shell, good luck". I'd guess it would also be horrifically painful to even do that in circumstances such as Auto Scaling Groups, where even TF and Pulumi don't know the actual IP or InstanceIds The way TF and Pulumi traditionally think about this problem would be to use cloud-init/ignition/Cloudformation Hooks to cause the machine to execute scripts upon itself. Ansible also has an approach do that via "ansible-pull" which one would use in a circumstance where the machine has no sshd nor SSM agent upon it but you still want some complex configuration management applied post-boot (or, actually even if they do have sshd/ssm but there are literally a hundred of them, since the machines doing the same operation to themselves is going to be much less error prone than trying to connect to each one of them and executing the same operations, regardless of the concurrency of any such CM tool)
• mrled 2y ago
  Oh man this is really cool. I have also written a Python infrastructure-as-code project (https://pages.micahrl.com/progfiguration/ https://pages.micahrl.com/progfiguration/), I really like the idea of using a programming language rather than a text document to define infrastructure. Yours looks very polished, and the built in support for testing in Docker is a brilliant idea.
• gnosek 2y ago
  [yet another reference to Ansible, sorry! :)] This looks like infinity times better than Ansible in some cases and somewhat worse in others (python.call every time I'd need to access a previous operation's result feels clunky, though I certainly understand why it works that way). Do you think it would be possible to use Ansible modules as pyinfra operations? As in, for example: - name: install foo apt: pkg: foo state: present could be available as: from pyinfra import ansible ansible(name='install foo').apt(pkg='foo', state='present') where the `ansible` function itself would know nothing about apt, just forward everything to the Ansible module. Note 1: I know pyinfra has a way to interface with apt, this is just an example :) Note 2: It's just my curiosity, my sysadmin days are long gone now.
  • Fizzadar 2y ago
    Definitely possible! Not familiar with the ansible Python API so partially guessing but the pyinfra op could yield a callback function that then calls ansible at execution time. Alternatively you could just yield ansible cli and execute from the local machine using the @local connector.
    • mdaniel 2y ago
      FWIW, ansible modules (all of them, to the best of my knowledge) operate via a stdin/stdout contract since that's the one universal api for "do this thing over (ssh|docker|ssm|local)". That's also why it supports writing plugins in any language (shell, compiled, python, etc) since `subprocess.Popen().communicate(b'{"do_awesome":true}')` works great DISCOVERING the available ansible actions is the JFC since, like all good things python, it depends on what's currently on the PYTHONPATH and what makes writing or using any such language-server some onoz And this wasn't what you asked, but ansible has a dedicated library for exec, since the normal `ansible` and `ansible-playbook` CLIs are really, really oriented toward interactive use: https://github.com/ansible/ansible-runner#readme https://github.com/ansible/ansible-runner#readme
• rbut 2y ago
  How does it compare to Fabric? At first glance it looks quite similar. All our scripts are written in Fabric, but Fabric appears to be somewhat abandoned and the latest version never reached full parity with v1. I'd be looking to try something new next time.
• linsomniac 2y ago
  What is different in v3? Didn't see it in the "Next" docs.
  • js2 2y ago
    Mostly this from the 3.x changelog: > pyinfra now executes operations at runtime, rather than pre-generating commands. Although the change isn't noticeable this fixes an entire class of bugs and confusion. See the limitations section in the v2 docs. All of those issues are now a thing of the past. https://github.com/pyinfra-dev/pyinfra/blob/3.x/CHANGELOG.md https://github.com/pyinfra-dev/pyinfra/blob/3.x/CHANGELOG.md
• orochimaaru 2y ago
  What’s the difference between pyinfra and fabric? Fabric seems to have overlaps especially for agentless execution.