18 ms·
Pyinfra – automate infrastructure super fast at scale
- totetsu 6y agoisn't there another python writable iac project?
- drcongo 6y agoThere was one by the creator of Ansible but sadly it never reached production status.
- mdellavo 6y agosaltstack, fabric
- eliaspro 6y ago- Pulumi [1], which supports amongst many other languages Python as well - SaltStack [2], which has a broad range of execution and state modules, uses by default a "Master/Minion" architecture, but can be used push-based through "salt-ssh" [3] as well - POP/Idem [4] - which originates in the concept of idempotent SaltStack states, but exposes this functionality as Python code and uses the POP paradigm [5] coined by SaltStack's founder Thomas Hatch. A lot of SaltStack itself will quite likely move towards this architecture in the foreseeable future as well [1] https://www.pulumi.com/docs/ https://www.pulumi.com/docs/ [2] https://github.com/saltstack/salt/ https://github.com/saltstack/salt/ [3] https://docs.saltstack.com/en/latest/topics/ssh/index.html https://docs.saltstack.com/en/latest/topics/ssh/index.html [4] https://gitlab.com/saltstack/pop/idem https://gitlab.com/saltstack/pop/idem [5] https://pop.readthedocs.io https://pop.readthedocs.io
- totetsu 6y agoThank you
- ghostwriter 6y agoWhat advantages does Pyinfra have over NixOps? https://github.com/NixOS/nixops https://github.com/NixOS/nixops
- jackhalford 6y ago> NixOps is a tool for deploying to NixOS machines in a network or the cloud seems like pyinfra is OS agnostic.
- ghostwriter 6y agoThe frontpage shows that it relies at least on apt-get: from pyinfra.modules import apt apt.packages( {'Install iftop'}, 'iftop', sudo=True, ) And search over docs doesn't produce results for Windows/MacOS * https://pyinfra.readthedocs.io/en/v0.14.5/search.html?q=windows https://pyinfra.readthedocs.io/en/v0.14.5/search.html?q=wind... * https://pyinfra.readthedocs.io/en/v0.14.5/search.html?q=macos# https://pyinfra.readthedocs.io/en/v0.14.5/search.html?q=maco...
- detaro 6y agoit supporting apt doesn't mean it "relies on apt". Looking at the list of options, it supports multiple package managers, including ones for Windows and MacOS.
- ghostwriter 6y agobut then, if one needs to use low-level primitives representing specific package managers, it's hardly aligned with a claim of "super fast infrastructure automation". With Nix I have exactly one package manager for any target platform.
- zoom6628 6y agoJust checked out the NixOps manual https://nixos.org/nixops/manual/ https://nixos.org/nixops/manual/ and it seems able to deploy to NixOS machines on any cloud or LAN platform. But i also do not find any mention of it being able to deploy non-NixOS servers.
- usrme 6y agoSorry if this seems willfully ignorant, but what advantage does this provide over just using something like Ansible?
- drcongo 6y agoFor me the answer is debuggability. Debugging Ansible can be incredibly painful at times.
- usrme 6y agoI guess I haven't used Ansible long enough to know such pain points. Would you elaborate with some examples?
- peterwwillis 6y agoThis is a bit like asking what the Marquis de Sade did that was so painful. I don't even know where to begin explaining how frustrating Ansible is. Over time they've compounded one bad design decision on top of another, while never adding any actually useful functionality to be able to troubleshoot or iterate on the many random failures. They also don't provide guidance on how "if you use feature X, features a, b, c, d, and e will not work correctly". Finding a working example of core functionality, like an AWS inventory plugin, requires you to dig through all of the code and scour the internet and experiment for a day before you have a simple working configuration. The more features you use, the more fragile and shitty it becomes, to the point that nobody wants to actually change any roles or playbooks because deployments might stop working and it'll take you two days to figure out how to make them work again in Ansible's bass-ackwards design. And of course, it's Python, so you have to teach all your users how to use virtualenvs, freeze deps, and run it with Docker, or you'll constantly hear "it didn't run correctly for me".
- godtoldmetodoit 6y agoI've been neck deep in Ansible for the past couple of months, and boy, I am not a fan. Documentation is middling at best imo and it just feels hacked together.
- Fizzadar 6y agoI wrote pyinfra! If anyone has questions/suggestions/issues I'm more than happy to answer :)
- 1337shadow 6y agoLooking great ! How is it executing remote commands ? Is it uploading a script like ansible, or just executing remote commands ? The latter seems to be the case glancing at the source code, nice one !
- Fizzadar 6y agoIt executes commands remotely, so no python/etc needed on the targets, just a shell!
- 1337shadow 6y agoWhat a huge improvement to Ansible ! Seems like a much more modern Ansible to me, more precise but more targeting Python users instead of trying to attract non-coders too, it's very promising. I wish I had found your project before I started my own too, it's shitty compared to yours but i'll swallow my shame and share it with you just in case you find an idea that you like yourlabs.io/oss/shlax
- pydry 6y agoCan/will it be adapted to handle provisioning in cloud environments? - e.g. set up EC2 instances or is it your expectation that it will work with something that will handle that part?
- Fizzadar 6y agoSo pyinfra itself won't do this - but because inventory/etc is written in Python it's possible to easily achieve this by using boto3 directly in the inventory file. The use of Python for the inventory + deploy code makes it possible to integrate pyinfra with almost any external tool, without specific support in pyinfra (that's the theory, at least!).
- deleted 6y ago[deleted]
- deleted 6y ago[deleted]
- whalesalad 6y agoThis sounds like Capistrano rewritten in Python?
- knodi 6y agoya fix pip and pipenv first. Not starting any more new python projects.
- throwaway894345 6y agoAnd deployment and crosscompilation (for native runtime dependencies) and performance, although those are probably not significant issues for this application.
- nickthemagicman 6y agoWhats your issue with it? Asking as a noob to python package management. I've always used venv and pip and didn't even know pipenv existed until I read your post. Should it be avoided?
- drcongo 6y agoPersonally I'd recommend Poetry [1] at this point for Python package management. 99% of the time it works perfectly and keeps out of your way. There's also a lot of good info in the comments here [2] [1] https://python-poetry.org https://python-poetry.org [2] https://news.ycombinator.com/item?id=23380113 https://news.ycombinator.com/item?id=23380113
- BiteCode_dev 6y agoThe author of Pyinfra is just a dev like you and me working on a side project. There is no relationship between pyinfra, pipenv and pip. There is not even a relationship between pip and pipenv, they are completly different teams. In fact, none of those projects are maintained by the Python core team or part of the Python project. Not even pip, which is separated from Python and provided using get-pip.py at install.
- eeZah7Ux 6y agoHow do you prevent human error from breaking production?
- VWWHFSfQ 6y agowhat kind of human error
- nerdponx 6y ago"Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?"
- 1337shadow 6y agoPlay your stuff on a staging server first, and if possible on a container or vm in the test pipeline of your infra code. And well, start with a backup and be careful when you're working in production but that's common sense right ?
- eeZah7Ux 6y agoStaging works well in my experience. How can you benefit from a test pipeline and a staging system with tools that bypass it and ssh right into the production systems?
- sandGorgon 6y agohave you looked at Pulumi ? https://www.pulumi.com/docs/reference/pkg/python/pulumi/ https://www.pulumi.com/docs/reference/pkg/python/pulumi/ https://www.pulumi.com/blog/programming-the-cloud-with-python/ https://www.pulumi.com/blog/programming-the-cloud-with-pytho...
- BiteCode_dev 6y agoIt seems more complementary than a competition, as Pulumi targets clouds, while pyinfra has a lot of tools for self hosting.