15 ms·

Show HN: Aviary.sh – A tiny Bash alternative to Ansible

276 points by dxchester 6y ago

Hackbraten 6y ago
I’ve been waiting for a tool like this.
- dxchester 6y ago
  Here you go! Yeah, I've used a bunch of CM tools over the years, and it always seems like you're learning a DSL of one sort or another, just to have it translate into the bash that you already know.
  - cloin 6y ago
    And don't get me wrong, I love bash. I automated my first job using bash. But relying on bash just relegates aviary to a certain job description and thus just low value, domain specific tasks. Seems like a neat way for a linux admin to run a homelab, but not something that's easily adopted for larger teams working across OS, network platforms or integrating with external services.
    - chillfox 6y ago
      Not everything needs to be for large teams. There’s value in simple solutions for small teams.
  - cloin 6y ago
    YAML is a standard for so many different tools now that I don't think it's really considered a DSL.
ahnick 6y ago
If you need a simple way to inject secrets into your aviary variables you may find encpass.sh (https://github.com/plyint/encpass.sh https://github.com/plyint/encpass.sh) a convenient choice.
- logie17 6y ago
  That's a pretty cool project. It also is written in bash as well!
nyxtom 6y ago
Nice, I plan on using this
freedomben 6y ago
Really neat. I still need to evaluate it more, but this is the kind of thing I've looking for for small infrastucture that isn't likely to grow (like home services or small office, etc).
ansible 6y ago
I do something sort of similar, also using git to maintain the inventory of what to run on what machine. In my case, I'm using docker-compose.yml files for each machine, and then running: docker-compose pull docker-compose up -d We initially wanted to use Watchtower, but it didn't work well for us.
- yonixw 6y ago
  Nitpicking: Why pull separately? it will pull if no image stored locally or even if the `docker-compose.yml` changed.
  - randallsquared 6y ago
    Two reasons I've done this in my own Makefiles: predictable start times, and separation of failure-prone steps.
  - yjftsjthsd-h 6y ago
    Stale images can be a reason. If I specify `image: foo:latest`, then push new versions, it won't automatically pull new versions unless you do an explicit pull.
- xomodo 6y ago
  Try docker contexts: https://www.docker.com/blog/how-to-deploy-on-remote-docker-hosts-with-docker-compose/ https://www.docker.com/blog/how-to-deploy-on-remote-docker-h...
  - darkteflon 6y ago
    This is a great feature, introduced in Feb this year. It addressees a substantial pain point with docker-compose. Thanks for pointing this out.
    - ptrincr 6y ago
      Indeed this does look very useful. I had a quick glance over this, but for contexts which are defined as remote ssh hosts, I couldn't see where the images are obtained. Are they being built remotely on the host, or are they being downloaded from your local machine over a tunnel or via a docker registry?
arminiusreturns 6y ago
Now this is something I can get behind! Very cool work. I think bash gets a bit of a bad rap and it's very popular to bandwagon hate on it, but it's one of my favorite tools ever.
easterncalculus 6y ago
Very nice! Ansible can be a little heavyweight for some of the things that me and a lot of people do. I'll definitely be taking a look at this.
ggregoire 6y ago
Ansible is agentless tho. That's a pretty big difference.
- gjulianm 6y ago
  Yeah, it was the thing that made me choose it instead of other alternatives. I manage machines in heterogeneous client networks without direct Internet access most of the time and agents for managing infrastructure are not useful at all.
- peterwwillis 6y ago
  Yes, but it's also horrible to use. I would rather set up a Puppet agent once than struggle with Ansible for an eternity.
  - pricechild 6y ago
    Would I be right if I guessed you have a development background? For reference, I have an ops background.
    - jeltz 6y ago
      I have a dev background and think that Ansible is much nicer to work with than Chef and Puppet.
    - peterwwillis 6y ago
      Nope, I'm an ops guy. Been banging my head against these tools for 15 years.
xbryanx 6y ago
I'm not sure if I am understanding this (aviary.sh) correctly, but it looks like this requires an agent to run on the configured hosts. One of the things that I love about Ansible's model is that we never need an agent on the host. Once you configure something with Ansible, there's no artifact of the configuration left on the machine.
- pampa 6y ago
  That is not entirely true. Ansible needs python installed on the target host, and a lot of modules (plugins?) require additional python libraries on the target.
  - soco 6y ago
    It doesn't need any python. Ansible opens a regular ssh session and can run anything from it - including python if available.
    - dfinninger 6y ago
      Only with the “raw” module... which is what you use to ensure Python is installed on a machine managed by Ansible.
  - pnutjam 6y ago
    you can use the "raw" module without python.
  - cloin 6y ago
    If python is considered an agent, isn't bash now an agent?
  - divbzero 6y ago
    The av agent that GP refers to is a script that runs periodically on target hosts to pull the latest configuration using Git. Ansible has a Python dependency but no agent running on target hosts, instead relying on SSH to push configuration changes. Ideally, I’d like to see a configuration tool that uses Ansible’s SSH push model but without the Python dependency.
MINIMAN10000 6y ago
Unfortunately all I could think of was "that sounds cool but I'd want to stick to dash instead of bash" Also didn't realize it wasn't agentless which is pretty critical imo
- yjftsjthsd-h 6y ago
  > Unfortunately all I could think of was "that sounds cool but I'd want to stick to dash instead of bash" Well, POSIX sh, ideally. That way you get free portability to ~everything; it's not a great burden to install bash or dash on ex. NetBSD or AIX, but sh is as close to universally preinstalled as you're going to get. > Also didn't realize it wasn't agentless which is pretty critical imo Ouch, yeah that outright disqualifies it for me.
t0astbread 6y ago
This looks really effective! I do something similar to keep track of what's on my workstations: I just have one large idempotent shell script to set up all the software I use. To help myself not loose track of updating it I have a little helper script to detect packages that aren't tracked in the config. I've recently thought about breaking it up into smaller "module-style" scripts. Maybe this could be a good fit (with minor adoptions)!
dheera 6y ago
> Install from the command line, on a box to be managed by aviary.sh: It would be even cooler if I didn't need to do anything to do the box besides install openssh-server. I should be able to: $ sudo apt-get install aviary $ aviary install my_server 192.168.1.25 # a fresh ubuntu 18.04 install and everything else should just be magic from there.
tony 6y ago
If you like this - Fabric is awesome. https://www.fabfile.org/ https://www.fabfile.org/ Use python over SSH. Super fast to begin, if you can ssh into the server, you can run the equivalent of shell commands (subprocess calls in python) remotely. With python, you can abstract and reuse, with the scriptability of bash, but higher level niceties of the language, libraries and toolchain (linting, formatting, autocompletion, imports). Can it pull the latest version of your app, build it on your server? Yep. Restart daemons? Update libraries (npm, python, etc)? Update system packages? All work fine. Can it configure a vanilla server from scratch? Yes, it's helpful to bring in a higher level reusable library like https://github.com/fabtools/fabtools https://github.com/fabtools/fabtools helps with that. When does it hit limitations? More complex orchestration where you have multiple servers (and variations of them) with configuration to talk to each other over networks. Eventually you get to the kind of setup where having a tool like Ansible / Salt / Puppet / etc. makes sense. For a basic PHP/python/ruby/node site that's just running on a single cloud server? A declarative config manager would be overkill for me.
- linuxftw 6y ago
  > Use python over SSH. Super fast to begin, if you can ssh into the server, you can run the equivalent of shell commands (subprocess calls in python) remotely. This literally describes ansible. In fact, ansible one-ups this because you can specify raw commands if python is not installed remotely [1]. Ansible provides lots of modules, you don't have to use any of them. I have plenty of 1-off ansible playbooks that I don't care about idempotency that are just a bunch of 'cmd' statements. It's a very flexible tool. 1: https://docs.ansible.com/ansible/latest/modules/raw_module.html https://docs.ansible.com/ansible/latest/modules/raw_module.h...
  - maxmalysh 6y ago
    Not really. Ansible is a complex DSL based on YAML. Fabric is really simple and uses real Python.
    - llama052 6y ago
      Can't say I've ever really thought of YAML as complex. Especially when the Ansible modules are translating things so you can just define the action you want to take, and each module is very well documented.
q3k 6y ago
> bash is just fine (yes, it is) No, it's not. It's 2020, we need to stop it with the stringly-typed, footgun-filled madness that is bash scripting. We deserve better than this. Unless I'm missing something, this README example already contains a bug: template $(dirname $0)/motd.template > /etc/motd This will fail if the script path contains a space.
- montroser 6y ago
  I think you are forgetting that serious operators do not have spaces in their paths.
  - hk__2 6y ago
    *People who are used to work with bad tools don’t have spaces in their paths.
    - tenebrisalietum 6y ago
      "C:\Documents and Settings" still hurts after all these years.
      - linuxftw 6y ago
        "C:\Program Files(x86)". Who ok'd that?
        GrinningFool 6y ago
        I haven't looked in a while, but last I checked you could also still get to it via C:\PROGRA~1? (This was years ago, that might have been 'fixed'?)
        joombaga 6y ago
        These are called 8.3 filenames. They're a filesystem feature. They're still created by default in Windows 10 on NTFS, but can be disabled or stripped using `fsutil`.
salasrod 6y ago
> But it's too funky you say. Well, yes, bash can have its quirks. But we need some funk every now and then. Let's just embrace it! My god, no, stop. Funkyness in production equals outages.
- ironmagma 6y ago
  As bad as Ruby is, it’s still better than Bash and you can do all the same effortless execution of random commands in your script like you would in bash. If one can’t be bothered to write it in a real language like Rust or even Python, you could at least do Ruby.
  - cortesoft 6y ago
    What makes python more of a real language than ruby?
    - ironmagma 6y ago
      It’s more mainstream for tools like this.
      - rad_gruchalski 6y ago
        Please explain. Have you heard of chef? That is ruby and it can be beautiful. Chef > Ansible. Chef server is meh.
        ironmagma 6y ago
        The point wasn’t that Python is better than Ruby, they are about the same IMO. The point was they are both better than Bash, and Ruby actually makes it super convenient to write bash-like scripts, so there’s no excuse to not be using Ruby.
      - e12e 6y ago
        Eh.. Puppet? [ed: and chef..]
nodesocket 6y ago
Typically I use Packer + Terraform and it works generally well (if you going with the immutable servers philosophy). However, often times immutable servers is hard to actually do in practice. If Hashicorp added a new product similar to Ansible / Aviary.sh / Fabric I think it would complete the entire DevOps loop under a single ecosystem.
- raffraffraff 6y ago
  If you're sticking with hashicorp tools and use consul, then something like consul-template should work. Your packer builds would just have to copy consul-template configs and template files for the things you want to manage. Each template watches the consul kv store for matching keys, and rewrites the target config file using their values. You can instantly rewrite configs and execute follow up actions across your fleet by updating a consul key value.
  - nodesocket 6y ago
    Agree Consul is an elegant solution for managing files, what about running commands?
    - moondev 6y ago
      cloud-init?
    - johntash 6y ago
      I don't see it used very often, but Consul actually can execute remote commands. You do have to enable it in the config though. https://www.consul.io/docs/commands/exec https://www.consul.io/docs/commands/exec edit: Also, Nomad would be another choice if you're looking at hashi tools.
- llama052 6y ago
  You can hook ansible roles into your Packer process pretty easily. That's what I do at least.
trynewideas 6y ago
it requires an agent, so it's a tiny Bash alternative to Puppet, but that doesn't sound remotely as cool
- stevekemp 6y ago
  I mentioned it above already, but I've been experimenting with an agentless puppet-lite: https://github.com/skx/marionette/ https://github.com/skx/marionette/ Early days, but the primitives are enough to do useful things on my own systems..
stevekemp 6y ago
I've been working on a localhost-only system for automation myself, inspired by puppet but with far fewer modules: https://github.com/skx/marionette/ https://github.com/skx/marionette/ As of this week it can now pull down docker-images from dockhub, which was something I'd been needing to rebuild my own systems. Early days, and it is as much proof-of-concept as anything else, but it seems robust and reliable in my own use. Of course I need to add "written in go" to advertise properly!
- dastx 6y ago
  On first glance this looks awesome. Certainly will be better than the bunch of scripts I've written to provision my laptops.
  - stevekemp 6y ago
    Fingers crossed it will work for you, and others! It replaces a simpler alternative I wrote in the past which used ssh/scp to upload files and run commands: https://github.com/skx/deployr https://github.com/skx/deployr Right now I have a git-repository containing per-host config. So I can run something like this via cron: cd /srv/config/$(hostname --fqdn)/ marionette main.in I should document that, and the inclusion-facility explicitly. But otherwise feel free to file bug-reports/feature-requests if you have ideas for modules that would be useful.
- ahnick 6y ago
  Does this save time in writing your own personal automation or make that personal automation less error prone? How would you compare it to just writing your own bash scripts or using something like Bashible (https://github.com/mig1984/bashible https://github.com/mig1984/bashible)?
  - stevekemp 6y ago
    Making things repeatable and scripting will usually be a time-saver (unless you spend 100+ hours writing the automation and never reinstall your hosts!) For me it was a learning experience as much as anything else. I'd probably have stuck with ansible, puppet, or similar established project if I didn't want to learn/experiment.
lifeisstillgood 6y ago
I think there is an Genuine Enterprise Need explosion at certain scales. It is not box ticking exercises or empire building but sometime after (my guess) 1,000 units of anything (users, servers, customers, whatever) you start finding that "simple as possible" does not help that much anymore. LDAP support becomes useful and so on. Maybe, I hope maybe, it's not a real portal that we pass through, but it does seem so.
- chillfox 6y ago
  The metric is number of different things you do as an organisation, or number of bored staff. You could probably keep things simple for longer if you issue everyone with a new fidget spinner monthly. A lot of what I see is just people who can’t resist fiddling with stuff that works until it doesn’t.
  - lifeisstillgood 6y ago
    The usual approach (cheaper than fidget spinners) is to have so few people and so much work they only do the barest essentials
    - chillfox 6y ago
      Yeah, but that doesn't work when you have middle managers who's pay is tied to the amount of people that they manage.
markstos 6y ago
I'm all for trying simpler approaches and have written some tooling in bash myself, but at this point Python is pre-installed on many servers like bash is, a large benefit of Ansible is the VAST number of modules already written for Ansible. Imagine if this grew to support all of Ansibles features and modules, but did so in bash. shudders
simonw 6y ago
This looks really promising. I've been wanting a very lightweight solution for this which runs on a single host (no need for another host just to run the scripts) and pulls its configuration from a GitHub repository periodically. I ended up writing my own - and hating that I had to do that - because I couldn't figure out how to use any of the existing options. Their documentation all seems to assume multiple hosts and a separate machine that you run the scripts on. "Each host periodically fetches the latest version of the inventory to see what roles should it be performing" - that's exactly what I want.
- miked85 6y ago
  > which runs on a single host (no need for another host just to run the scripts) This isn't a limitation of Ansible, you can run playbooks against localhost.
  - zhengyi13 6y ago
    True, but I think OP's point at least in part was that an awful lot of the tutorials out there start with the assumption that the audience are sysads and SREs who want to manage a fleet.
    - simonw 6y ago
      Yes, exactly. I could tell that Ansible could do it but so much of the documentation assumed you would be using it with a fleet that I eventually gave up figuring it out.
- VWWHFSfQ 6y ago
  Sounds like you're looking for ansible-pull
- atsaloli 6y ago
  CFEngine is the most lightweight solution I have seen (in terms of resource utilization). The agent is a small C binary. It’s fast! You can set it up to pull its configuration from git.
drinchev 6y ago
> Installation > curl https://aviary.sh/install https://aviary.sh/install | sudo bash I would really try to avoid this, especially when it's targeted for sysadmins ( I guess DevOps nowadays ). What's the big deal of having this instead : > $ wget https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary-install.sh https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary... > ./aviary-install.sh
- nerdponx 6y ago
  Better still: wget https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary-install.sh less aviary-install.sh ./aviary-install.sh
- _frkl 6y ago
  I like having the option, tbh. I usually self host the install scripts, which removes most of the potential issues.
- geofft 6y ago
  Any sysadmin (or devops engineer) worth trusting with root on production machines a) knows how to convert from one to the other b) knows there isn't a difference between the two unless you actually go look at the script in between the wget and the execution (and, probably, go look at what it downloads too and read through all the code). On non-production machines the first one is easier.
efrecon 6y ago
I have been working lately on something a bit similar, but in pure POSIX sh. It's called primer, https://github.com/efrecon/primer https://github.com/efrecon/primer, has no agent, cannot work on remote host but supports amalgamation for easy remote installation and operation. Primer is very much docker oriented in the sense that it seeks to automate the installation of all tools before a docker project, then supposes you run docker-compose for the rest of your projects, or orchestrate in any other way.
mothsonasloth 6y ago
Currently I do a fresh install of Ubuntu every time I do new work (every 6months). Normally I spend about 6 hours getting my machine configured up (installing softwarem configuring my terminal, etc) Will Aviary be able to solve this?
- whycombagator 6y ago
  Maybe you have an involved set up and already do these things, but surely you could just write a bash script that installs all the software you need and store it with your dotfiles. After a new Ubuntu install you can just pull down your dotfiles and run your install script. Aside from some things like ssh keys, gnome tweaks, etc. - this approach would likely get you 90% of the way to set up and would take ~15 mins. Arriving at this set up would take a little bit of time, and require periodic maintenance, but given a weekend you could have it set up and tested. Here[0] is an interesting way to store dot files. [0] https://www.anand-iyer.com/blog/2018/a-simpler-way-to-manage-your-dotfiles.html https://www.anand-iyer.com/blog/2018/a-simpler-way-to-manage...
peterwwillis 6y ago
I don't care what language or paradigm the tool is written with, Configuration Management tools are evil and should be avoided at all costs. I would literally rather maintain a crappy server and fragile software install manually and treat it as a pet, than use any CM tool at all and pretend my servers are cattle. In reality, the CM becomes the pet. The solution to CM is to use immutable infrastructure, and versioned immutable artifacts. With these paradigms, state never drifts, so there is never a need for configuration management at all. Everything becomes stable and predictable and you no longer have to maintain a finicky pet. But how do you bootstrap your systems, you say? The simplest way possible: make a crappy procedural program that bootstraps the very beginning of your system just enough to push versioned immutable artifacts and run arbitrary commands (essentially just "scp", "reboot", "docker pull", "exec"). With cloud-based systems, you shouldn't need to bootstrap anything at all. Build your versioned system images and containers, deploy them, destroy them, re-deploy them. No offense meant to Aviary, I'm sure there are still legacy systems that require some CM before they can be abandoned, but I really hope people building new systems will abandon them ASAP.
Propolice 6y ago
If you like rerun, drist or bashing. I made a short Go program to replicate the experience. https://github.com/tongson/rr https://github.com/tongson/rr Yeah, naming conflict with the debugger. Was a simpler replacement for rerun so the shorter command name made sense to me.
rawoke083600 6y ago
Bash just use bash ! Nothing new to learn for developers or to debug. Ok i'm ready for my down-votes...
burritosnob 6y ago
This was written by team.video who is in a hyper competitive space. Every hour spent on Aviary is an hour not spent on differentiating customer features. Why? There are a ton of open source CM tools that give you same functionality and do it better. This is a problem you should have solved after you were a) profitable b) at scale and c) probably not even then.
- deleted 6y ago
  [deleted]
- Uehreka 6y ago
  So like, unless you're one of their investors upset at how your money's being spent, why is this any of your business? Startups often take unusual paths to profitability. They all seem crazy at first, many are in fact crazy, a few turn out to be quite prescient.
  - burritosnob 6y ago
    Not being an investor doesn't preclude me from having an opinion. There is nothing prescient here... just an engineering team burning cycles better spent elsewhere at current stage of the company.
hummo56 6y ago
Looked good but I needed to leave the site after seeing they suggested doing curl https://aviary.sh/install https://aviary.sh/install | sudo bash
- lukevp 6y ago
  What’s the problem with running a script this way? Several major tools install like this (docker, netdata...) here’s a take on this that was posted to HN a while back: https://www.arp242.net/curl-to-sh.html https://www.arp242.net/curl-to-sh.html
  - effie 6y ago
    The problem is that this way, you're giving total control of your machine to a script that curl will download from a website, before you check the script. A better way to try a new software is to download the sources, check them, build/install/run them under a low user with lowest required access to your system. Even better, do this in a virtual machine.
- Carpetsmoker 6y ago
  Discounting an entire tool just because you don't like their recommended installation method (which you're hardly forced to use) sounds rather strange to me.
- tannhaeuser 6y ago
  I think you should reconsider. Curl-bashing (or curl-kshing or whatever) at least ties the download to a domain name in control of the project and uses https. The alternatives - such as serving up a .deb or .rpm with a digest - aren't a good fit when the whole point of a project is to provide a single portable shell script (ok it's only a bash script), and especially when the project itself is acting as an installer of sorts, like this one. Would you rather have dozens of aviary.sh scripts with different versions and potentially malicious modifications flying around on github and download sites?
thayne 6y ago
Wow, perfect timing. I was just looking for something like this. I haven't looked at it too in-depth yet, but the design goals align with what I want. Minimal configuration management that pulls config from git and runs locally. Ansible pull, salt masterless, chef solo, etc. work, but they weren't designed to be run that way, so have unneeded complexity (and are kind of heavy for what I want).
senorsmile 6y ago
How is this idempotent? cat <<EOF > /etc/motd "Ever make mistakes in life? Let’s make them birds. Yeah, they’re birds now." --Bob Ross EOF
- mike_d 6y ago
  Paraphrased from Wikipedia: "A function looking up a customer's name and address in a database is typically idempotent, since this will not cause the database to change. Similarly, changing a customer's address to XYZ is typically idempotent, because the final address will be the same no matter how many times XYZ is submitted. However, placing an order for a cart for the customer is typically not idempotent, since running the call several times will lead to several orders being placed. Canceling an order is idempotent, because the order remains canceled no matter how many requests are made." The greater-than redirector fits that description, as it is normally implemented as an atomic write. Note that it does not say anything about optimization or efficiency. Only that the result remains unchanged. Web developers have bastardized the term and take it to mean many things it does not because of the definition of HTTP GET, even though no GET operation is ever idempotent in the real world.
  - senorsmile 6y ago
    In automation (like Ansible), the term is overloaded. Something is only idempotent if it only tries to take action if the actual state does not match the desired state. This will overwrite every time, which is inefficient. As a plus, Ansible also notifies you which things have changed. This also enables easy dry runs (check_mode in Ansible).
comba 6y ago
More modular, smaller, simpler than aviary is rrconf https://github.com/rrconf/rrconf https://github.com/rrconf/rrconf

276 points by dxchester 6y ago

Hackbraten 6y ago

I’ve been waiting for a tool like this.

dxchester 6y ago
Here you go! Yeah, I've used a bunch of CM tools over the years, and it always seems like you're learning a DSL of one sort or another, just to have it translate into the bash that you already know.
- cloin 6y ago
  And don't get me wrong, I love bash. I automated my first job using bash. But relying on bash just relegates aviary to a certain job description and thus just low value, domain specific tasks. Seems like a neat way for a linux admin to run a homelab, but not something that's easily adopted for larger teams working across OS, network platforms or integrating with external services.
  - chillfox 6y ago
    Not everything needs to be for large teams. There’s value in simple solutions for small teams.
- cloin 6y ago
  YAML is a standard for so many different tools now that I don't think it's really considered a DSL.

ahnick 6y ago

If you need a simple way to inject secrets into your aviary variables you may find encpass.sh (https://github.com/plyint/encpass.sh https://github.com/plyint/encpass.sh) a convenient choice.

logie17 6y ago
That's a pretty cool project. It also is written in bash as well!

nyxtom 6y ago

Nice, I plan on using this

freedomben 6y ago

Really neat. I still need to evaluate it more, but this is the kind of thing I've looking for for small infrastucture that isn't likely to grow (like home services or small office, etc).

ansible 6y ago

I do something sort of similar, also using git to maintain the inventory of what to run on what machine. In my case, I'm using docker-compose.yml files for each machine, and then running: docker-compose pull docker-compose up -d We initially wanted to use Watchtower, but it didn't work well for us.

yonixw 6y ago
Nitpicking: Why pull separately? it will pull if no image stored locally or even if the `docker-compose.yml` changed.
- randallsquared 6y ago
  Two reasons I've done this in my own Makefiles: predictable start times, and separation of failure-prone steps.
- yjftsjthsd-h 6y ago
  Stale images can be a reason. If I specify `image: foo:latest`, then push new versions, it won't automatically pull new versions unless you do an explicit pull.
xomodo 6y ago
Try docker contexts: https://www.docker.com/blog/how-to-deploy-on-remote-docker-hosts-with-docker-compose/ https://www.docker.com/blog/how-to-deploy-on-remote-docker-h...
- darkteflon 6y ago
  This is a great feature, introduced in Feb this year. It addressees a substantial pain point with docker-compose. Thanks for pointing this out.
  - ptrincr 6y ago
    Indeed this does look very useful. I had a quick glance over this, but for contexts which are defined as remote ssh hosts, I couldn't see where the images are obtained. Are they being built remotely on the host, or are they being downloaded from your local machine over a tunnel or via a docker registry?

arminiusreturns 6y ago

Now this is something I can get behind! Very cool work. I think bash gets a bit of a bad rap and it's very popular to bandwagon hate on it, but it's one of my favorite tools ever.

easterncalculus 6y ago

Very nice! Ansible can be a little heavyweight for some of the things that me and a lot of people do. I'll definitely be taking a look at this.

ggregoire 6y ago

Ansible is agentless tho. That's a pretty big difference.

gjulianm 6y ago
Yeah, it was the thing that made me choose it instead of other alternatives. I manage machines in heterogeneous client networks without direct Internet access most of the time and agents for managing infrastructure are not useful at all.
peterwwillis 6y ago
Yes, but it's also horrible to use. I would rather set up a Puppet agent once than struggle with Ansible for an eternity.
- pricechild 6y ago
  Would I be right if I guessed you have a development background? For reference, I have an ops background.
  - jeltz 6y ago
    I have a dev background and think that Ansible is much nicer to work with than Chef and Puppet.
  - peterwwillis 6y ago
    Nope, I'm an ops guy. Been banging my head against these tools for 15 years.

xbryanx 6y ago

I'm not sure if I am understanding this (aviary.sh) correctly, but it looks like this requires an agent to run on the configured hosts. One of the things that I love about Ansible's model is that we never need an agent on the host. Once you configure something with Ansible, there's no artifact of the configuration left on the machine.

pampa 6y ago
That is not entirely true. Ansible needs python installed on the target host, and a lot of modules (plugins?) require additional python libraries on the target.
- soco 6y ago
  It doesn't need any python. Ansible opens a regular ssh session and can run anything from it - including python if available.
  - dfinninger 6y ago
    Only with the “raw” module... which is what you use to ensure Python is installed on a machine managed by Ansible.
- pnutjam 6y ago
  you can use the "raw" module without python.
- cloin 6y ago
  If python is considered an agent, isn't bash now an agent?
- divbzero 6y ago
  The av agent that GP refers to is a script that runs periodically on target hosts to pull the latest configuration using Git. Ansible has a Python dependency but no agent running on target hosts, instead relying on SSH to push configuration changes. Ideally, I’d like to see a configuration tool that uses Ansible’s SSH push model but without the Python dependency.

MINIMAN10000 6y ago

Unfortunately all I could think of was "that sounds cool but I'd want to stick to dash instead of bash" Also didn't realize it wasn't agentless which is pretty critical imo

yjftsjthsd-h 6y ago
> Unfortunately all I could think of was "that sounds cool but I'd want to stick to dash instead of bash" Well, POSIX sh, ideally. That way you get free portability to ~everything; it's not a great burden to install bash or dash on ex. NetBSD or AIX, but sh is as close to universally preinstalled as you're going to get. > Also didn't realize it wasn't agentless which is pretty critical imo Ouch, yeah that outright disqualifies it for me.

t0astbread 6y ago

This looks really effective! I do something similar to keep track of what's on my workstations: I just have one large idempotent shell script to set up all the software I use. To help myself not loose track of updating it I have a little helper script to detect packages that aren't tracked in the config. I've recently thought about breaking it up into smaller "module-style" scripts. Maybe this could be a good fit (with minor adoptions)!

dheera 6y ago

> Install from the command line, on a box to be managed by aviary.sh: It would be even cooler if I didn't need to do anything to do the box besides install openssh-server. I should be able to: $ sudo apt-get install aviary $ aviary install my_server 192.168.1.25 # a fresh ubuntu 18.04 install and everything else should just be magic from there.

tony 6y ago

If you like this - Fabric is awesome. https://www.fabfile.org/ https://www.fabfile.org/ Use python over SSH. Super fast to begin, if you can ssh into the server, you can run the equivalent of shell commands (subprocess calls in python) remotely. With python, you can abstract and reuse, with the scriptability of bash, but higher level niceties of the language, libraries and toolchain (linting, formatting, autocompletion, imports). Can it pull the latest version of your app, build it on your server? Yep. Restart daemons? Update libraries (npm, python, etc)? Update system packages? All work fine. Can it configure a vanilla server from scratch? Yes, it's helpful to bring in a higher level reusable library like https://github.com/fabtools/fabtools https://github.com/fabtools/fabtools helps with that. When does it hit limitations? More complex orchestration where you have multiple servers (and variations of them) with configuration to talk to each other over networks. Eventually you get to the kind of setup where having a tool like Ansible / Salt / Puppet / etc. makes sense. For a basic PHP/python/ruby/node site that's just running on a single cloud server? A declarative config manager would be overkill for me.

linuxftw 6y ago
> Use python over SSH. Super fast to begin, if you can ssh into the server, you can run the equivalent of shell commands (subprocess calls in python) remotely. This literally describes ansible. In fact, ansible one-ups this because you can specify raw commands if python is not installed remotely [1]. Ansible provides lots of modules, you don't have to use any of them. I have plenty of 1-off ansible playbooks that I don't care about idempotency that are just a bunch of 'cmd' statements. It's a very flexible tool. 1: https://docs.ansible.com/ansible/latest/modules/raw_module.html https://docs.ansible.com/ansible/latest/modules/raw_module.h...
- maxmalysh 6y ago
  Not really. Ansible is a complex DSL based on YAML. Fabric is really simple and uses real Python.
  - llama052 6y ago
    Can't say I've ever really thought of YAML as complex. Especially when the Ansible modules are translating things so you can just define the action you want to take, and each module is very well documented.

q3k 6y ago

> bash is just fine (yes, it is) No, it's not. It's 2020, we need to stop it with the stringly-typed, footgun-filled madness that is bash scripting. We deserve better than this. Unless I'm missing something, this README example already contains a bug: template $(dirname $0)/motd.template > /etc/motd This will fail if the script path contains a space.

montroser 6y ago
I think you are forgetting that serious operators do not have spaces in their paths.
- hk__2 6y ago
  *People who are used to work with bad tools don’t have spaces in their paths.
  - tenebrisalietum 6y ago
    "C:\Documents and Settings" still hurts after all these years.
    - linuxftw 6y ago
      "C:\Program Files(x86)". Who ok'd that?
      - GrinningFool 6y ago
        I haven't looked in a while, but last I checked you could also still get to it via C:\PROGRA~1? (This was years ago, that might have been 'fixed'?)
        joombaga 6y ago
        These are called 8.3 filenames. They're a filesystem feature. They're still created by default in Windows 10 on NTFS, but can be disabled or stripped using `fsutil`.

salasrod 6y ago

> But it's too funky you say. Well, yes, bash can have its quirks. But we need some funk every now and then. Let's just embrace it! My god, no, stop. Funkyness in production equals outages.

ironmagma 6y ago
As bad as Ruby is, it’s still better than Bash and you can do all the same effortless execution of random commands in your script like you would in bash. If one can’t be bothered to write it in a real language like Rust or even Python, you could at least do Ruby.
- cortesoft 6y ago
  What makes python more of a real language than ruby?
  - ironmagma 6y ago
    It’s more mainstream for tools like this.
    - rad_gruchalski 6y ago
      Please explain. Have you heard of chef? That is ruby and it can be beautiful. Chef > Ansible. Chef server is meh.
      - ironmagma 6y ago
        The point wasn’t that Python is better than Ruby, they are about the same IMO. The point was they are both better than Bash, and Ruby actually makes it super convenient to write bash-like scripts, so there’s no excuse to not be using Ruby.
    - e12e 6y ago
      Eh.. Puppet? [ed: and chef..]

nodesocket 6y ago

Typically I use Packer + Terraform and it works generally well (if you going with the immutable servers philosophy). However, often times immutable servers is hard to actually do in practice. If Hashicorp added a new product similar to Ansible / Aviary.sh / Fabric I think it would complete the entire DevOps loop under a single ecosystem.

raffraffraff 6y ago
If you're sticking with hashicorp tools and use consul, then something like consul-template should work. Your packer builds would just have to copy consul-template configs and template files for the things you want to manage. Each template watches the consul kv store for matching keys, and rewrites the target config file using their values. You can instantly rewrite configs and execute follow up actions across your fleet by updating a consul key value.
- nodesocket 6y ago
  Agree Consul is an elegant solution for managing files, what about running commands?
  - moondev 6y ago
    cloud-init?
  - johntash 6y ago
    I don't see it used very often, but Consul actually can execute remote commands. You do have to enable it in the config though. https://www.consul.io/docs/commands/exec https://www.consul.io/docs/commands/exec edit: Also, Nomad would be another choice if you're looking at hashi tools.
llama052 6y ago
You can hook ansible roles into your Packer process pretty easily. That's what I do at least.

trynewideas 6y ago

it requires an agent, so it's a tiny Bash alternative to Puppet, but that doesn't sound remotely as cool

stevekemp 6y ago
I mentioned it above already, but I've been experimenting with an agentless puppet-lite: https://github.com/skx/marionette/ https://github.com/skx/marionette/ Early days, but the primitives are enough to do useful things on my own systems..

stevekemp 6y ago

I've been working on a localhost-only system for automation myself, inspired by puppet but with far fewer modules: https://github.com/skx/marionette/ https://github.com/skx/marionette/ As of this week it can now pull down docker-images from dockhub, which was something I'd been needing to rebuild my own systems. Early days, and it is as much proof-of-concept as anything else, but it seems robust and reliable in my own use. Of course I need to add "written in go" to advertise properly!

dastx 6y ago
On first glance this looks awesome. Certainly will be better than the bunch of scripts I've written to provision my laptops.
- stevekemp 6y ago
  Fingers crossed it will work for you, and others! It replaces a simpler alternative I wrote in the past which used ssh/scp to upload files and run commands: https://github.com/skx/deployr https://github.com/skx/deployr Right now I have a git-repository containing per-host config. So I can run something like this via cron: cd /srv/config/$(hostname --fqdn)/ marionette main.in I should document that, and the inclusion-facility explicitly. But otherwise feel free to file bug-reports/feature-requests if you have ideas for modules that would be useful.
ahnick 6y ago
Does this save time in writing your own personal automation or make that personal automation less error prone? How would you compare it to just writing your own bash scripts or using something like Bashible (https://github.com/mig1984/bashible https://github.com/mig1984/bashible)?
- stevekemp 6y ago
  Making things repeatable and scripting will usually be a time-saver (unless you spend 100+ hours writing the automation and never reinstall your hosts!) For me it was a learning experience as much as anything else. I'd probably have stuck with ansible, puppet, or similar established project if I didn't want to learn/experiment.

lifeisstillgood 6y ago

I think there is an Genuine Enterprise Need explosion at certain scales. It is not box ticking exercises or empire building but sometime after (my guess) 1,000 units of anything (users, servers, customers, whatever) you start finding that "simple as possible" does not help that much anymore. LDAP support becomes useful and so on. Maybe, I hope maybe, it's not a real portal that we pass through, but it does seem so.

chillfox 6y ago
The metric is number of different things you do as an organisation, or number of bored staff. You could probably keep things simple for longer if you issue everyone with a new fidget spinner monthly. A lot of what I see is just people who can’t resist fiddling with stuff that works until it doesn’t.
- lifeisstillgood 6y ago
  The usual approach (cheaper than fidget spinners) is to have so few people and so much work they only do the barest essentials
  - chillfox 6y ago
    Yeah, but that doesn't work when you have middle managers who's pay is tied to the amount of people that they manage.

markstos 6y ago

I'm all for trying simpler approaches and have written some tooling in bash myself, but at this point Python is pre-installed on many servers like bash is, a large benefit of Ansible is the VAST number of modules already written for Ansible. Imagine if this grew to support all of Ansibles features and modules, but did so in bash. shudders

simonw 6y ago

This looks really promising. I've been wanting a very lightweight solution for this which runs on a single host (no need for another host just to run the scripts) and pulls its configuration from a GitHub repository periodically. I ended up writing my own - and hating that I had to do that - because I couldn't figure out how to use any of the existing options. Their documentation all seems to assume multiple hosts and a separate machine that you run the scripts on. "Each host periodically fetches the latest version of the inventory to see what roles should it be performing" - that's exactly what I want.

miked85 6y ago
> which runs on a single host (no need for another host just to run the scripts) This isn't a limitation of Ansible, you can run playbooks against localhost.
- zhengyi13 6y ago
  True, but I think OP's point at least in part was that an awful lot of the tutorials out there start with the assumption that the audience are sysads and SREs who want to manage a fleet.
  - simonw 6y ago
    Yes, exactly. I could tell that Ansible could do it but so much of the documentation assumed you would be using it with a fleet that I eventually gave up figuring it out.
VWWHFSfQ 6y ago
Sounds like you're looking for ansible-pull
atsaloli 6y ago
CFEngine is the most lightweight solution I have seen (in terms of resource utilization). The agent is a small C binary. It’s fast! You can set it up to pull its configuration from git.

drinchev 6y ago

> Installation > curl https://aviary.sh/install https://aviary.sh/install | sudo bash I would really try to avoid this, especially when it's targeted for sysadmins ( I guess DevOps nowadays ). What's the big deal of having this instead : > $ wget https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary-install.sh https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary... > ./aviary-install.sh

nerdponx 6y ago
Better still: wget https://gitlab.com/dchester/aviary.sh/-/archive/1.3.2/aviary-install.sh less aviary-install.sh ./aviary-install.sh
_frkl 6y ago
I like having the option, tbh. I usually self host the install scripts, which removes most of the potential issues.
geofft 6y ago
Any sysadmin (or devops engineer) worth trusting with root on production machines a) knows how to convert from one to the other b) knows there isn't a difference between the two unless you actually go look at the script in between the wget and the execution (and, probably, go look at what it downloads too and read through all the code). On non-production machines the first one is easier.

efrecon 6y ago

I have been working lately on something a bit similar, but in pure POSIX sh. It's called primer, https://github.com/efrecon/primer https://github.com/efrecon/primer, has no agent, cannot work on remote host but supports amalgamation for easy remote installation and operation. Primer is very much docker oriented in the sense that it seeks to automate the installation of all tools before a docker project, then supposes you run docker-compose for the rest of your projects, or orchestrate in any other way.

mothsonasloth 6y ago

Currently I do a fresh install of Ubuntu every time I do new work (every 6months). Normally I spend about 6 hours getting my machine configured up (installing softwarem configuring my terminal, etc) Will Aviary be able to solve this?

whycombagator 6y ago
Maybe you have an involved set up and already do these things, but surely you could just write a bash script that installs all the software you need and store it with your dotfiles. After a new Ubuntu install you can just pull down your dotfiles and run your install script. Aside from some things like ssh keys, gnome tweaks, etc. - this approach would likely get you 90% of the way to set up and would take ~15 mins. Arriving at this set up would take a little bit of time, and require periodic maintenance, but given a weekend you could have it set up and tested. Here[0] is an interesting way to store dot files. [0] https://www.anand-iyer.com/blog/2018/a-simpler-way-to-manage-your-dotfiles.html https://www.anand-iyer.com/blog/2018/a-simpler-way-to-manage...

peterwwillis 6y ago

I don't care what language or paradigm the tool is written with, Configuration Management tools are evil and should be avoided at all costs. I would literally rather maintain a crappy server and fragile software install manually and treat it as a pet, than use any CM tool at all and pretend my servers are cattle. In reality, the CM becomes the pet. The solution to CM is to use immutable infrastructure, and versioned immutable artifacts. With these paradigms, state never drifts, so there is never a need for configuration management at all. Everything becomes stable and predictable and you no longer have to maintain a finicky pet. But how do you bootstrap your systems, you say? The simplest way possible: make a crappy procedural program that bootstraps the very beginning of your system just enough to push versioned immutable artifacts and run arbitrary commands (essentially just "scp", "reboot", "docker pull", "exec"). With cloud-based systems, you shouldn't need to bootstrap anything at all. Build your versioned system images and containers, deploy them, destroy them, re-deploy them. No offense meant to Aviary, I'm sure there are still legacy systems that require some CM before they can be abandoned, but I really hope people building new systems will abandon them ASAP.

Propolice 6y ago

If you like rerun, drist or bashing. I made a short Go program to replicate the experience. https://github.com/tongson/rr https://github.com/tongson/rr Yeah, naming conflict with the debugger. Was a simpler replacement for rerun so the shorter command name made sense to me.

rawoke083600 6y ago

Bash just use bash ! Nothing new to learn for developers or to debug. Ok i'm ready for my down-votes...

burritosnob 6y ago

This was written by team.video who is in a hyper competitive space. Every hour spent on Aviary is an hour not spent on differentiating customer features. Why? There are a ton of open source CM tools that give you same functionality and do it better. This is a problem you should have solved after you were a) profitable b) at scale and c) probably not even then.

deleted 6y ago
[deleted]
Uehreka 6y ago
So like, unless you're one of their investors upset at how your money's being spent, why is this any of your business? Startups often take unusual paths to profitability. They all seem crazy at first, many are in fact crazy, a few turn out to be quite prescient.
- burritosnob 6y ago
  Not being an investor doesn't preclude me from having an opinion. There is nothing prescient here... just an engineering team burning cycles better spent elsewhere at current stage of the company.

hummo56 6y ago

Looked good but I needed to leave the site after seeing they suggested doing curl https://aviary.sh/install https://aviary.sh/install | sudo bash

lukevp 6y ago
What’s the problem with running a script this way? Several major tools install like this (docker, netdata...) here’s a take on this that was posted to HN a while back: https://www.arp242.net/curl-to-sh.html https://www.arp242.net/curl-to-sh.html
- effie 6y ago
  The problem is that this way, you're giving total control of your machine to a script that curl will download from a website, before you check the script. A better way to try a new software is to download the sources, check them, build/install/run them under a low user with lowest required access to your system. Even better, do this in a virtual machine.
Carpetsmoker 6y ago
Discounting an entire tool just because you don't like their recommended installation method (which you're hardly forced to use) sounds rather strange to me.
tannhaeuser 6y ago
I think you should reconsider. Curl-bashing (or curl-kshing or whatever) at least ties the download to a domain name in control of the project and uses https. The alternatives - such as serving up a .deb or .rpm with a digest - aren't a good fit when the whole point of a project is to provide a single portable shell script (ok it's only a bash script), and especially when the project itself is acting as an installer of sorts, like this one. Would you rather have dozens of aviary.sh scripts with different versions and potentially malicious modifications flying around on github and download sites?

thayne 6y ago

Wow, perfect timing. I was just looking for something like this. I haven't looked at it too in-depth yet, but the design goals align with what I want. Minimal configuration management that pulls config from git and runs locally. Ansible pull, salt masterless, chef solo, etc. work, but they weren't designed to be run that way, so have unneeded complexity (and are kind of heavy for what I want).

senorsmile 6y ago

How is this idempotent? cat <<EOF > /etc/motd "Ever make mistakes in life? Let’s make them birds. Yeah, they’re birds now." --Bob Ross EOF

mike_d 6y ago
Paraphrased from Wikipedia: "A function looking up a customer's name and address in a database is typically idempotent, since this will not cause the database to change. Similarly, changing a customer's address to XYZ is typically idempotent, because the final address will be the same no matter how many times XYZ is submitted. However, placing an order for a cart for the customer is typically not idempotent, since running the call several times will lead to several orders being placed. Canceling an order is idempotent, because the order remains canceled no matter how many requests are made." The greater-than redirector fits that description, as it is normally implemented as an atomic write. Note that it does not say anything about optimization or efficiency. Only that the result remains unchanged. Web developers have bastardized the term and take it to mean many things it does not because of the definition of HTTP GET, even though no GET operation is ever idempotent in the real world.
- senorsmile 6y ago
  In automation (like Ansible), the term is overloaded. Something is only idempotent if it only tries to take action if the actual state does not match the desired state. This will overwrite every time, which is inefficient. As a plus, Ansible also notifies you which things have changed. This also enables easy dry runs (check_mode in Ansible).

comba 6y ago

More modular, smaller, simpler than aviary is rrconf https://github.com/rrconf/rrconf https://github.com/rrconf/rrconf