8 ms·
Show HN: Bash-My-AWS – CLI Commands for AWS
- mike-bailey 7y agoIt's probably my fault if you haven't heard of Bash-My-AWS. Bash-My-AWS is a simple but extremely powerful set of CLI commands for managing resources on Amazon Web Services. They harness the power of Amazon's AWSCLI, while abstracting away the verbosity. The project implements some innovative patterns but (arguably) remains simple, beautiful, readable and easily extensible. The project started in 2014 and while many hundreds of hours have gone into it, far less has gone into promotion. I'm speaking about it at LinuxConf and have created a documentation site at https://bash-my-aws.org https://bash-my-aws.org https://linux.conf.au/schedule/presentation/144/ https://linux.conf.au/schedule/presentation/144/
- bob33 7y agofor anyone on this thread that is interested. I run https://getcommandeer.com https://getcommandeer.com which is a tool to manage your AWS and IAC infrastructure from a desktop GUI. I love this bash-my-aws, as we are about to release Bash, Docker Compose, and Terraform Runners. We already have Serverless and Ansible runners. They enable you to run your command line system from a GUI, so that you can instantly switch between AWS accounts/regions and even LocalStack. Because it is a desktop, under the hood we are really running cli tools mixed in with some AWS JS SDK.
- GhettoMaestro 7y agoVery cool app. Giving it a try right now.
- dastx 7y agoWhy does your main website (linked in your comment) require JS to run? I've not seen anything that warrants that requirement. I wish people would stop making websites that's require JS.
- Hamuko 7y agoSpoiler alert: if you don’t want to run JavaScript in your browser, you’re not going to enjoy this application.
- bob33 7y agoyea, it works terrible without a monitor as well. The site does not work headless. what was I thinking? It's a desktop application. The site is where you download the app.
- bob33 7y agoTo be a little more constructive with my answer. The website needs javascript for our chat service, our newsletter service, our ability for users to purchase licenses for the app, the night/dark mode switch. The codebase also shares components with our desktop app as they are both Vue, but with the desktop app utilizing electron. We will look to make the site be html only at some point, but it is not a focus of our dev team. If you download the app, you will see that it is a first rate experience for managing many AWS services. The app itself probably has about 1,000 man hours or more put into it. Would love to hear insight into how we can make it even more powerful for the community.
- deleted 7y ago[deleted]
- mike-bailey 7y agoHere's the video from my talk on Bash-my-AWS at LinuxConf 2020 in the Gold Coast Australia. https://www.youtube.com/watch?v=UbH_cg7Ev1Q https://www.youtube.com/watch?v=UbH_cg7Ev1Q
- pensatoio 7y agoWhat really sells me on this tool is the ability to examine the underlying awscli command and transformations. I’ll be giving this a go in the new year!
- failmode 7y agoThanks! The intent has always been to enhance rather than replace AWCLI (which is an amazing tool!). If you're ever wondering how a Bash-My-AWS command works, use `bma type` (it even supports tab completion for all the commands). $ bma type instances instances is a function instances () { local instance_ids=$(__bma_read_inputs); local filters=$(__bma_read_filters $@); aws ec2 describe-instances $([[ -n ${instance_ids} ]] && echo --instance-ids ${instance_ids}) --query " Reservations[].Instances[][ InstanceId, InstanceType, State.Name, [Tags[?Key=='Name'].Value][0][0], LaunchTime, Placement.AvailabilityZone, VpcId ]" --output text | grep -E -- "$filters" | LC_ALL=C sort -b -k 6 | column -s' ' -t }
- zk68420 7y agoyou can do this in a very simple tip check details here;http://bit.ly/2lRXocyz http://bit.ly/2lRXocyz
- Terretta 7y agoInteresting this requires ‘jq’ when JMESpath is built into AWS CLI already. http://jmespath.org/ http://jmespath.org/
- fiddlerwoaroof 7y agoI find jq’s language a lot nicer than JMESpath and trend to use it whenever possible
- kesor 7y agojmespath has quite a few limitations, even the official AWS CLI documentation states that for the more advanced stuff `jq` is probably the go to tool. https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output.html#json-output https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-o... "For more advanced filtering that you might not be able to do with --query, you can consider jq, a command line JSON processor. You can download it and find the official tutorial at http://stedolan.github.io/jq/." http://stedolan.github.io/jq/."
- justin_oaks 7y agoMy HN comment detailing several limitations of JMESPath: https://news.ycombinator.com/item?id=16400320 https://news.ycombinator.com/item?id=16400320
- failmode 7y agojq is only used in three of the >120 functions (for sort-keys functionality). All the rest use JMESPath. If anyone can help with a solution I'd be delighted to remove the dependency on jq. https://github.com/bash-my-aws/bash-my-aws/blob/b74d92a902bb54ab7527eb40e79d397b3895bf0c/lib/stack-functions#L326 https://github.com/bash-my-aws/bash-my-aws/blob/b74d92a902bb...
- kafana 7y agoI would definitely keep jq dependency there, especially if you plan to expand the code base to provide ecs and Fargate commands. You will quickly run into use cases where JMESpath is not capable of parsing json outputs for different tasksdefintion commands.
- dopylitty 7y agoIf you want to easily manipulate your AWS environment from the command line use the AWS cmdlets for PowerShell. The fact that PowerShell cmdlets work on objects instead of text makes them miles better than this or the AWS CLI because you don't spend most of your time figuring out how to wrangle text into meaningful output.
- dvtrn 7y agoHasn’t AWSCLI supported toggling the cmd output to either text, json or csv for quite some time now or have I misunderstood your comment here?
- jolux 7y agoYou don’t get the impedance mismatch of text to objects that bash has when dealing with the complexity of AWS resources.
- failmode 7y agoBash-My-AWS wraps AWSCLI as thinly as possible and makes use of JMESPath and the text output. The result is you have a simple set of commands that don't require you to type hundreds of characters. instances() { local instance_ids=$(__bma_read_inputs) local filters=$(__bma_read_filters $@) aws ec2 describe-instances \ $([[ -n ${instance_ids} ]] && echo --instance-ids ${instance_ids}) \ --query " Reservations[].Instances[][ InstanceId, InstanceType, State.Name, [Tags[?Key=='Name'].Value][0][0], LaunchTime, Placement.AvailabilityZone, VpcId ]" \ --output text | grep -E -- "$filters" | LC_ALL=C sort -b -k 6 | column -s$'\t' -t }
- jrockway 7y agoDo you have any insights on how someone who is used to the text-only world of Bash transition to using Powershell cmdlets? The problem I run into is that it just feels like so much typing to me. I have to read documentation. All the attributes HaveReallyLongNamesThatContainCapitalLetters. By the time I've made my beta version of the command I want to run, I feel like I need to open a text editor to finish it. Maybe add some error checking. Some comments too. Maybe a unit test or three. And now I have an entire project and all I wanted to do was add a line of text to the end of a file. Part of the problem on my part is my own ignorance of the APIs and what commands are available to me. But it all seems too verbose to use practically. The Powershell language seems very good for what you would write a shell script to do, but for interactive commands, I have a hard time believing that people use it. It's just so verbose.
- TheSpiciestDev 7y agoJust the other day I was looking for an official docker image that includes the AWS CLI. On top of that, and mainly, I was looking to find more documentation or tooling to better automate the deployment of new AWS projects. Does anyone here have any experience of (starting from scratch or with no AWS resources) setting up policies/users/resources/configurations via something similar to the Deployment Managers of GCP and Azure?.. preferably something declarative or via templates? Bash-my-AWS looks like a great step towards the goal I have in mind but I may also just be unaware of other tooling or AWS capabilities.
- zbruhnke 7y agoGruntwork has a lot of Open Source tooling around AWS and their new guides are pretty great for some of what you're mentioning https://gruntwork.io/guides/ https://gruntwork.io/guides/ I am in no way affiliated with them other than being a customer
- weberc2 7y agoIf I’m understanding you correctly, I think you want CloudFormation?
- TheSpiciestDev 7y agoThanks, AWS CloudFormation looks like what I've experienced with other cloud service providers.
- stevekemp 7y agoThe other alternative is terraform: https://www.terraform.io/ https://www.terraform.io/
- mypalmike 7y agoThere is also AWS Cloud Development Kit, which generates CloudFormation from Typescript, C#, Java, or Python. https://aws.amazon.com/blogs/developer/getting-started-with-the-aws-cloud-development-kit-and-python/ https://aws.amazon.com/blogs/developer/getting-started-with-...
- nahikoa 7y agoThis looks like an awesome project! Meta note: All things considered, Amazon has it pretty good. They put out a barely usable, bare-bones, but fully functional tool in awscli. Paying customers of AWS have to perform the engineering effort to make the API more usable, and some even open-source their projects like this. AWS is an incredible business model.
- dajohnson89 7y agoturning the flywheel
- Aperocky 7y agoAnything above bare bones will be opinionated, imo this is the best solution for infrastructure provider - maximum freedom, but also providing a UI for simpler access.
- failmode 7y agoI have sometimes questioned whether I should be spending my personal time developing an open source tool so tied to a single companies services. My reasons for continuing include: - I prefer to use command line over ClickOps - Using Bash-My-AWS makes me more effective at work - The emergent UX is equally applicable to other services (e.g. bash-my-github, bash-my-spotify) - The intrinsic satisfaction from creating - Helping improve the experience for others
- notyourwork 7y agoThanks for the tool and reasons for building it. I will check it out tomorrow!
- NotSammyHagar 7y agoI looked, no one created packages with those 2 names that I could fine, although there are some clis for controlling spotify it seems. Were you referring to specific packages somewhere for github and spotify?
- 7y ago
- ak217 7y agoI have developed something similar on top of the AWS CLI that incorporates a bunch of integrations with other tools like the cloudinit and various bits of Batch-related instrumentation: https://github.com/kislyuk/aegea https://github.com/kislyuk/aegea
- alpb 7y agoIs this primarily required because AWS CLI is not good enough at listing resources in desired format (json, jsonpath, yaml, table..)?
- failmode 7y agoAWSCLI is amazingly flexible and powerful. Bash-My-AWS thinly wraps AWSCLI commands that would otherwise be too long to type. So you're still using AWSCLI and can improve your skill with it by inspecting the source of Bash-My-AWS functions. https://news.ycombinator.com/item?id=21931298 https://news.ycombinator.com/item?id=21931298
- m0zg 7y agoComing from Google Cloud, I couldn't deal with the atrocity that is awscli, so I ended up eventually implementing the bare minimum of shell wrappers to at least start, stop, ssh into, rsync files to and from, etc, my aws instances _by name_, not by instance ID. Took me a couple of hours to cobble it together. Google cloud CLI offers all of this out of the box. Why Amazon wants to make such basic commands difficult, I'll never understand.
- efitz 7y agoInteracting with individual DNS records in Route 53 is very hard using AWSCLI. I wrote a Python wrapper around the Route 53 API to make it easier to do command line records management (and also to do dynamic DNS with your own Route 53 hosted domain): https://github.com/ericfitz/r53 https://github.com/ericfitz/r53
- iCarrot 7y agoThe first thing I'd do after installing this is to alias prefixing "aws-" to every commands. I like namespacing things as I suck at remembering names...
- failmode 7y agoI don't like to rely on my memory either! I forget the names of commands and use tab completion to list them. You can just type bma[TAB][TAB] and it will list them all. If you know the type of resource you are working with, you can use TAB completion to see it's commands: $ stack- stack-arn stack-exports stack-tag-apply stack-asg-instances stack-failure stack-tag-delete stack-asgs stack-instances stack-tags stack-cancel-update stack-outputs stack-tags-text stack-create stack-parameters stack-tail stack-delete stack-recreate stack-template stack-diff stack-resources stack-update stack-elbs stack-status stack-validate stack-events stack-tag
- wjoe 7y agoThis looks pretty great, while the AWS CLI is very comprehensive, I always struggle to remember which flags are needed for each command, and it's not very consistent. One thing I've not been able to work out with bash-my-aws yet was how to easily switch between regions and accounts. I noticed you can use `region` on it's own to set the current default region, but I'm often working with multiple regions, and it'd be a pain to have to run `region us-west-1` separately each time I want to use a different region. I couldn't see a way to just specify a region for a given command (eg how you'd do `aws get-instances --region us-west-1`). I guess you could do this with the environment variable `AWS_DEFAULT_REGION=us-west-1 instances` but that's a bit verbose. Similarly with AWS accounts, I use multiple AWS accounts, which are accessed with different access keys, which are defined as profiles in my ~/.aws/config. Normally I'd use these with the AWS CLI like `aws ec2 get-instances --profile production`, I couldn't see any way in the docs to use or set this?
- failmode 7y agoThey're good questions. I can tell you how I manage regions and accounts but am interested in learning how people think Bash-my-AWS might better support users in this regard. The AWCLI, as well as SDKs all support grabbing Regions and account credentials from environment variables. For Regions, I work tend to use the following aliases: alias au='export AWS_DEFAULT_REGION=ap-southeast-2' alias us='export AWS_DEFAULT_REGION=us-east-1' alias dr='export AWS_DEFAULT_REGION=ap-southeast-1' I normally work in a single Region and swap when required by typing the 2 character alias. To run a script or command (doesn't have to be Bash-my-AWS) across all Regions I use region-each: $ region-each stacks | column -t example-ec2-ap-northeast-1 CREATE_COMPLETE 2011-05-23T15:47:44Z NEVER_UPDATED NOT_NESTED #ap-northeast-1 example-ec2-ap-northeast-2 CREATE_COMPLETE 2011-05-23T15:47:44Z NEVER_UPDATED NOT_NESTED #ap-northeast-2 ... example-ec2-us-west-2 CREATE_COMPLETE 2011-05-23T15:47:44Z NEVER_UPDATED NOT_NESTED #us-west-2 For AWS accounts, I type the name of the account and I'm in. For accounts using IDP (ldap/AD backed corporate logins) I generate aliases so I have tab completion and simple naming. In accounts that are only setup to use AWS keys, I use aliases that export credentials kept in GPG encrypted files. Last time I looked, AWS docs suggested keeping these long lives credentials in plaintext files readable by your account. That's asking for trouble IMO, especially if they're kept in a known location that a compromised node library could exfiltrate them from. AWSCLI v2 beta includes support for SSO so it's probably a good time to look at how BMA could include support for auth.
- deleted 7y ago[deleted]
- deleted 7y ago[deleted]