6 ms·

Show HN: Ssh2ec2 – Connect to EC2 instances via tags/metadata instead of hostname

36 points by mryan 11y ago

micah_chatt 11y ago
This reminds me of an Instagram tool I used to use, ec2-ssh https://github.com/facebookarchive/ec2-ssh https://github.com/facebookarchive/ec2-ssh My biggest headache with most tools (like the article) is that they don't easily support cross-account roles. This leaves me using a combination of the awscli and jq. https://blogs.aws.amazon.com/security/post/Tx70F69I9G8TYG/How-to-enable-cross-account-access-to-the-AWS-Management-Console https://blogs.aws.amazon.com/security/post/Tx70F69I9G8TYG/Ho...
- mryan 11y ago
  I don't make much use of cross-account roles so I haven't added this yet. If you're interested in making a pull request describing how this could be changed to fit in to your workflow, I can look at adding it. https://github.com/mikery/ssh2ec2 https://github.com/mikery/ssh2ec2
  - micah_chatt 11y ago
    Done!
    - mryan 11y ago
      Thanks! 0.6 now supports cross-account roles (available on pip). 0.7 will support MFA.
zimbatm 11y ago
Is there any tool that can do the same but with ansible inventories as a data source ? Bonus points if it opens all the sessions in tmux if there are multiple results.
- mikealbert 11y ago
  A colleague of mine wrote an ansible playbook that generates dynamic iTerm2 profiles from an AWS inventory https://github.com/davidhollenberger/ansible-iterm-profile https://github.com/davidhollenberger/ansible-iterm-profile
ak217 11y ago
Cool. I've been using a script to do something similar :) I recently published a related tool, keymaker: https://github.com/kislyuk/keymaker https://github.com/kislyuk/keymaker It lets you connect to EC2 instances using your IAM account (specifically, by uploading your public SSH key to your IAM user account).
tootie 11y ago
Wow. I was trying to write this exact tool 2 days ago. Thing that stuck me was how to pick the user name. There's no easy way from the API to figure out what the OS image name is and map it to a default user.
- mryan 11y ago
  I suppose you could store a mapping of AMI IDs to default user names in the script, but this would add a lot of overhead when it comes to creating and releasing new AMI versions. How about using tags to store the default user name for each AMI? Then you could modify this script to look up the username before connecting.
buremba 11y ago
I'm using ssh2 (https://github.com/soheil/ssh2 https://github.com/soheil/ssh2) to connect AWS instances and pretty happy with that. It might be a good alternative.
JustinAiken 11y ago
I have bad luck w/ pip - any homebrew cask?
- mryan 11y ago
  I'm afraid I've only packaged this for pip. All of the logic is in one script though - if you download this from the repo and rename it to ssh2ec2 it will work (as long as you have the Boto library installed).
- nikolay 11y ago
  Try this: brew install brew-pip brew pip ssh2ec2
grhmc 11y ago
This looks pretty good. I built and use something similar called ish: https://github.com/grahamc/ish https://github.com/grahamc/ish Seems like maybe we all just scratched our own itches :)
- mryan 11y ago
  Very cool. Based on the other links in the thread this does indeed appear to be a popular problem to solve.
nikolay 11y ago
Just like most similar tools, this lacks indices. For example, if you use ASGs (Auto Scaling Groups), all instances would have the same name tag, so, you need to be able to specify an index if there are multiple instances under the same name. Edit 1: Created an issue [0] already. Edit 2: Added a request [1] to integrate fwknop [2] for full automation and better security. [0]: https://github.com/mikery/ssh2ec2/issues/1 https://github.com/mikery/ssh2ec2/issues/1 [1]: https://github.com/mikery/ssh2ec2/issues/2 https://github.com/mikery/ssh2ec2/issues/2 [2]: http://www.cipherdyne.org/fwknop/ http://www.cipherdyne.org/fwknop/
- mryan 11y ago
  That's a feature, not a bug :-) I sometimes find myself needing to connect to any instance in a particular ASG, and I don't really care which one. This tool is built to solve that particular problem. If I want to access a specific instance I will address it by its instance ID (ssh2ec2 --instance-id i-123456). What would the index refer to? The list of instances in an ASG would need to be sorted (perhaps by instance ID) in order for an index to be useful. The behaviour would still be undefined to a certain extent, as the logic would be "return the instance at position n in the list", when the number of instances in the ASG might have changed due to a scaling event. Edit: Thanks for the pull requests - we can discuss further there.
  - nikolay 11y ago
    Why do I need an index? So that I can do something aginst all - one by one. Otherwise, I need to list, copy and paste, etc.
    - mryan 11y ago
      As you mentioned in the GH issue, Fabric would probably be a better tool - this isn't supposed to be a config management framework or anything in that direction. Having said that, you can do something on all matching instances with: ssh2ec2 --tag foo=bar --all-matching-instances /some/command
chatmasta 11y ago
Or you could just use google cloud and have nice functionality like this by default. :) We recently began migrating all our infrastructure from EC2 to Google Cloud, and my first impression is that google cloud is ORDERS OF MAGNTITUDE more usable than EC2, rapidly approaching major feature-set parity, cheaper, and seemingly more performant. Google has the advantage over Amazon of releasing its cloud product half a decade after Amazon, so Google could learn from the mistakes of EC2. Not to mention Google has run its cloud internally for 10+ years, and likely has more capable engineers working on it than Amazon does. After all, Google is a tech company first and foremost... Amazon started as a retailer. The preparedness of Google is clearly evident in the superiority of its cloud offerings. Google was able to avoid all the bad choices of EC2, and Google Cloud is not gimped by arcane dependencies and backwards compatibility requirements resulting from bad initial choices. A few things I like much better on Google: - Networks (equivalent of VPC) are first class citizens. If you launch two virtual machines in the same region, by default they are placed into the same network, and addressable by machine name. Routing is much easier and more intuitive to setup than in Amazon VPC. - The GUI is way, way, way nicer than the EC2 GUI. Honestly, it does not even compare. The EC2 console is a complete and utter mess, finding what you need is nearly impossible. The google console is plain white, basic HTML, and intuitively structured such that it's obvious where to look for what you need. - The CLI is also way better than the EC2 CLI. It's one comprehensive SDK with authentication and tab completion built into it from the beginning. There's no need to install multiple SDKs and libraries to interface with the "cloud" like there is with EC2 (aws-cli, boto, s3, are all inconsistently structured tools that you need to install and configure separately). Google is just one single SDK that works perfectly. - The EC2 documentation is complete shit, often outdated and incorrect, not to mention difficult to browse through (multiple pages on the same topic, all linking to each other). If you have a problem, you usually have to google it and find an EC2 forum thread with 50 people complaining to a support rep about the problem. There is rarely any acknowledgement of bugs or changes in the original documentation. In contrast, the google documentation is clean, follows sane best-practices, and is easily navigable and searchable. My one complaint about Google so far is that it's not immediately clear how to use native `ssh` and `scp` commands to talk to instances. Instead you need to go through the `gcloud` client. I'm sure there's a way around it, but for now the `gcloud` client has been sufficient.
eric_h 11y ago
Do people really have all the instances in their infrastructure serving ssh to the public internet? We use a gateway box that's the only machine in the cluster serving ssh to the public and jump to our other machines from there. Why would you build it any other way?
- mryan 11y ago
  It depends on the context. In some cases a bastion host makes sense. In others, a security group that limits SSH to a set of IPs is sufficient. I agree with the point of your comment - allowing SSH from the public internet is not a good idea. However, there is nothing about this script that assumes SSH is publicly available - you could even run this script on your bastion host. I'm also planning to add an option to use the private IP of the instance so it works seamlessly when you are on a VPN connection linking you to a VPC.
  - eric_h 11y ago
    Fair enough. I had an employer many years ago (~12) who insisted in spite of my objections that all of our servers be accessible via ssh (as root! On port 22!) with a shared, 7 character password on the public internet. Worst of all it was an (very small, working with other small businesses) IT consulting company, and my boss insisted on the same thing for client servers. Needless to say, a number of boxes, both our own and belonging to our clients) ended up as C&C machines for various botnets and I learned a very valuable lesson about how to treat a machine serving ssh to the public internet (namely: don't listen to a boss who doesn't know wtf they're talking about with regards to security, don't allow password ssh logins _ever_, for fuck's sake don't serve ssh to the public internet if you don't absolutely have to, and bloody hell denyhosts!). The headaches of dealing with pwned machines that were critical to the business in question while I was being paid 30k/year very much colored my opinion of how to handle ssh. Obviously I quit that job as soon as I had a credible alternative and was able to double my salary twice in two years (amazing what you can do once you know what you're actually worth on the market). I'm primarily a developer now, but do share some infrastructure management responsibility at the small (9 person) startup where I work now. Aws and opsworks has enabled us to run without a dedicated ops team, and I'm the most security paranoid person on the engineering team.
jperras 11y ago
Perhaps neither here nor there, but an often overlooked feature of boto (the official python AWS client) is that you can create an SSH client to an instance: http://boto.cloudhackers.com/en/latest/ref/manage.html#boto.manage.cmdshell.sshclient_from_instance http://boto.cloudhackers.com/en/latest/ref/manage.html#boto.... given that you have an EC2 `Instance` object, which can be hydrated in a variety of ways. Edit: Wow, a lot of you have built or found ssh-to-ec2 clients.
- mryan 11y ago
  Thanks for the link - I wasn't aware of that feature in boto. I think I still would have chosen to shell out to ssh as it gives me more control over SSH options (key forwarding etc.), although I see that boto is using paramiko under the hood so perhaps it would still be possible to get key forwarding and other features. Being able to go from an EC2 Instance object directly to an SSH session in pure Python will definitely come in handy in future. > Edit: Wow, a lot of you have built or found ssh-to-ec2 clients. Indeed, it seems to be a popular itch to scratch. At some point AWS will probably integrate something like this in awscli, thus rendering all of these 'ssh to ec2' scripts pointless :-)
brightball 11y ago
I've done something very similar to this using the Perl SSH::Batch module. That module uses a file called .fornodesrc to list and aggregate servers. I have a ruby script that uses the AWS SDK to create that file based on tag names, both individual and aggregate sets. SSH::Batch does the dirty work from there since it lets you to simple transformations like SETS - THESE, etc.
laurencerowe 11y ago
Another option is to run a DNS server that reurns results based on the ec2 name tag: https://github.com/ConradIrwin/aws-name-server https://github.com/ConradIrwin/aws-name-server

36 points by mryan 11y ago

This reminds me of an Instagram tool I used to use, ec2-ssh https://github.com/facebookarchive/ec2-ssh https://github.com/facebookarchive/ec2-ssh My biggest headache with most tools (like the article) is that they don't easily support cross-account roles. This leaves me using a combination of the awscli and jq. https://blogs.aws.amazon.com/security/post/Tx70F69I9G8TYG/How-to-enable-cross-account-access-to-the-AWS-Management-Console https://blogs.aws.amazon.com/security/post/Tx70F69I9G8TYG/Ho...

mryan 11y ago
I don't make much use of cross-account roles so I haven't added this yet. If you're interested in making a pull request describing how this could be changed to fit in to your workflow, I can look at adding it. https://github.com/mikery/ssh2ec2 https://github.com/mikery/ssh2ec2
- micah_chatt 11y ago
  Done!
  - mryan 11y ago
    Thanks! 0.6 now supports cross-account roles (available on pip). 0.7 will support MFA.

zimbatm 11y ago

Is there any tool that can do the same but with ansible inventories as a data source ? Bonus points if it opens all the sessions in tmux if there are multiple results.

mikealbert 11y ago
A colleague of mine wrote an ansible playbook that generates dynamic iTerm2 profiles from an AWS inventory https://github.com/davidhollenberger/ansible-iterm-profile https://github.com/davidhollenberger/ansible-iterm-profile

ak217 11y ago

Cool. I've been using a script to do something similar :) I recently published a related tool, keymaker: https://github.com/kislyuk/keymaker https://github.com/kislyuk/keymaker It lets you connect to EC2 instances using your IAM account (specifically, by uploading your public SSH key to your IAM user account).

tootie 11y ago

Wow. I was trying to write this exact tool 2 days ago. Thing that stuck me was how to pick the user name. There's no easy way from the API to figure out what the OS image name is and map it to a default user.

mryan 11y ago
I suppose you could store a mapping of AMI IDs to default user names in the script, but this would add a lot of overhead when it comes to creating and releasing new AMI versions. How about using tags to store the default user name for each AMI? Then you could modify this script to look up the username before connecting.

buremba 11y ago

I'm using ssh2 (https://github.com/soheil/ssh2 https://github.com/soheil/ssh2) to connect AWS instances and pretty happy with that. It might be a good alternative.

JustinAiken 11y ago

I have bad luck w/ pip - any homebrew cask?

mryan 11y ago
I'm afraid I've only packaged this for pip. All of the logic is in one script though - if you download this from the repo and rename it to ssh2ec2 it will work (as long as you have the Boto library installed).
nikolay 11y ago
Try this: brew install brew-pip brew pip ssh2ec2

grhmc 11y ago

This looks pretty good. I built and use something similar called ish: https://github.com/grahamc/ish https://github.com/grahamc/ish Seems like maybe we all just scratched our own itches :)

mryan 11y ago
Very cool. Based on the other links in the thread this does indeed appear to be a popular problem to solve.

nikolay 11y ago

Just like most similar tools, this lacks indices. For example, if you use ASGs (Auto Scaling Groups), all instances would have the same name tag, so, you need to be able to specify an index if there are multiple instances under the same name. Edit 1: Created an issue [0] already. Edit 2: Added a request [1] to integrate fwknop [2] for full automation and better security. [0]: https://github.com/mikery/ssh2ec2/issues/1 https://github.com/mikery/ssh2ec2/issues/1 [1]: https://github.com/mikery/ssh2ec2/issues/2 https://github.com/mikery/ssh2ec2/issues/2 [2]: http://www.cipherdyne.org/fwknop/ http://www.cipherdyne.org/fwknop/

mryan 11y ago
That's a feature, not a bug :-) I sometimes find myself needing to connect to any instance in a particular ASG, and I don't really care which one. This tool is built to solve that particular problem. If I want to access a specific instance I will address it by its instance ID (ssh2ec2 --instance-id i-123456). What would the index refer to? The list of instances in an ASG would need to be sorted (perhaps by instance ID) in order for an index to be useful. The behaviour would still be undefined to a certain extent, as the logic would be "return the instance at position n in the list", when the number of instances in the ASG might have changed due to a scaling event. Edit: Thanks for the pull requests - we can discuss further there.
- nikolay 11y ago
  Why do I need an index? So that I can do something aginst all - one by one. Otherwise, I need to list, copy and paste, etc.
  - mryan 11y ago
    As you mentioned in the GH issue, Fabric would probably be a better tool - this isn't supposed to be a config management framework or anything in that direction. Having said that, you can do something on all matching instances with: ssh2ec2 --tag foo=bar --all-matching-instances /some/command

chatmasta 11y ago

Or you could just use google cloud and have nice functionality like this by default. :) We recently began migrating all our infrastructure from EC2 to Google Cloud, and my first impression is that google cloud is ORDERS OF MAGNTITUDE more usable than EC2, rapidly approaching major feature-set parity, cheaper, and seemingly more performant. Google has the advantage over Amazon of releasing its cloud product half a decade after Amazon, so Google could learn from the mistakes of EC2. Not to mention Google has run its cloud internally for 10+ years, and likely has more capable engineers working on it than Amazon does. After all, Google is a tech company first and foremost... Amazon started as a retailer. The preparedness of Google is clearly evident in the superiority of its cloud offerings. Google was able to avoid all the bad choices of EC2, and Google Cloud is not gimped by arcane dependencies and backwards compatibility requirements resulting from bad initial choices. A few things I like much better on Google: - Networks (equivalent of VPC) are first class citizens. If you launch two virtual machines in the same region, by default they are placed into the same network, and addressable by machine name. Routing is much easier and more intuitive to setup than in Amazon VPC. - The GUI is way, way, way nicer than the EC2 GUI. Honestly, it does not even compare. The EC2 console is a complete and utter mess, finding what you need is nearly impossible. The google console is plain white, basic HTML, and intuitively structured such that it's obvious where to look for what you need. - The CLI is also way better than the EC2 CLI. It's one comprehensive SDK with authentication and tab completion built into it from the beginning. There's no need to install multiple SDKs and libraries to interface with the "cloud" like there is with EC2 (aws-cli, boto, s3, are all inconsistently structured tools that you need to install and configure separately). Google is just one single SDK that works perfectly. - The EC2 documentation is complete shit, often outdated and incorrect, not to mention difficult to browse through (multiple pages on the same topic, all linking to each other). If you have a problem, you usually have to google it and find an EC2 forum thread with 50 people complaining to a support rep about the problem. There is rarely any acknowledgement of bugs or changes in the original documentation. In contrast, the google documentation is clean, follows sane best-practices, and is easily navigable and searchable. My one complaint about Google so far is that it's not immediately clear how to use native `ssh` and `scp` commands to talk to instances. Instead you need to go through the `gcloud` client. I'm sure there's a way around it, but for now the `gcloud` client has been sufficient.

eric_h 11y ago

Do people really have all the instances in their infrastructure serving ssh to the public internet? We use a gateway box that's the only machine in the cluster serving ssh to the public and jump to our other machines from there. Why would you build it any other way?

mryan 11y ago
It depends on the context. In some cases a bastion host makes sense. In others, a security group that limits SSH to a set of IPs is sufficient. I agree with the point of your comment - allowing SSH from the public internet is not a good idea. However, there is nothing about this script that assumes SSH is publicly available - you could even run this script on your bastion host. I'm also planning to add an option to use the private IP of the instance so it works seamlessly when you are on a VPN connection linking you to a VPC.
- eric_h 11y ago
  Fair enough. I had an employer many years ago (~12) who insisted in spite of my objections that all of our servers be accessible via ssh (as root! On port 22!) with a shared, 7 character password on the public internet. Worst of all it was an (very small, working with other small businesses) IT consulting company, and my boss insisted on the same thing for client servers. Needless to say, a number of boxes, both our own and belonging to our clients) ended up as C&C machines for various botnets and I learned a very valuable lesson about how to treat a machine serving ssh to the public internet (namely: don't listen to a boss who doesn't know wtf they're talking about with regards to security, don't allow password ssh logins _ever_, for fuck's sake don't serve ssh to the public internet if you don't absolutely have to, and bloody hell denyhosts!). The headaches of dealing with pwned machines that were critical to the business in question while I was being paid 30k/year very much colored my opinion of how to handle ssh. Obviously I quit that job as soon as I had a credible alternative and was able to double my salary twice in two years (amazing what you can do once you know what you're actually worth on the market). I'm primarily a developer now, but do share some infrastructure management responsibility at the small (9 person) startup where I work now. Aws and opsworks has enabled us to run without a dedicated ops team, and I'm the most security paranoid person on the engineering team.

jperras 11y ago

Perhaps neither here nor there, but an often overlooked feature of boto (the official python AWS client) is that you can create an SSH client to an instance: http://boto.cloudhackers.com/en/latest/ref/manage.html#boto.manage.cmdshell.sshclient_from_instance http://boto.cloudhackers.com/en/latest/ref/manage.html#boto.... given that you have an EC2 `Instance` object, which can be hydrated in a variety of ways. Edit: Wow, a lot of you have built or found ssh-to-ec2 clients.

mryan 11y ago
Thanks for the link - I wasn't aware of that feature in boto. I think I still would have chosen to shell out to ssh as it gives me more control over SSH options (key forwarding etc.), although I see that boto is using paramiko under the hood so perhaps it would still be possible to get key forwarding and other features. Being able to go from an EC2 Instance object directly to an SSH session in pure Python will definitely come in handy in future. > Edit: Wow, a lot of you have built or found ssh-to-ec2 clients. Indeed, it seems to be a popular itch to scratch. At some point AWS will probably integrate something like this in awscli, thus rendering all of these 'ssh to ec2' scripts pointless :-)

brightball 11y ago

I've done something very similar to this using the Perl SSH::Batch module. That module uses a file called .fornodesrc to list and aggregate servers. I have a ruby script that uses the AWS SDK to create that file based on tag names, both individual and aggregate sets. SSH::Batch does the dirty work from there since it lets you to simple transformations like SETS - THESE, etc.

laurencerowe 11y ago

Another option is to run a DNS server that reurns results based on the ec2 name tag: https://github.com/ConradIrwin/aws-name-server https://github.com/ConradIrwin/aws-name-server