12 ms·

The New AWS Command-Line Interface

265 points by jeffbarr 13y ago

simonebrunozzi 13y ago
Well done!
sivers 13y ago
FWIW, I've been using the aws command-line tool by Tim Kay for over 6 years. It's only Perl and cURL. A single self-contained script with no other dependencies. It's been rock solid in production that whole time: http://timkay.com/aws/ http://timkay.com/aws/ The syntax looks quite the same, but Amazon's awscli Python installer has loads of dependencies. I'll have to see if it's worth switching. Anyone already know if Amazon's new CLI thing has any big advantages over Tim Kay's Perl aws?
- jaytaylor 13y ago
  Thanks for sharing, I'd never heard of this tool. Indeed, an official supported friendly and easy-to-use CLI interface is long overdue.
  - wuster 13y ago
    actually, ec2-api-tools has been provided for years: http://aws.amazon.com/developertools/351 http://aws.amazon.com/developertools/351 - I just didn't like the command interface provided by them. Haven't had time to check out the new CLI yet but it looks much better at quick glance.
    - vacri 13y ago
      Unfortunately it doesn't support tab-completion... which would be nice given that there's oodles of ec2 commands.
      - jeffbarr 13y ago
        The new AWS CLI supports tab completion. You need to configure it using the info found at http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-get...
- gtaylor 13y ago
  > The syntax looks quite the same, but Amazon's awscli Python installer has loads of dependencies. I'll have to see if it's worth switching. Why are the dependencies a problem? By combining a handful of smaller, focused modules that each do something well, you can end up with something better than if you were to re-invent the wheel for every need. AWS and the Python dev team are doing a heck of a job on botocore, and have cranked up the pace of improvement in the last 6 months. This CLI reaching "official" status will guarantee (at least until further notice) that it will see updates and fixes. It's likely to see early or earlier support for new AWS services.
  - spullara 13y ago
    The issue is that they can't release them all compiled into one thing. Instead the dependencies have to pollute the rest of the operating system. This is one of those things that Java got right and virtually everything else except npm (local node_modules) got wrong out of the gate. I'd love to have a simple way to package python apps that depend on other python and native libraries without having to install things separately.
    - cmccabe 13y ago
      If you don't want the pip dependencies to "pollute the rest of the system," you can just use pip install with the --root option. Java's CLASSPATH causes enormous pain for end-users. Just read the Hadoop mailing list. The fact that Java doesn't have a sane default for where to put anything or how to manage dependencies is a huge flaw. Back when I was doing AWS, I just used the C binaries (I forget what they were called) to transfer things to or from S3. I just wanted to avoid installing hundreds of megs of dependencies. We paid money to transfer our AMIs around, after all! Still, a more full-featured tool will no doubt come in handy in some scenarios.
      - spullara 13y ago
        If you are using CLASSPATH for Java you are doing it wrong. Full dependency jars with every class needed is really the only way to do this reliably. That requires only a single file to be passed around. You can even include native libraries in it. Hadoop is a nightmare, I agree. They are doing it wrong.
mafro 13y ago
This latest release marks a milestone in the transition from the old Java based tools, to the new Python ones. Mitch Garnaat[1] who built and maintained boto over the years was picked up by Amazon last year and since has been building out botocore[2] - which the aws-cli[3] tools use under the hood. [1] https://github.com/garnaat https://github.com/garnaat [2] https://github.com/boto/botocore https://github.com/boto/botocore [3] https://github.com/aws/aws-cli https://github.com/aws/aws-cli
- jevinskie 13y ago
  Thank you for this no-nonsense layout of the situation. =) Can you elaborate more on how this FOSS Github-er was picked up by Amazon? It sounds like a story that many here would like to replicate!
  - mafro 13y ago
    I'm afraid I don't actually know anything more than the intertubes can tell you :) I noticed it had happened sometime last year, and thought I'd share what little I know.
unono 13y ago
About time. More startups should realize that they could increase developer adoption of their products if also published shell script interfaces to their product. In fact, your startup should really start off as command line accessible and add the gui after.
- rrrhys 13y ago
  "start off as command line accessible and add the gui after" As much as I'd love this to be true, it is so wrong unless you are targeting other very technical users. Most people I know (ie, potential customers) think anything in the command line is old/impossible to use/nerdy.
  - thex86 13y ago
    Right, in this case it actually makes sense because you will already be technically proficient if you are using something like AWS. But in other cases, the GUI can be the main interface, but a CLI to back it up is pretty awesome and I had happily pay some provider extra bucks for it.
    - jeffbarr 13y ago
      I'm with you on this. As an "old" command line person (I first learned Unix in 1980 or so), I still tend to think of the GUI as an add-on.
- jeffbarr 13y ago
  There have been CLI tools for some parts of AWS since the beginning. This release replaces a set of disparate tools (each with their own installation instructions and configuration issues) with a unified set. > In fact, your startup should really start off as command line accessible... I would argue that startups should always start at the API level and work upwards. At least that's what we did with AWS.
  - MichaelGG 13y ago
    Starting off from the API seems like the only sensible way to approach things. It exposes core design issues early on, and if done right, eliminates a wide source of security issues that might be introduced by, say, the front-end developers.
kondro 13y ago
Whilst this is a much better solution that the existing slow tools Amazon provided, I don't understand why this is being reported as if command-line tools for AWS are a new thing. Command-line tools used to be your only option for managing AWS and Amazon always create their API and shell tools before the Console.
- dmourati 13y ago
  Yea, not a new thing, even the command names are mostly the same. $ ec2-describe-instances $ aws ec2 describe-instances … Pretty strange decision IMO.
  - vacri 13y ago
    It's unifying the commands. Just this morning I've been struggling with ascli (autoscaling cli), which has its own method for listing AWS keys that don't match the way other tools do it.
    - dmourati 13y ago
      Meh, 90% of what I do is ec2 anyway. I did have to break out: cli53 rrcreate.
- benologist 13y ago
  It's probably 'news' because AWS is at risk of losing mindshare on HN as more competitors target this community - Rackspace just started giving everyone free credit, Linode gave everyone big free upgrades, DigitalOcean are just killing it, and then there's Docker etc threatening to reduce them all to interchangeable dumb infrastructure.
  - tekacs 13y ago
    Given the number of major, established services running on AWS that go down perfectly in sync every time there's an AWS outage, it seems unlikely that the service is likely to fall out of most of HN's mindset for quite some time to come. ;) Might lose some favour for new projects, though.
zeckalpha 13y ago
This makes a project I was almost done with much easier. Start over, or finish with the way I was previously doing it?
deleted 13y ago
[deleted]
727374 13y ago
The new CLI looks like a massive improvement over the old one, but ironically I will probably still prefer boto + ipython because of the very robust autocompletion.
- xorgar831 13y ago
  I wonder if the fish shell could autocomplete the commands; it has a feature to generate autocompleation settings from man pages.
akurilin 13y ago
I've been using s3cmd for a while now. Am I correctly seeing that the new CLI tools, or at least the s3 portion, are pretty much a replacement for that?
- crazysim 13y ago
  First-party parallel upload and splitting support immediately trumps s3cmd for me.
throwaway9848 13y ago
Okay.. glad to see the effort, but I've been using boto and euca2ools for years. This will have to be good for people to switch.
- nailer 13y ago
  Same author. boto dude (garn something, he's posting here but I'm on my phone) wrote boto and wrote these.
  - throwaway9848 13y ago
    Thanks!
vacri 13y ago
Interesting - I must have had an older version and not played with it. The version I had did not have 's3 sync', and the help files were a long list of ec2 commands and some boilerplate headings. Reinstalled and now they have shorter, saner commands and real help pages... and they also changed the pager from 'less' (my default pager) to the crappy 'more'... because (apparently) you never want to go back a page in a help file. More detailed content, but it's harder to review. Odd.
- jevinskie 13y ago
  Amazon doesn't use $PAGER? [0] [0]: https://en.wikibooks.org/wiki/Guide_to_Unix/Environment_Variables#PAGER https://en.wikibooks.org/wiki/Guide_to_Unix/Environment_Vari...
  - jessaustin 13y ago
    Well, GP isn't just imagining things. The pager for this is at most the equivalent of crap-ass "more" right out of the box, on systems where all other cli executables have better pagers. $ PAGER=/usr/bin/less aws help ...does seem to do the right thing, but why does this program require it when no other program does?
pkj 13y ago
Just tried it out and it is excellent ! The integration of various aws services is really neat. The help and correction-suggestion are nice too. Great to hear that this is built on the awesome boto library. Will serve as an useful reference for boto developers.
djrobstep 13y ago
If anybody is after similar s3 functionality within python, I wrote a small python wrapper around boto a while back that does similar parallel/multipart upload to s3, and a bunch of other AWS stuff: https://github.com/djrobstep/motorboto https://github.com/djrobstep/motorboto
snagage 13y ago
I think the only thing "new" here is aws-cli hitting version 1.0.0. It's been the recommended aws command line tool for quite a while now. I've personally used it for the last 6 months.
sparkzilla 13y ago
As a semi-technical small business user of AWS I really wonder why so few resources seem to be put to the AWS GUI. For example, why is autoscaling not part of the GUI/AWS console? It's one of the most basic functions and it beggars belief that in this day and age it is done through the command line only. I'd rather not have to pay extra for Ylastic or some other hack when I'm sure Amazon can throw together even the most basic UI in a few weeks.
- deleted 13y ago
  [deleted]
- imperialWicket 13y ago
  Use Asgard from NetflixOSS (https://github.com/Netflix/asgard https://github.com/Netflix/asgard). Shameless project plug: for anyone using Hubot, easily get Asgard up and running, then interface with Asgard via Hubot (https://github.com/imperialwicket/hubot-asgard https://github.com/imperialwicket/hubot-asgard). * EDIT: that is, use Asgard because it does a great job of managing autoscaling groups.
dencold 13y ago
Long-time user of boto[1] here. It has been the go to library to hook your python code into AWS and has a fairly active following on github[2]. One API point that I've found lacking in boto is a "sync" command for S3. Take a source directory and a target bucket and push up the differences ala rsync, that's the dream. Boto gives you a the ability to push/get S3 resources, but I've had to write my own sync logic. So, the first thing I went digging into is the S3 interface of the new CLI, and to my surprise, they've put a direct sync command on the interface[3], huzzah! Their implementation is a little wacky though. Instead of using some computed hashes, they are relying on a combination of file modtimes and filesize. Weird. Anyways, glad to see AWS is investing in a consistent interface to make managing their services easier. [1] http://boto.readthedocs.org/en/latest/index.html http://boto.readthedocs.org/en/latest/index.html [2] https://github.com/boto/boto https://github.com/boto/boto [3] http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html#examples http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html...
- simonebrunozzi 13y ago
  Good to hear this feedback. I work for AWS; I will pass this to the team. Feel free to shoot me an email: simone attt amazon do0tcom if you have more comments. Thanks!
- HeyImAlex 13y ago
  s3cmd has a sync option that will do this. If you're looking for something a little more robust, I just released this a couple of days ago- https://github.com/HeyImAlex/s3tup https://github.com/HeyImAlex/s3tup It's still in a really early stage but I've been using it to sync and configure my personal site that's hosted on s3 and it's worked well so far.
- garnaat 13y ago
  The rsync command uses a combination of file modtimes and file sizes as it's default algorithm. It's very fast and efficient. I agree, though, that like rsync, it would be good to add a --checksum option to the s3 sync command in AWS CLI. Feel free to create an issue on our github site https://github.com/aws/aws-cli https://github.com/aws/aws-cli so we can track that.
hendry 13y ago
aws-cli/0.16.0 gripes: • slow • crap ton of dependencies, making it a pain to install on Arch https://aur.archlinux.org/packages/aws-cli/ https://aur.archlinux.org/packages/aws-cli/ Hint: static build ffs • broken help. `aws s3 help` instead of `aws help s3` which is more natural to git users • no glacier support • no s3 progress for downloads or uploads • many jarring UX issues https://github.com/aws/aws-cli/issues/305 https://github.com/aws/aws-cli/issues/305 https://github.com/aws/aws-cli/issues/304 https://github.com/aws/aws-cli/issues/304
- rlpb 13y ago
  > • no glacier support I'm the author of glacier-cli (github.com/basak/glacier-cli). I'd be happy to see it move into aws-cli. If anyone wants to do this, please get in touch to coordinate.
- kolev 13y ago
  No CloudFront support either!
oblio 13y ago
Well, a rant: * Why does each one of them use different parameters for the same stuff? WHATEVER_URL could be REGION (WHATEVER = EC2, ELB, ETC, ...). One uses a config file for ACCESS_KEY_ID, another one wants a environment variable. Plus they use different names for common stuff. * Why do are the command line arguments named inconsistently across these tools? --whatever, --what-ever * Why don't they fail on command error? Right now this only happens if there's a configuration problem - if you send a command and there's a problem (like S3 access denied), it still returns 0. * Why don't they provide synchronous commands? Right now I have to do the polling myself. Super annoying. Anyway, I've been using the ones included in Amazon Linux - I hope they were the latest version. If the new version fixes this problem, feel free to correct me :)
js-py-pl 13y ago
Interesting, I've been using CloudSigma's built-in tool for quite some time. You just hit ' in their webapp :) See https://autodetect.cloudsigma.com/ui/#/trynow https://autodetect.cloudsigma.com/ui/#/trynow
craigsmitham 13y ago
Azure has had a node.js CLI for awhile and it's awesome. It's advertised for Mac/Linux, but I use it on Windows all the time. http://www.windowsazure.com/en-us/develop/nodejs/how-to-guides/command-line-tools/ http://www.windowsazure.com/en-us/develop/nodejs/how-to-guid...
deleted 13y ago
[deleted]
ausjke 13y ago
Can I use this for Glacier parallel uploading?
kolev 13y ago
I've been using a combination of aws-cli and the Java-based tools (as some features are unsupported by aws-cli yet). I've been patiently waiting for CloudFront support - no luck yet! Also, I'm switching my new projects to use botocore, but it lacks any documentation and it's been really painful!

265 points by jeffbarr 13y ago

Well done!

FWIW, I've been using the aws command-line tool by Tim Kay for over 6 years. It's only Perl and cURL. A single self-contained script with no other dependencies. It's been rock solid in production that whole time: http://timkay.com/aws/ http://timkay.com/aws/ The syntax looks quite the same, but Amazon's awscli Python installer has loads of dependencies. I'll have to see if it's worth switching. Anyone already know if Amazon's new CLI thing has any big advantages over Tim Kay's Perl aws?

jaytaylor 13y ago
Thanks for sharing, I'd never heard of this tool. Indeed, an official supported friendly and easy-to-use CLI interface is long overdue.
- wuster 13y ago
  actually, ec2-api-tools has been provided for years: http://aws.amazon.com/developertools/351 http://aws.amazon.com/developertools/351 - I just didn't like the command interface provided by them. Haven't had time to check out the new CLI yet but it looks much better at quick glance.
  - vacri 13y ago
    Unfortunately it doesn't support tab-completion... which would be nice given that there's oodles of ec2 commands.
    - jeffbarr 13y ago
      The new AWS CLI supports tab completion. You need to configure it using the info found at http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-get...
gtaylor 13y ago
> The syntax looks quite the same, but Amazon's awscli Python installer has loads of dependencies. I'll have to see if it's worth switching. Why are the dependencies a problem? By combining a handful of smaller, focused modules that each do something well, you can end up with something better than if you were to re-invent the wheel for every need. AWS and the Python dev team are doing a heck of a job on botocore, and have cranked up the pace of improvement in the last 6 months. This CLI reaching "official" status will guarantee (at least until further notice) that it will see updates and fixes. It's likely to see early or earlier support for new AWS services.
- spullara 13y ago
  The issue is that they can't release them all compiled into one thing. Instead the dependencies have to pollute the rest of the operating system. This is one of those things that Java got right and virtually everything else except npm (local node_modules) got wrong out of the gate. I'd love to have a simple way to package python apps that depend on other python and native libraries without having to install things separately.
  - cmccabe 13y ago
    If you don't want the pip dependencies to "pollute the rest of the system," you can just use pip install with the --root option. Java's CLASSPATH causes enormous pain for end-users. Just read the Hadoop mailing list. The fact that Java doesn't have a sane default for where to put anything or how to manage dependencies is a huge flaw. Back when I was doing AWS, I just used the C binaries (I forget what they were called) to transfer things to or from S3. I just wanted to avoid installing hundreds of megs of dependencies. We paid money to transfer our AMIs around, after all! Still, a more full-featured tool will no doubt come in handy in some scenarios.
    - spullara 13y ago
      If you are using CLASSPATH for Java you are doing it wrong. Full dependency jars with every class needed is really the only way to do this reliably. That requires only a single file to be passed around. You can even include native libraries in it. Hadoop is a nightmare, I agree. They are doing it wrong.

mafro 13y ago

This latest release marks a milestone in the transition from the old Java based tools, to the new Python ones. Mitch Garnaat[1] who built and maintained boto over the years was picked up by Amazon last year and since has been building out botocore[2] - which the aws-cli[3] tools use under the hood. [1] https://github.com/garnaat https://github.com/garnaat [2] https://github.com/boto/botocore https://github.com/boto/botocore [3] https://github.com/aws/aws-cli https://github.com/aws/aws-cli

jevinskie 13y ago
Thank you for this no-nonsense layout of the situation. =) Can you elaborate more on how this FOSS Github-er was picked up by Amazon? It sounds like a story that many here would like to replicate!
- mafro 13y ago
  I'm afraid I don't actually know anything more than the intertubes can tell you :) I noticed it had happened sometime last year, and thought I'd share what little I know.

unono 13y ago

About time. More startups should realize that they could increase developer adoption of their products if also published shell script interfaces to their product. In fact, your startup should really start off as command line accessible and add the gui after.

rrrhys 13y ago
"start off as command line accessible and add the gui after" As much as I'd love this to be true, it is so wrong unless you are targeting other very technical users. Most people I know (ie, potential customers) think anything in the command line is old/impossible to use/nerdy.
- thex86 13y ago
  Right, in this case it actually makes sense because you will already be technically proficient if you are using something like AWS. But in other cases, the GUI can be the main interface, but a CLI to back it up is pretty awesome and I had happily pay some provider extra bucks for it.
  - jeffbarr 13y ago
    I'm with you on this. As an "old" command line person (I first learned Unix in 1980 or so), I still tend to think of the GUI as an add-on.
jeffbarr 13y ago
There have been CLI tools for some parts of AWS since the beginning. This release replaces a set of disparate tools (each with their own installation instructions and configuration issues) with a unified set. > In fact, your startup should really start off as command line accessible... I would argue that startups should always start at the API level and work upwards. At least that's what we did with AWS.
- MichaelGG 13y ago
  Starting off from the API seems like the only sensible way to approach things. It exposes core design issues early on, and if done right, eliminates a wide source of security issues that might be introduced by, say, the front-end developers.

kondro 13y ago

Whilst this is a much better solution that the existing slow tools Amazon provided, I don't understand why this is being reported as if command-line tools for AWS are a new thing. Command-line tools used to be your only option for managing AWS and Amazon always create their API and shell tools before the Console.

dmourati 13y ago
Yea, not a new thing, even the command names are mostly the same. $ ec2-describe-instances $ aws ec2 describe-instances … Pretty strange decision IMO.
- vacri 13y ago
  It's unifying the commands. Just this morning I've been struggling with ascli (autoscaling cli), which has its own method for listing AWS keys that don't match the way other tools do it.
  - dmourati 13y ago
    Meh, 90% of what I do is ec2 anyway. I did have to break out: cli53 rrcreate.
benologist 13y ago
It's probably 'news' because AWS is at risk of losing mindshare on HN as more competitors target this community - Rackspace just started giving everyone free credit, Linode gave everyone big free upgrades, DigitalOcean are just killing it, and then there's Docker etc threatening to reduce them all to interchangeable dumb infrastructure.
- tekacs 13y ago
  Given the number of major, established services running on AWS that go down perfectly in sync every time there's an AWS outage, it seems unlikely that the service is likely to fall out of most of HN's mindset for quite some time to come. ;) Might lose some favour for new projects, though.

zeckalpha 13y ago

This makes a project I was almost done with much easier. Start over, or finish with the way I was previously doing it?

deleted 13y ago

[deleted]

727374 13y ago

The new CLI looks like a massive improvement over the old one, but ironically I will probably still prefer boto + ipython because of the very robust autocompletion.

xorgar831 13y ago
I wonder if the fish shell could autocomplete the commands; it has a feature to generate autocompleation settings from man pages.

akurilin 13y ago

I've been using s3cmd for a while now. Am I correctly seeing that the new CLI tools, or at least the s3 portion, are pretty much a replacement for that?

crazysim 13y ago
First-party parallel upload and splitting support immediately trumps s3cmd for me.

throwaway9848 13y ago

Okay.. glad to see the effort, but I've been using boto and euca2ools for years. This will have to be good for people to switch.

nailer 13y ago
Same author. boto dude (garn something, he's posting here but I'm on my phone) wrote boto and wrote these.
- throwaway9848 13y ago
  Thanks!

vacri 13y ago

Interesting - I must have had an older version and not played with it. The version I had did not have 's3 sync', and the help files were a long list of ec2 commands and some boilerplate headings. Reinstalled and now they have shorter, saner commands and real help pages... and they also changed the pager from 'less' (my default pager) to the crappy 'more'... because (apparently) you never want to go back a page in a help file. More detailed content, but it's harder to review. Odd.

jevinskie 13y ago
Amazon doesn't use $PAGER? [0] [0]: https://en.wikibooks.org/wiki/Guide_to_Unix/Environment_Variables#PAGER https://en.wikibooks.org/wiki/Guide_to_Unix/Environment_Vari...
- jessaustin 13y ago
  Well, GP isn't just imagining things. The pager for this is at most the equivalent of crap-ass "more" right out of the box, on systems where all other cli executables have better pagers. $ PAGER=/usr/bin/less aws help ...does seem to do the right thing, but why does this program require it when no other program does?

pkj 13y ago

Just tried it out and it is excellent ! The integration of various aws services is really neat. The help and correction-suggestion are nice too. Great to hear that this is built on the awesome boto library. Will serve as an useful reference for boto developers.

djrobstep 13y ago

If anybody is after similar s3 functionality within python, I wrote a small python wrapper around boto a while back that does similar parallel/multipart upload to s3, and a bunch of other AWS stuff: https://github.com/djrobstep/motorboto https://github.com/djrobstep/motorboto

snagage 13y ago

I think the only thing "new" here is aws-cli hitting version 1.0.0. It's been the recommended aws command line tool for quite a while now. I've personally used it for the last 6 months.

sparkzilla 13y ago

As a semi-technical small business user of AWS I really wonder why so few resources seem to be put to the AWS GUI. For example, why is autoscaling not part of the GUI/AWS console? It's one of the most basic functions and it beggars belief that in this day and age it is done through the command line only. I'd rather not have to pay extra for Ylastic or some other hack when I'm sure Amazon can throw together even the most basic UI in a few weeks.

deleted 13y ago
[deleted]
imperialWicket 13y ago
Use Asgard from NetflixOSS (https://github.com/Netflix/asgard https://github.com/Netflix/asgard). Shameless project plug: for anyone using Hubot, easily get Asgard up and running, then interface with Asgard via Hubot (https://github.com/imperialwicket/hubot-asgard https://github.com/imperialwicket/hubot-asgard). * EDIT: that is, use Asgard because it does a great job of managing autoscaling groups.

dencold 13y ago

Long-time user of boto[1] here. It has been the go to library to hook your python code into AWS and has a fairly active following on github[2]. One API point that I've found lacking in boto is a "sync" command for S3. Take a source directory and a target bucket and push up the differences ala rsync, that's the dream. Boto gives you a the ability to push/get S3 resources, but I've had to write my own sync logic. So, the first thing I went digging into is the S3 interface of the new CLI, and to my surprise, they've put a direct sync command on the interface[3], huzzah! Their implementation is a little wacky though. Instead of using some computed hashes, they are relying on a combination of file modtimes and filesize. Weird. Anyways, glad to see AWS is investing in a consistent interface to make managing their services easier. [1] http://boto.readthedocs.org/en/latest/index.html http://boto.readthedocs.org/en/latest/index.html [2] https://github.com/boto/boto https://github.com/boto/boto [3] http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html#examples http://docs.aws.amazon.com/cli/latest/reference/s3/sync.html...

simonebrunozzi 13y ago
Good to hear this feedback. I work for AWS; I will pass this to the team. Feel free to shoot me an email: simone attt amazon do0tcom if you have more comments. Thanks!
HeyImAlex 13y ago
s3cmd has a sync option that will do this. If you're looking for something a little more robust, I just released this a couple of days ago- https://github.com/HeyImAlex/s3tup https://github.com/HeyImAlex/s3tup It's still in a really early stage but I've been using it to sync and configure my personal site that's hosted on s3 and it's worked well so far.
garnaat 13y ago
The rsync command uses a combination of file modtimes and file sizes as it's default algorithm. It's very fast and efficient. I agree, though, that like rsync, it would be good to add a --checksum option to the s3 sync command in AWS CLI. Feel free to create an issue on our github site https://github.com/aws/aws-cli https://github.com/aws/aws-cli so we can track that.

hendry 13y ago

aws-cli/0.16.0 gripes: • slow • crap ton of dependencies, making it a pain to install on Arch https://aur.archlinux.org/packages/aws-cli/ https://aur.archlinux.org/packages/aws-cli/ Hint: static build ffs • broken help. `aws s3 help` instead of `aws help s3` which is more natural to git users • no glacier support • no s3 progress for downloads or uploads • many jarring UX issues https://github.com/aws/aws-cli/issues/305 https://github.com/aws/aws-cli/issues/305 https://github.com/aws/aws-cli/issues/304 https://github.com/aws/aws-cli/issues/304

rlpb 13y ago
> • no glacier support I'm the author of glacier-cli (github.com/basak/glacier-cli). I'd be happy to see it move into aws-cli. If anyone wants to do this, please get in touch to coordinate.
kolev 13y ago
No CloudFront support either!

oblio 13y ago

Well, a rant: * Why does each one of them use different parameters for the same stuff? WHATEVER_URL could be REGION (WHATEVER = EC2, ELB, ETC, ...). One uses a config file for ACCESS_KEY_ID, another one wants a environment variable. Plus they use different names for common stuff. * Why do are the command line arguments named inconsistently across these tools? --whatever, --what-ever * Why don't they fail on command error? Right now this only happens if there's a configuration problem - if you send a command and there's a problem (like S3 access denied), it still returns 0. * Why don't they provide synchronous commands? Right now I have to do the polling myself. Super annoying. Anyway, I've been using the ones included in Amazon Linux - I hope they were the latest version. If the new version fixes this problem, feel free to correct me :)

js-py-pl 13y ago

Interesting, I've been using CloudSigma's built-in tool for quite some time. You just hit ' in their webapp :) See https://autodetect.cloudsigma.com/ui/#/trynow https://autodetect.cloudsigma.com/ui/#/trynow

craigsmitham 13y ago

Azure has had a node.js CLI for awhile and it's awesome. It's advertised for Mac/Linux, but I use it on Windows all the time. http://www.windowsazure.com/en-us/develop/nodejs/how-to-guides/command-line-tools/ http://www.windowsazure.com/en-us/develop/nodejs/how-to-guid...

deleted 13y ago

[deleted]

ausjke 13y ago

Can I use this for Glacier parallel uploading?

kolev 13y ago

I've been using a combination of aws-cli and the Java-based tools (as some features are unsupported by aws-cli yet). I've been patiently waiting for CloudFront support - no luck yet! Also, I'm switching my new projects to use botocore, but it lacks any documentation and it's been really painful!