4 ms·
Uploading to S3 in 18 lines of Shell
- phireal 11y agoSearching GitHub for S3SECRET and S3KEY might get interesting if this script gets used a lot.
- chrismdp 11y agoYeah, I know :/ People have crawled github for these in the past and turned up some pretty "interesting" results...
- scrollaway 11y agoI find the wasted opportunity to use the S3CRET environment variable a shame.
- hijinks 11y agogithub now auto notifies AWS if it sees you publish secrets.. Then you get an email from AWS saying your keys are public and replace them asap
- toomuchtodo 11y agoThat's actually pretty awesome.
- estefan 11y agoaws s3 cp src dest... OK curl is installed by default but if you have to install this script anyway, it's only a `pip install` away...
- skywhopper 11y agoSome platforms and environments don't have pip handy, or you might not have root access, or you can't install new software for any number of reasons, or the required libraries are not installed. curl and /bin/sh and openssl are more universal, so it's good to be able to do what you need to do without resorting to installing software.
- chrismdp 11y agoExactly my thinking. Nice to have one thing less to install on a new Jenkins node, for example.
- mdellabitta 11y agoExcept this won't work with files bigger than 5gb. It might be worth the time to do it right.
- skywhopper 11y agoI don't think the author intended this script to solve everyone's problems. It's an interesting exercise and can be useful in some situations. If it's not useful for your needs, then as has been pointed out there are many many other options.
- toomuchtodo 11y ago> It might be worth the time to do it right. The right way is the most simple way for your use case. For files under 5GB, this is one of the right ways.
- chrismdp 11y agoI'll fix it when my game requires update files of >5GB :)
- insertnickname 11y agoYou don't need root to install Python packages with pip. You can install them just for your user. See https://pip.readthedocs.org/en/latest/user_guide.html#user-installs https://pip.readthedocs.org/en/latest/user_guide.html#user-i...
- skywhopper 11y ago... right, which is why I listed all the other reasons you might not be able to install a python package.
- falcolas 11y agoEven if the script itself is not "practical", it's very informative. It makes it quite simple to translate into another language for the times when you don't want to shell out to another process.
- sbochins 11y agoI found this script a couple weeks ago and was very useful for the minimal docker image I was creating. In some cases you have certain constraints when accessing s3 and this script is helpful in those cases.
- jaryd 11y agoImportant to note that this uses the public-read ACL by default.
- tonyhburns 11y agoHere's a couple I made as well for upload/download to S3 before Travis CI had Docker containers with support for caching in OSS projects: https://github.com/tabolario/ci-scripts https://github.com/tabolario/ci-scripts These use the V4 signature, which I grabbed from an article somewhere on the subject that I can't seem to find anymore :( The cached-bundle script is inspired and based off the one from devise: https://github.com/plataformatec/devise/blob/master/script/cached-bundle https://github.com/plataformatec/devise/blob/master/script/c...
- deleted 11y ago[deleted]
- mietek 11y agoIf you like this, you may find bashmenot useful. Among other things, it includes GNU bash functions to work with S3. Documentation: https://bashmenot.mietek.io/reference https://bashmenot.mietek.io/reference Source: https://github.com/mietek/bashmenot https://github.com/mietek/bashmenot
- blackace 11y agoThere's a 2007 bash implementation of s3 commands here too: https://code.google.com/p/s3-bash/ https://code.google.com/p/s3-bash/
- chrismdp 11y agoYeah, thanks for linking to this: the reason I didn't use it was mostly because it was longer than 18 lines :) Nice not to have to track and maintain a version of other people's code when you don't have to.