3 ms·
If there's anything I've learned about object storage over the last year, it is: don't use object storage unless you need it for a particularly narrow use case
by setq 9y ago
If there's anything I've learned about object storage over the last year, it is: don't use object storage unless you need it for a particularly narrow use case of silly data volumes or CDN storage for a massively distributed content network. Anything else, forget it.
It rarely if ever works properly with standard Linux or Windows tools (s3), it has a rat's nest of arbitrary restrictions which require a language lawyer to decypher (s3/iam/vpc/roles), the APIs are vendor specific and sometimes even region specific (s3), the APIs are obtuse (s3 multipart), the clients are buggy (boto/boto3), suddenly you inherit extra costs and configuration requirements if you want to do something like expose it over http (route53/cloudfront/s3), credential storage is a nightmare for distribution compared to rsync/ssh etc. Ugh.
Please note I have used Google Storage as well and all of the above also apply.
The only thing that is positive is capital expenditure is low.
- chris_wot 9y agoIs there a stand for object storage?
- setq 9y agoopenstack swift supposedly but this is a slightly incompatible S3 implementation that nothing else talks. The filesystem works pretty well though ;)
- 5ilv3r 9y agoSwift3 speaks aws4 auth now. The real problem is S3 clients that expect an amazon signed cert. No good way to fake that nicely.
- tedmiston 9y agoCeph / librados provides a compatible layer on top of S3 and Swift. http://ceph.com/ceph-storage/object-storage/ http://ceph.com/ceph-storage/object-storage/
- indogooner 9y agoI think AWS CLI is pretty handy for lot of tasks. Not sure if you are referring to which multipart API of s3[1]. There is a higher level API for quite some time now which is very convenient. Agree about documentation for iam/vpc/roles being arcane but tbh I have no idea how that could be simplified. [1] http://docs.aws.amazon.com/AmazonS3/latest/dev/usingHLmpuJava.html http://docs.aws.amazon.com/AmazonS3/latest/dev/usingHLmpuJav... Edit: Added link to high level API
- CobrastanJorji 9y ago> credential storage is a nightmare for distribution compared to rsync/ssh. I don't understand this argument. Why is it harder to store credentials for S3 or GCS than it is to store credentials for rsync or ssh?
- setq 9y agoBecause everything has no ownership, no standard place to put things and no logical consistency.
- CobrastanJorji 9y agoI think I'm even more confused now. If you're going to have an app ssh or rsync, it's going to have a password or a private key to use that will be associated with some account, right? And if you want to use cloud storage, you'll also need a credential or a private key to use that will be associated with some account, right? What's the difference?