5 ms·

My first thought: great - we can retire our in-house multi-part uploader (EvaporateJS) [0] and use AWS's. But digging into the API, it seems like most of the re

My first thought: great - we can retire our in-house multi-part uploader (EvaporateJS) [0] and use AWS's. But digging into the API, it seems like most of the real work of multipart uploading (managing part failures) is not done by the SDK [1], so we'll stick with EvaporateJS. It's working pretty well for us. We've seen 22GB uploads go through (direct from browser to S3). The main issue that we know of [2] should be fairly easy to fix if anyone wants to contribute! [0] https://github.com/TTLabs/EvaporateJS https://github.com/TTLabs/EvaporateJS [1] http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/frames.html http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/frames.ht... [2] https://github.com/TTLabs/EvaporateJS/issues/6 https://github.com/TTLabs/EvaporateJS/issues/6

akoumjian 13y ago
We had similar hopes, but for now we will continue to use our in-house multi part uploader. We have to sign every multi-part request server side, using an auth scheme that depends on the key and the user. I'm not sure if it would be an improvement or not to give each of our users an IAM account that correlates to specific bucket acl which is based on user namespaces. It would lighten the load on our servers (don't need to sign each request) but now each user needs resources in IAM to be managed.
martingordon 13y ago
Thanks for the link to EvaporateJS. The recent price changes at Ink Filepicker make it untenable for our small needs and this looks like a great alternative.
sha90 13y ago
The SDK handles retries on failed requests. There is a little bit of extra legwork to actually do the slicing of a Blob, but you should be able to call createMultipartUpload(), uploadPart() x N, followed by completeMultipartUpload(). The SDK will make sure that each individual part gets retried on failures. That said, it would be nice to have a helper for that process.

akoumjian 13y ago

We had similar hopes, but for now we will continue to use our in-house multi part uploader. We have to sign every multi-part request server side, using an auth scheme that depends on the key and the user. I'm not sure if it would be an improvement or not to give each of our users an IAM account that correlates to specific bucket acl which is based on user namespaces. It would lighten the load on our servers (don't need to sign each request) but now each user needs resources in IAM to be managed.

martingordon 13y ago

Thanks for the link to EvaporateJS. The recent price changes at Ink Filepicker make it untenable for our small needs and this looks like a great alternative.

sha90 13y ago

The SDK handles retries on failed requests. There is a little bit of extra legwork to actually do the slicing of a Blob, but you should be able to call createMultipartUpload(), uploadPart() x N, followed by completeMultipartUpload(). The SDK will make sure that each individual part gets retried on failures. That said, it would be nice to have a helper for that process.