3 ms·
It looks like it works but it's not reliable so your reliability concerns then get externalised into your application which multiplies complexity terribly.
by m0xte 7y ago
It looks like it works but it's not reliable so your reliability concerns then get externalised into your application which multiplies complexity terribly.
- nostrebored 7y agoWhat do you mean by it's not reliable?
- scarface74 7y agoI’ve never had the experience myself, but I would assume he means you would have to build in some type of retry logic in your script. Just from a cursory glance, I couldn’t find any samples of how to do a multipart upload with retries in Python with Boto3. This is an example of how to do multipart uploads though. https://medium.com/@niyazi_erd/aws-s3-multipart-upload-with-python-and-boto3-9d2a0ef9b085 https://medium.com/@niyazi_erd/aws-s3-multipart-upload-with-... Fun Trivia note: when you do a multipart upload, the S3 hash of the object is not the same as it is when you do a single part upload. I had a file with the same contents but a different hash when I used Python than when it was transferred with the CLI or CloudBerry. The quick and dirty way to fix the hash is to copy the file to itself with Boto3.