3 ms·
Arguably, UploadToS3 should not be a method of *Payload. Suggestion: Make an S3-uploader package with _internal_ connection pooling, upload queueing and concur
by thwd 11y ago
Arguably, UploadToS3 should not be a method of *Payload.
Suggestion: Make an S3-uploader package with _internal_ connection pooling, upload queueing and concurrency handling.
- zimbatm 11y agoThis. Each S3-uploader can then hold a http.Client instance that keeps the connection open to S3 between uploads.
- placeybordeaux 11y agoI haven't used this library, but this appears to be what you want. https://github.com/rlmcpherson/s3gof3r https://github.com/rlmcpherson/s3gof3r Nice features include: > Retry Everything: All http requests and every part is retried on both uploads and downloads. > Configurable conncurrency > Uses an io.Writer (you could actually start posting to S3 before all of the data gets in on your side.) etc.