If you know the size ahead of time, you can use multipart uploads. Otherwise you would have to buffer to disk. you could consider kinesis firehose which had dum

If you know the size ahead of time, you can use multipart uploads. Otherwise you would have to buffer to disk. you could consider kinesis firehose which had dumping to s3 built in The google storage api has a mode where you can stream bytes, and it doesn’t become visible until you close it(and then can’t modify like s3). And unlike s3, it requires a delete permission to be able to overwrite, though with s3 you can turn on versioning and not give deleteobjectversion.