3 ms·
> Append workloads are common in distributed systems. Bringing back the topic to what the parent was saying; since S3 is a pretty common system, and a distribu
by capableweb 3y ago
> Append workloads are common in distributed systems.
Bringing back the topic to what the parent was saying; since S3 is a pretty common system, and a distributed system at that, are you saying that S3 does support appending data? AFAIK, S3 never supported any append operations.
- ozfive 3y agoIt does not. Consider using services like Amazon Kinesis Firehose, which can buffer and batch logs, then periodically write them to S3.
- vlovich123 3y agoI’ll add some nuance here. You can implement append yourself in a clunk way. Create a new multipart upload for the file, copy its existing contents, create a new part appending what you need, and then complete the upload. Not as elegant / fast as GCS’s and there may be other subtleties, but it’s possible to simulate.