4 ms·

Partial updates could be useful for certain kinds of (mostly binary) files, but block storage is going to handle that much better in general than object storage

Partial updates could be useful for certain kinds of (mostly binary) files, but block storage is going to handle that much better in general than object storage. The concurrency and consistency guarantees are quite different between object and block storage. Making partial updates atomic would be quite difficult in general in S3, though simple preconditions like compare-and-swap (which is sorely needed anyway) might be sufficient to make it possible for certain use cases.

sillysaurusx 2y ago
I think gcloud has CAS by comparing etags. Surprised to hear S3 can’t.
- nathants 2y ago
  the common approach is cas dynamodb pointers to uuid named objects in a named s3 prefix. would it be better to merge ddb and s3? maybe, maybe not. gcp/azure/etc are there to provide fancy cloud solutions. no need for aws to serve that market as well.
  - kbolino 2y ago
    One of the important premises of object storage is that if your PutObject or multipart upload succeed, the entire object is atomically replaced. It is eventually consistent, so you may not immediately retrieve the just-uploaded object with GetObject, but you should see the new version eventually, and never see part of one version mixed with part of another. This should natively support compare-and-swap: "hey if the existing etag is what I expect, apply my change, otherwise ignore my change and tell me so". This has nothing to do with DynamoDB and is not reimplementing its feature set. It is just a natural extension of how the service already works (from an API consumer perspective, not necessarily an implementation perspective).
skrtskrt 2y ago
The paradigm can be flipped now for these distributed storage systems. The blocks of a filesystem can now be objects, replicated or erasure-coded - like Ceph running filesystems on top of its low-level object storage protocol, which is done on raw disks, not a filesystem. This can't be done for something like Minio just running on your filesystem, but if you're building the storage system from the ground up it can. We will see more and more of these products appearing. Vast Data is another interesting one. Global deduplication and compression for your data, with an S3, block, or NFS interface. Storing differential backups for thousands or millions of VMs? You'll only store the data from base Ubuntu image once.
cduzz 2y ago
I was referring more to the list of hypervisor features not implemented: >> Because EC2's hypervisor was (when it was launched) lacking features (no hot swap, not shared block storage, no host movements, no online backup/clone, no live recovery, no Highly available host failover ) S3 had to step in to pick up some of the slack that proper block or file storage would have taken. I don't want any of that nonsense in my compute layer or an application (at scale) that relies on shared block storage or host movements or live recovery. I'm sure S3 append would be super handy.

sillysaurusx 2y ago

I think gcloud has CAS by comparing etags. Surprised to hear S3 can’t.

nathants 2y ago
the common approach is cas dynamodb pointers to uuid named objects in a named s3 prefix. would it be better to merge ddb and s3? maybe, maybe not. gcp/azure/etc are there to provide fancy cloud solutions. no need for aws to serve that market as well.
- kbolino 2y ago
  One of the important premises of object storage is that if your PutObject or multipart upload succeed, the entire object is atomically replaced. It is eventually consistent, so you may not immediately retrieve the just-uploaded object with GetObject, but you should see the new version eventually, and never see part of one version mixed with part of another. This should natively support compare-and-swap: "hey if the existing etag is what I expect, apply my change, otherwise ignore my change and tell me so". This has nothing to do with DynamoDB and is not reimplementing its feature set. It is just a natural extension of how the service already works (from an API consumer perspective, not necessarily an implementation perspective).

skrtskrt 2y ago

The paradigm can be flipped now for these distributed storage systems. The blocks of a filesystem can now be objects, replicated or erasure-coded - like Ceph running filesystems on top of its low-level object storage protocol, which is done on raw disks, not a filesystem. This can't be done for something like Minio just running on your filesystem, but if you're building the storage system from the ground up it can. We will see more and more of these products appearing. Vast Data is another interesting one. Global deduplication and compression for your data, with an S3, block, or NFS interface. Storing differential backups for thousands or millions of VMs? You'll only store the data from base Ubuntu image once.

cduzz 2y ago

I was referring more to the list of hypervisor features not implemented: >> Because EC2's hypervisor was (when it was launched) lacking features (no hot swap, not shared block storage, no host movements, no online backup/clone, no live recovery, no Highly available host failover ) S3 had to step in to pick up some of the slack that proper block or file storage would have taken. I don't want any of that nonsense in my compute layer or an application (at scale) that relies on shared block storage or host movements or live recovery. I'm sure S3 append would be super handy.