3 ms·

After teaching customers for years that S3 shouldn't be mounted as a filesystem because of its whole object-or-nothing semantics, and even offering a paid solut

After teaching customers for years that S3 shouldn't be mounted as a filesystem because of its whole object-or-nothing semantics, and even offering a paid solution named "storage gateway" to prevent issues between FS and S3 semantics, it's rather interesting they'd release a product like this. Amazon should really just fix the underlying issue of semantics by providing a PatchObjectPart API call that overwrites a particular multipart upload chunk with a new chunk uploaded from the client. CopyObjectPart+CompleteMultipartUpload still requires the client to issue CopyObjectPart calls for the entire object.

nostrebored 4y ago
Distributed patching becomes hell. You need transactional semantics and files are not laid out well to help you define invariants that should reject the transaction.
- supriyo-biswas 4y ago
  There is no reason why the descriptor of objects can’t be updated with a new value that has all of the old chunks and a new one, since S3 doesn’t do deduplication anyway the other chunks may be resized internally with an asynchronous process that gets rid of the excess data corresponding to the now overridden chunk.
wmf 4y ago
Eventually AWS always gives customers what they want even if it's a "bad idea".
- mlhpdx 4y ago
  Indeed.
- znpy 4y ago
  Bad ideas are very relative. Some bad ideas work extremely well if they fit your use case, you understand very well the tradeoffs and you’re building safeguards (disaster recovery). Some other companies try to convince (force?) you into a workflow or into a specific solution. Aws just gives you the tools and some guidance on how to use them best.
FridgeSeal 4y ago
> it's rather interesting they'd release a product like this Azure has a feature where you can mount a blob store storage container into a container/VM, is this possibly aiming to match that feature? I definitely think people should stop trying to pretend S3 is a file system and embrace what’s it’s good at instead, but I have had many times when having an easy and fast read-only view into an S3 bucket would be insanely useful.

nostrebored 4y ago

Distributed patching becomes hell. You need transactional semantics and files are not laid out well to help you define invariants that should reject the transaction.

supriyo-biswas 4y ago
There is no reason why the descriptor of objects can’t be updated with a new value that has all of the old chunks and a new one, since S3 doesn’t do deduplication anyway the other chunks may be resized internally with an asynchronous process that gets rid of the excess data corresponding to the now overridden chunk.

wmf 4y ago

Eventually AWS always gives customers what they want even if it's a "bad idea".

mlhpdx 4y ago
Indeed.
znpy 4y ago
Bad ideas are very relative. Some bad ideas work extremely well if they fit your use case, you understand very well the tradeoffs and you’re building safeguards (disaster recovery). Some other companies try to convince (force?) you into a workflow or into a specific solution. Aws just gives you the tools and some guidance on how to use them best.

FridgeSeal 4y ago

> it's rather interesting they'd release a product like this Azure has a feature where you can mount a blob store storage container into a container/VM, is this possibly aiming to match that feature? I definitely think people should stop trying to pretend S3 is a file system and embrace what’s it’s good at instead, but I have had many times when having an easy and fast read-only view into an S3 bucket would be insanely useful.