3 ms·
S3 Files was launched today without support for atomic rename. This is not something you can bolt on. Can you imagine running Claude Code on your S3 Files and i
by jamesblonde 6mo ago
S3 Files was launched today without support for atomic rename. This is not something you can bolt on. Can you imagine running Claude Code on your S3 Files and it just wants to do a little house cleaning, renaming a directory and suddenly a full copy is needed for every file in that directory?
The hardest part in building a distributed filesystem is atomic rename. It's always rename. Scalable metadata file systems, like Collosus/Tectonic/ADLSv2/HopsFS, are either designed around how to make rename work at scale* or how work around it at higher levels in the stack.
* https://www.hopsworks.ai/post/scalable-metadata-the-new-breed-of-file-systems https://www.hopsworks.ai/post/scalable-metadata-the-new-bree...
- thomas_fa 6mo agoIndeed this is not an easy problem. And our s3-compatible system do support the atomic rename with extended protocol in a graceful way, see the demo with our tool [1]. [1] https://github.com/fractalbits-labs/fractalbits-main/tree/main/crates/fractal-s3#example-output https://github.com/fractalbits-labs/fractalbits-main/tree/ma...
- jamesblonde 6mo agoWe have advanced to building S3 stores with claude code - impressive: https://github.com/fractalbits-labs/fractalbits-main/graphs/contributors https://github.com/fractalbits-labs/fractalbits-main/graphs/...
- sb-gcs 6mo agoHierarchical Namespace buckets in Google Cloud Storage support folder operations, including atomic folder renames. https://docs.cloud.google.com/storage/docs/hns-overview#features https://docs.cloud.google.com/storage/docs/hns-overview#feat...