4 ms·
The tl;dr of this is -- yes. We have a durable caching layer that we use to stage writes before we asynchronously replicate them to S3. This means that we are a
by huntaub 2y ago
The tl;dr of this is -- yes. We have a durable caching layer that we use to stage writes before we asynchronously replicate them to S3. This means that we are able to quickly (<1ms) perform operations like single-byte updates and renames and provide strong read-after-write consistency to other file system clients.
Once the operation is stored in our durable cache, then we update your S3 bucket to match what the file system expects. This generally takes around a minute, but could take longer depending on the number of S3 operations a file operation translates to (for example, a directory rename requires that CopyObject each object in the directory in S3).
I think that the POSIX API is to here to stay (like the S3 API). I agree that it would be better to have timeouts and deadlines, but I don't think that those make it impossible to provide a good distributed file system experience on POSIX (look at Amazon's EFS, Oracle's FSS, Google's FileStore, etc). It just makes the bar for availability higher.