3 ms·

Posix is actually pretty terrible. It is funny how Linux people still defend it. My criticism. Inflexible. Poor ACL management. All this recursive bs that I st

Posix is actually pretty terrible. It is funny how Linux people still defend it. My criticism. Inflexible. Poor ACL management. All this recursive bs that I still can't wrap my head around. User groups, not a flat hierarchy? Poor observability. Windows is terrible in many ways but with files and in comparison to Posix Windows actually got it right. EDIT: I did not mention the article. It is silly to compare S3 to Posix. However. Azure and other cloud providers that did offer Posix storage have dropped those offerings.

ilyt 3y ago
The question is compared to what ? Yes, POSIX-like file access have plenty of rough edges but alternatives are mostly trading this for that, not straight up upgrade. I'd like to see deburred version (cutting all the edge case crap with fsyncing and such, among other problems), but good luck getting everyone on board of that Every feature comes with drawbacks. For example ability to just lock the file might be great from app perspective but it is PITA from any distributed backend perspective. Ability to change it after write comes with similar problems so S3 and other cloudy ones tend to skip that. ACLs will always be fucking mess because the more powerful ones you need the more messy access control becomes. And it always becomes married with whatever underlying system uses for authentication. Like, you can rewrite app to use S3 but suddenly you now can't rewrite header of the file without going full read-modify-write cycle because modifying an object would be hard at scale so Amazon decided to just not do it. So stream-encoding MP4 file directly to S3 (which at end of encoding requires writing some stuff in header) can't be done directly on S3 and you need a temporary file (guess in what? POSIX local filesystem) before the upload, or use different format. > Poor observability. What that has to do with anything ? It's just API, you can observe it as any other API... hell, it's actually far more observable than app using S3 as you can't strace that but you can strace filesystem operations.
- cuno 3y ago
  Hi, author here. Funny you should give the stream-encoding MP4 example, because yes that is what people's experience has been for S3. We've solved that - no temporary local file needed - all streamed directly to S3, for example using ordinary ffmpeg. The trick is a deeper understanding of how multi-part upload works, and if necessary, server-side copy semantics on those parts.
  - ilyt 3y ago
    So how does it work if say application would edit some metadata in existing file on existing object ?
    - cuno 3y ago
      Depends on what you mean by metadata. MP4 metadata is data inside the file - and is modified by server-side-copy semantics that replaces only the bits that are changed. If you mean POSIX metadata, we avoid storing that in the object, and for performance store that elsewhere (it's encoded and compressed in the actual filename of hidden files).

ilyt 3y ago

The question is compared to what ? Yes, POSIX-like file access have plenty of rough edges but alternatives are mostly trading this for that, not straight up upgrade. I'd like to see deburred version (cutting all the edge case crap with fsyncing and such, among other problems), but good luck getting everyone on board of that Every feature comes with drawbacks. For example ability to just lock the file might be great from app perspective but it is PITA from any distributed backend perspective. Ability to change it after write comes with similar problems so S3 and other cloudy ones tend to skip that. ACLs will always be fucking mess because the more powerful ones you need the more messy access control becomes. And it always becomes married with whatever underlying system uses for authentication. Like, you can rewrite app to use S3 but suddenly you now can't rewrite header of the file without going full read-modify-write cycle because modifying an object would be hard at scale so Amazon decided to just not do it. So stream-encoding MP4 file directly to S3 (which at end of encoding requires writing some stuff in header) can't be done directly on S3 and you need a temporary file (guess in what? POSIX local filesystem) before the upload, or use different format. > Poor observability. What that has to do with anything ? It's just API, you can observe it as any other API... hell, it's actually far more observable than app using S3 as you can't strace that but you can strace filesystem operations.

cuno 3y ago
Hi, author here. Funny you should give the stream-encoding MP4 example, because yes that is what people's experience has been for S3. We've solved that - no temporary local file needed - all streamed directly to S3, for example using ordinary ffmpeg. The trick is a deeper understanding of how multi-part upload works, and if necessary, server-side copy semantics on those parts.
- ilyt 3y ago
  So how does it work if say application would edit some metadata in existing file on existing object ?
  - cuno 3y ago
    Depends on what you mean by metadata. MP4 metadata is data inside the file - and is modified by server-side-copy semantics that replaces only the bits that are changed. If you mean POSIX metadata, we avoid storing that in the object, and for performance store that elsewhere (it's encoded and compressed in the actual filename of hidden files).