4 ms·
I think the concept of a file as a bag of bytes is a leaky abstraction. That model is fit within the context of file system implementations, which are concerned
by pilgrim0 3y ago
I think the concept of a file as a bag of bytes is a leaky abstraction. That model is fit within the context of file system implementations, which are concerned with allocation, but unfit for high level file usage, which is concerned with interpretation. Anyone who has written code for handling file uploads knows that identifying a file format happens on a best effort basis, and it’s never 100% guaranteed to work. Even attempting to check if a file is a “plain text” is not guaranteed to work at all times. File extensions cannot be relied upon, and trusting a magic number prefix is also not good, as it can be easily tampered. Hence the bag of bytes abstraction forces the development of context-specific conventions, thus exploding the complexity in the transactional information space. If we account for the Web and APIs, HTML and JSON are probably the most prevalent file formats. Imagine attempting to develop for such ecosystems using files without embedded metadata, e.g a separate payload for <head> and tag attributes — it doesn’t make any sense. My point is: metadata _is_ data, and I disagree completely with the author in regard to side-channels being “where metadata is supposed to be”. Perhaps for _file-system specific_ metadata, sure, but for file-content metadata, hell no. In the end, because of this abstraction, the trust level upon a file’s metadata after the file is transferred is practically zero. I think that consistently structured data _objects_ with embedded and verifiable cryptographic signatures would be a better fit for the interconnected world. The fact we rely so much on webpages and their associated internet domains as a means of structured context and provenance hints at the importance of objects and signatures.