6 ms·
Think of Lightroom or automation over files. Many semi professionals from wedding photographers to designers want some form of batch automation and organization
by batmansmk 2mo ago
Think of Lightroom or automation over files. Many semi professionals from wedding photographers to designers want some form of batch automation and organization system over their files.
Several megabytes/gigabytes assets and you want to extract metadata, a preview, running as a batch some filter/compression/, conversion to CMYK, text injection ... fast partial read/write access would be nice.
Right now, most reads are performed through indexes because those files are slow to read.
If we take 10k sqlite files and want to retrieve a row, we would be around 3s on SSD, maintaining preemptive indexes become less important for a lot of use cases.
Change management and versioning also becomes quite efficient - sqlite can be configured to not offset bytes, so CVS like Epic Lore can efficiently delta the files and store minimal delta, or the file format itself can keep its edit history. Oh and it's 3x-10x less bytes without compressing the whole thing, so pages are stable through time.
About needing SQLite client, it's real but it's roughly the same size as an XML parser.
- lentil_soup 2mo ago> About needing SQLite client, it's real but it's roughly the same size as an XML parser. what I like about it being XML is I can just open it with any text tool and inspect it. It's human readable so I can edit and debug it manually, no need to have a parser or an extra application just to see what's in my file
- simonask 2mo agoThat's such an unbelievably niche use case for an image file format, though. Extremely the wrong thing to optimize for.
- lentil_soup 1mo agofor the end user, potentially niche, yes. I would argue it allows for anyone to hack around with the file which is a positive in my book for development though, I'd say it's quite useful. I work in a 3D editor and having data in a binary blob is a pain as you need to use your custom serialiser if you want to inspect the data. If a file is corrupted, good luck figuring out what was there, what happened, etc