3 ms·
Years ago this was very similar to our use case, only much smaller file sizes, and Reiser saved an amazing amount of overhead. The systems were hosting map til
by linsomniac 5y ago
Years ago this was very similar to our use case, only much smaller file sizes, and Reiser saved an amazing amount of overhead.
The systems were hosting map tiles of the entire Earth, and a lot of them were solid blue (ocean) or solid beige (unoccupied land). Those files were something like 60-300 bytes, going from memory. Reiser was basically the only FS that would handle that reasonably.
- touisteur 5y agoOh I have a repository of tens of millions of small json files, and xfs chokes on that specific partition it's amazing. 5 minutes for a ls in the root folder (with only 2 folders). Tried moving it disk to disk in case I was on a broken disk, but no.
- karteum 5y agoUnless there is a good reason, I think in that situation I would probably try to avoid storing those tens of millions of small json files directly on the filesystem. For example : have you considered storing that data in jsonb/postgresql https://www.compose.com/articles/faster-operations-with-the-jsonb-data-type-in-postgresql https://www.compose.com/articles/faster-operations-with-the-... ? (or even just in an sqlite db https://www.sqlite.org/fasterthanfs.html https://www.sqlite.org/fasterthanfs.html )