3 ms·
> I once saw a project in the wild where the "database" was implemented using filesystem directories as "tables" with JSON files inside as "rows". I did this s
by philipwhiuk 2y ago
> I once saw a project in the wild where the "database" was implemented using filesystem directories as "tables" with JSON files inside as "rows".
I did this sort of thing recently. I felt bad doing it, I still objectively hate it, because I do know enough to know that basically I'm re-implementing what years of hardworking O/S developers have done, piecemeal. But at least I'm going in with my eyes open which feels better.
The only real mitigating factor I have is that the application is largely 'never-read' and then when reading is done, it's sequential batches. Which is not normally something databases optimise for and works okay for file-storage.
(If someone does know a lightweight database architecture that performs like this, let me know).
- sahil-kang 2y agofwiw, I normally think of LMDB when the idea of using the fs instead of a db crosses my mind
- djhn 2y agoWhy not sqlite? put the json in a single column, maybe copy some parts of it or metadata to another two or three. Should be faster than the filesystem for reading multiple rows.
- lpapez 2y agoWhy not start with SQLite? There are benchmarks out there proving that for some use cases (i.e. many small updates) where using SQLite is faster than using the filesystem. [1] So not only do you get all of the benefits of a relational database, and literally centuries of engineering hours and bugfixes invested into SQLite, you might also get better performance (which is why I presume you even considered rolling your own). [1] https://www.sqlite.org/fasterthanfs.html https://www.sqlite.org/fasterthanfs.html