2 ms·
Do you mean with the OS file cache disabled? No, I mean with a dataset which is too large to fit into the amount of RAM on the system. 1. What are, off top of
by cperciva 6y ago
Do you mean with the OS file cache disabled?
No, I mean with a dataset which is too large to fit into the amount of RAM on the system.
1. What are, off top of your head, some design changes or code changes required that'd bring drastic performance improvements?
Nothing immediately comes to mind. Profiling may reveal some improvements, of course.
2. What are some key internals that you think differentiate Kivaloo from other embedded KV stores? I assume you must have gone through a lot of existing literature on the topic before building this. For example, LMDB, BDB, RocksDB, LevelDB, SQLite and the likes come to mind that can double-up as KV stores.
Well... kivaloo isn't an embedded KV store, so that would be a big differentiating factor. It's a network daemon.
3. Does it store the database in flat files with a WAL in front? Is the file format of the database custom, or based on existing formats?
The "on-disk" format is the pages of a append-only B+Tree, with the last page being the tree root.
I put "on-disk" in scare quotes because there are other backends, e.g. using Amazon DynamoDB to store pages.
4. Does the database auto index the fields? Or, use any other such aids to speed up access to data?
There are no fields. Key-value pairs, nothing more.