3 ms·
Could you expand on what an "index on the fly" would look like? I imagine the engine could provision and maintain a separate index file, but the guarantees for
by ttymck 4y ago
Could you expand on what an "index on the fly" would look like? I imagine the engine could provision and maintain a separate index file, but the guarantees for keeping it in sync with the recfile would be tenuous. Interested in targeting something like this in my python implementation, since this is also my biggest concern with recutils.
- rurban 4y agoon the fly is different to a cached index, just keep a hash table or b-tree in memory, with just offset values to the text. with range queries a tree would make sense, with lookups a hash table. the perl variant uses a hash. in some of my bigger text engines I prefer a serialized hash table (berkeley db) over real databases because it's 10x faster under 20.000 records. constant database overhead is huge
- rurban 4y agoI've now looked at the code deeper, and whilst the records are only stored in a linked list, there is another mset (multiset) built on demand. So my concerns are void. Good architecture.