4 ms·

Still no built-in index on the fly for larger numbers of records? linear search doesn't fly with >5.000 records, it's not even sorted on the fly for logarithmic

• ttymck 4y ago
  Could you expand on what an "index on the fly" would look like? I imagine the engine could provision and maintain a separate index file, but the guarantees for keeping it in sync with the recfile would be tenuous. Interested in targeting something like this in my python implementation, since this is also my biggest concern with recutils.
  • rurban 4y ago
    on the fly is different to a cached index, just keep a hash table or b-tree in memory, with just offset values to the text. with range queries a tree would make sense, with lookups a hash table. the perl variant uses a hash. in some of my bigger text engines I prefer a serialized hash table (berkeley db) over real databases because it's 10x faster under 20.000 records. constant database overhead is huge
    • rurban 4y ago
      I've now looked at the code deeper, and whilst the records are only stored in a linked list, there is another mset (multiset) built on demand. So my concerns are void. Good architecture.