4 ms·
That sounds pretty interesting (in a sick way ;) ) Are you able to elaborate a bit more what you were doing?
by spicybright 5y ago
That sounds pretty interesting (in a sick way ;) ) Are you able to elaborate a bit more what you were doing?
- marginalia_nu 5y agoI'm using it for the lexicon in my search engine. I basically need a mapping from arbitrary short strings to unique integer identifiers, with a cardinality in the billions. Too much to keep in memory, most db solutions keel over as well.
- rurban 5y agofor this I would use CHM, ordered minimal perfect hashes with external mmap'ed keys. still constant lookup, just the position needs to be stored in the value also.
- marginalia_nu 5y agoProblem is I don't know the items beforehand, instead I need to build the hash as I encounter new pages. I don't know any way of reconcile that with a minimal perfect hash.
- rurban 5y agook, with trillions keys creating a CHM MPH is risky, and needs a few secs. then your external hash is good enough.