4 ms·
SQLite (and SQL in general) is ridiculously slow for specialized applications like this. http://www.lmdb.tech/bench/microbench/ http://www.lmdb.tech/bench/micro
by hyc_symas 8y ago
SQLite (and SQL in general) is ridiculously slow for specialized applications like this. http://www.lmdb.tech/bench/microbench/ http://www.lmdb.tech/bench/microbench/
- patelajay285 8y agoIt depends what you are optimizing for. It's a little slower for random disk reads, but it doesn't matter given we utilize an in-memory LRU cache and most text has a Zipf-ian distribution which will hit the cache very often even with a small cache size. (see my comment elsewhere on this thread) Developers are more familiar with SQLite, it's bundled with Python, it is easy to add to nearly every language, and it has specialized features we use like the Full Text Search module for out of vocabulary lookups.
- hyc_symas 8y agoWrappers for LMDB are available for nearly every language as well. https://symas.com/lmdb/technical/#wrappers https://symas.com/lmdb/technical/#wrappers