It depends what you are optimizing for. It's a little slower for random disk reads, but it doesn't matter given we utilize an in-memory LRU cache and most text

It depends what you are optimizing for. It's a little slower for random disk reads, but it doesn't matter given we utilize an in-memory LRU cache and most text has a Zipf-ian distribution which will hit the cache very often even with a small cache size. (see my comment elsewhere on this thread) Developers are more familiar with SQLite, it's bundled with Python, it is easy to add to nearly every language, and it has specialized features we use like the Full Text Search module for out of vocabulary lookups.