3 ms·
You can go a long way with a SQL database too. I'm writing up an article about getting the two billion words of Wikipedia into an inverted index in a SQLite da
by lsb 17y ago
You can go a long way with a SQL database too. I'm writing up an article about getting the two billion words of Wikipedia into an inverted index in a SQLite database entirely in memory, and that's another other order of magnitude bigger.
- amix 17y agoIf you only know how to use a hammer then everything else looks like a nail. In other words, re-implementing an inverted index in SQL is waste of time when you can use tools like Sphinx and Lucene - which are highly optimized to do inverted indexes and that can easily handle 2 billion words. The same can be said about FlockDB - it's possible to emulate a graph database, but is the effort really worth it when there are such tools like Redis and neo4j which seem to be optimized for graph like structures.