3 ms·

"I mean, there is a whole industry developed around databases, they must add value..." The traditional RDBMS adds random access by computed criteria, transacti

by chris_l 20y ago

"I mean, there is a whole industry developed around databases, they must add value..." The traditional RDBMS adds random access by computed criteria, transaction safety, multi-threadedness, a standard query language, ... If you don't need these, you're likely to suffer a performance hit / programming overhead compared to a system that does not offer them. A filesystem only offers random access by one key (pathname). Also, the more specialised your needs are, the further down the stack you should start your own code. For a simple web application with a bit of user data, I would probably go with a DB. For your own search engine, you probably want something a little more customized, building on top of a standard fs.

neilc 20y ago
Another reason to not use a DBMS for a search engine is that typical implementations of transaction-oriented SQL databases are a terrible fit for the performance requirements of a search engine. For example, search engines don't need concurrent writes or ACID transactions, or SQL-like query language; search engines want to optimize for large-scale updates, not small, random writes; typical DBMS index structures (btree) don't work well for search engine indices. Eric Brewer has an interesting paper that lays out an architecture for a search engine that is consistent with DBMS design principles, but differs significantly in the implementation details: http://www.cs.berkeley.edu/~brewer/papers/SearchDB.pdf http://www.cs.berkeley.edu/~brewer/papers/SearchDB.pdf