3 ms·
100M? Of course you'd scale an RDBMS for that, especially if you want searchability and analytics. It's way easier than a Hadoop -> Elasticsearch pipeline (or p
by adenverd 10y ago
100M? Of course you'd scale an RDBMS for that, especially if you want searchability and analytics. It's way easier than a Hadoop -> Elasticsearch pipeline (or pick your flavor).
NoSQL databases are for BIG data. As in, billions of rows big.
- giaosudau 10y agototally agree with that. 100M rows doesn't make any sense.
- tracker1 10y agoI think it depends on the shape of your data... if your data is mostly collected in sets (as a single object base), and mostly key lookups, then a document store may be the best solution... Example, used to work for a mid sized classifieds site... most of the data was used as a single-listing query, and pulled in from a single base record. The SQL database was over-normalized and required a couple dozen joins if you wanted to see it flat... the system was crumbling... Replicating the data to mongodb at the time, with a thin API for search and read queries, and omg, it was a staggering difference. Beyond just caching, all the search queries. Today, I'd be more inclined to use ElasticSearch (there was an issue with geo indexing at the time iirc)... just the same, it really depends on the shape of your data. I feel that the storage shape should serve the needs of the application. SQL databases encourage normalization to such a degree, that it's costly to construct a semi-complex object in memory, especially when your queries will run across many tables for many thousands of users. Joins kill performance at scale... If you can get around that, you're often better off. Duplicating data to a system that is a better fit for mostly-read/query scenarios is a good idea. There's nothing that says you can't have your data in two places, and it's easy enough to setup services that copy on update.