4 ms·
The problem I am describing is a situation in which you have and want to maintain high amounts of granular data that can be queried for specific ranges with var
by belluchan 13y ago
The problem I am describing is a situation in which you have and want to maintain high amounts of granular data that can be queried for specific ranges with varying amounts of detail.
For example look at something like Google Analytics. As simple as page views with timestamps. If you have high traffic data, you can be writing to your database quite a bit. You could bucket that data per second per unique viewer. Then in your YUI you can select to see a line graph with varying degrees of detail. Per hour, per day, per month, per year.
No way would I want to store this in an SQL database. The processing that happens later are the per time period rollups. Creating buckets of minutes, days, etc per user. This can be done when asked for and cached (slow experience for the user), or scheduled and persisted (expensive - consumes more space). Again I would not want to store this in SQL. All the while I maintain my raw data for further processing later.
These kinds of solutions are great for monitoring data, tracking, any kind of high volume, real time statistical data.
- Spearchucker 13y agoThe example you use is Google Analytics. I've no idea how big that dataset is, but I imagine bigger than anything I've encountered before. That said, I'd do everything you mention in a relational database. SELECT COUNT(fieldname) FROM {tablename} and all that. Are you ok with eventual consistency as a price for partition tolerance and availability? The data sizes I typically work with allow me to go with availability and consistency at the expense of partition tolerance. And that's ok because my data replicates, and my response time objectives and recovery point objectives are set to something appropriate. This is worth it to me if the benefit is an operational data store that I can query in real time without impacting writes. Still haven't heard of a scenario which would compel me to go a NoSQL only route (a scenario that isn't Google or Facebook or similar).