3 ms·
There are ways to get around the latency you're seeing without (or while) moving away from Mongo. If you rewrite your queries as mapreduce operations, then you
by giaour 12y ago
There are ways to get around the latency you're seeing without (or while) moving away from Mongo. If you rewrite your queries as mapreduce operations, then you can run them at a scheduled interval to update a reports table with the previous interval's data. This is called incremental mapreduce and Mongo has a great guide on how to get started: http://docs.mongodb.org/manual/tutorial/perform-incremental-map-reduce/ http://docs.mongodb.org/manual/tutorial/perform-incremental-...
You won't get real time data any more, but switching to batch processing will speed up your users' experience.
The cynic in me, however, wants to just tell you that you've outgrown Mongo and should move on. It happens; not every database is designed to support every need, and Mongo happens to be a spectacularly shitty fit for the kind of OLAP scenario you're describing.
If you absolutely must have real-time analytics crunched on the fly, I recommend that you look at data stores that are a better fit for your application. InfluxDB is great for time series data. Aerospike is getting a lot of press in the AdTech industry. And old standbys like MySQL or Postgres can handle queries over a million records without breaking a sweat.
- deleted 12y ago[deleted]
- giaour 12y agoWould it be possible to write the data you run reports on to both Mongo and something that can handle queries a bit better? If you need to display something like pageviews or widgets sold you can write to Mongo and then also write the minimal amount of data you need for the report to something like InfluxDB.